I mean, to each their own.
You've got so many years on this planet. There's a cliche about how no one is on their death bed wishing they'd worked more.
If you're working an extra 5-10 years for "gravy," just realize you're sacrificing years of your life for money you may never spend. Worse, they're almost definitely the best years you have left (youth and health).
As far as your "bad year," the 4% rule includes two World Wars, Vietnam, the Great Depression, stagflation of the 70's, the dotcom bust...all of it. Now, there are caveats like how it's only a 30 year study (the Trinity study), and the "success" condition is you don't completely run out of money. But it was also super rigid, where in bad years you just keep plugging away with withdrawals without any adjustment. It also doesn't factor in other sources of income that you pick up, nor how you can mitigate sequence of return risk (essentially if you can manage your portfolio from black swan events in the first decade, you're good indefinitely). It also ignores stuff like Social Security (which unless it's completely slashed as a program, will pay out a minimum of what the cash inflows from current workers is; generally state as about 75% of current rates), inheritances, pensions, or even how people spend less in their later years.
There are calculators I can share that are really good to get a visual of chances of failure. One of my favorites shows how much more likely you are to just die, and how that's a much bigger risk than going broke. They also show the chances of you ending up with multiple times more than you started with. For instance, I just looked it up and was like a 20% chance we end up with 5x more than we started with, and 30% chance 2x (so 50% chance we end up with more money than we know what to do with).
A FI/RE early retirement calculator visualizing longevity risk in early retirement and enables you to compare the probability success, failure and mortality.
engaging-data.com
Attaching a picture of it. The red is the "broke" part people worry about, meanwhile the massive grey one is "dead."