The point he was making was that if you have $5m, you're not flying a private jet. No matter how well your portfolio does, you'll never be the super rich. Yeah, it will provide you with more comfort, but you're still only living marginally better.
I hope this doesn't sound gloating, but when you've got as much money as we do, money isn't really a concern anymore. If I had an extra $5 million, what would I do with it? Not a whole lot different. Like I said, I'd probably build a nicer house in a nicer area. But it's not going to be a $10 million mansion on Hawaii. I'm not going to fly a private jet. I'd fly first class probably (assuming I fly, which I refuse to, but that's a different issue). I'd drive a slightly better car. I'd probably build a drum studio. Are any of these things life changing from where I'm at now, as someone who retired at 39? Absolutely not.