The only thing I can see that is largely missing out of the modern gaming sphere is this: The willingness to let the player fail. Old games had this in spades, to the point where it was the primary source of content for some games. Few modern titles are willing to push the player very hard, without tons of advanced warning. The few games that do decide that killing the player with no remorse is perfectly acceptable are not all instant classics. But the ones that allow it AND get it right, are typically hailed as masterpieces.
It is a very thin line to walk and takes an extremely thorough knowledge of what you are asking the player to do. But it offers the most satisfying experiences in gaming when the effort to fine tune the game play right up to what is possible, is put forth.
I went to the DigX conference in London, ON last year, and had a very interesting conversation with Mike Laidlaw of Bioware about exactly this. He was giving a talk on "Freedom vs Narrative" and the concept of agency in video games. He actually did a really solid breakdown of the problems with Dragon Age 2 (and the comparative success of the new XCOM). When he took questions at the end, I asked him what ever happened to the "freedom to fail" and have it actually affect the game world. I cited things like Ultima III (attack the king and the guards punt you out of town and never, EVER let you back in) and Final Fantasy III (failing to save Shadow and ending up with -1 character).
He really, REALLY liked this question. He's a writer turned designer, so story means a lot to him, and being hamstrung into "your player can never really lose anything" really hinders story. He spent 20-25 minutes answering it.
The short version of what he said is this: A writer can go pitch a story that could involve something like this. We'll use losing access to a city because you were a bad boy as an example. So his team lead goes to the team lead in charge of building that city. The response is roughly "What do you mean you want some players to not get to explore this city?! We just spent 3 million dollars and 500 man-hours on it!" And that's that.
Basically, it has to do with scaling development costs. They just can't see the value in it. Not when your average gamer will just throw a fit that "my friend saw something cool but it wasn't like that in my game!" The "correct" solution is to have divergent story, not culled story. Getting ousted by the guards should give you diplomatic options, rogueish options, criminal connections, that sort of thing. Obviously they didn't have the time, experience, or technology to do this with Ultima III, but even today they still don't have the time (or arguably the technology).
He DID have good news in the long run though. People who know more about the industry than I do have been saying that players are desiring more and more to have gaming be an activity they can talk about with other people in any setting (water cooler talk). For this to be any good, it means that the developers have to let go a little bit and allow for players to actually have different experiences, so that they'll have something to talk about.
