Our ape ancestors had a diet high in fibrous and starch foods, mainly roots, leaves, tubers and the like. Their physiology accounted for this fact. They had extremely long intestines (ever notice how so many large primates have those large extended guts? Example:
) which helped to break down that fiber and pull the nutrients from it. They also had massive jaws, thick, dense flat teeth, small brain sizes and often large bony protrusions from their skulls which anchored the large mastoid muscles required to chew these heavy, uncooked fibrous plants. Take a look at the gorilla skull in this image, and compare it to that of the homo habilis and the homo erectus, sapiens, and neanderthalensis, to get an idea of what I mean by all this
Notice how their face is very large, their brain case very small, there's a ridge of bone on top of the cranium which is used as an anchor for the mastoid muscles, the massive mandible and maxilla, and the large, thick flat teeth. Meanwhile all the others are sort of reversed. The face is smaller, the brain case is large, the mandible and maxilla are gracile, the teeth are small, rounded, cusps more and sharper overall.
So we can actually see these changes through comparative anatomy and also in the fossil record.
Gradually, as Africa dried out and became more arid, we moved out of the trees and we began to eat more meat as a result (mostly what we did was scavange the remains of carcasses other animals had left behind). This is where anthropologists believe basic tool making began. If you could shape a stone into a sharp blade, you could cut pieces of meat off carcasses, carry it back to your shelter to eat in safety, etc. This is also when we began to become totally bipedal. Those hominin ancestors who could stand up straighter in the savanna could see further, and therefore could detect predators at a further distance, increasing their rate of survival fractionally over those who could not.
Anyway, this new and increased influx of fatty acids and amino acids, over long stretches of time, like a million years, gradually increased our brain size (our brains increased by as much as 3x over some of our ancestors during this time frame!). As our brain size increased, we learned how to better process our foods by learning how to capture, control and create fire and to cook our foods, which led to better nutrition. Those hominin ancestors with better nutrition obviously had a much stronger selective advantage than those who could not.
All things in physiology come with a trade off. As brain size increased, and access to cooked and highly nutritious foods increased, our physiology began to alter to compensate. Our teeth, jaws, intestinal tracts, and mastoid muscles began to shrink, as we they were no longer required to process these heavy starch and fiber based food stuffs which were the staple foods of our earliest ancestors, which helps offset the physiological changes in the cranium which resulted from larger brain size. So brain case size goes up, the bones of the face, mandible and maxilla, etc. shrank to compensate.
Then along came agriculture, where I talked above about how our skeletal structure began to become much more gracile (thinner, lighter, though still strong), this is reflected in the continued shrinkage of the mandible, hence why we slowly evolving such that our wisdom teeth are becoming vestigial and eventually will probably disappear completely or nearly so. You'd still have the occassional person with a vestigial set of wisdom teeth, just as people are occassionally born with tiny vestigial tails, but these would be considered anomalies at that point.
As for sexual selection in regards to this issue, we could hypothesize several explanations, such as people are unconsciously selecting for larger brain capacity or there is a sexual selection advantage in terms of attractiveness of facial features in individuals who lack wisdom teeth versus those that don't, etc.
But really, its just more about the fact that, as our nutrition increased, our craniums and other anatomically relevant system altered to compensate. That linear trajectory continues today, the alleles which guide development of cranial features are already present in the population, and already pretty well dominate in it, so they're just getting shuffled and reshuffled around and the progression continues. Mutations which go in the other direction are selected against in part because they are now anomalous and the extremely large population drowns out their impact and drives them further towards extinction, so forth.
