It's like this, if a farmer shot every white faced cow on a ranch year after year, the white face gene will be eliminated.
Except that you're forgetting two things here:
1) Half of the genetics for antlers comes from the does, and is not visibly expressed
and
2) Genetics doesn't have a whole lot of influence as to whether a deer is ever a 2 point. Age and nutrition will play a much larger role in how many points a deer has.
You're simply seeing a larger population in that age class because they are protected.