A little wiggle is normal, but a lot of people, myself included, find it annoying.
Put a small O-ring around the front lug of the upper; it acts as a spring to keep the joint tight. They are cheap, keep a couple extras for spares because they wear out sooner or later.