It sounds like you already have this all figured out, but just to echo others above, with a full-house elk load (>100gr powder, >240gr bullet), it shouldn't feel like a gentle push. Ka-BOOM is what it sounds like from a distance, but I just hear a loud crack when I'm shooting. You should feel like the recipient of the bullet is going to get hurt.
As an aside, a cheap 240gr bullet with a single pellet lit up by a musket cap is a great plinking/practice load. It's really nice not to have to measure and pour powder when you're just sending some lead downrange. Less charge also means easier swabbing between shots.