I'd focus on rifle more than anything else. Remember - practice makes permanent, not perfect. If you're not able to keep 1-2" groups at 100 yards, maybe get some training/lessons. If you're at a local range, some of the older RSOs might be able to give you some good tips.
I carry a handgun anywhere that has browns/grizzlies. I will carry one if I'm bowhunting as well, depending on the area. Generally speaking though, they're a lot of extra weight to carry and not a lot of value.
Your rifle, a good pair of boots and decent binoculars will go a lot further for you than a handgun. Oh, and tires and plenty of gas money.
...And Swedish fish
