I've been in this position before archery hunting mule deer. I had a nice buck aggressively rubbing a tree 20 yards from me, but I needed to move a little to get an angle. I was so scared I might bump him that I froze hoping he'd give me a shot when he was done. He did not. In hindsight, I believe that if they are occupied rubbing a tree, you probably have a little leeway to make a move and try to get a line of sight on them.