Bullytrout put a trail cam up at that box and see what happens and yes Fisher will eat them no problem
After having a couple chewed by rodents I’ve been setting them so when they pull the trap out they swing out and hang in the air , I don’t think that would help with a Fisher but I think it would solve another marten or weasel and rodents from doing damage! Any pics of your sets ?