The reason you seem to be having more accuracy with flu flus is because the excessive drag on the back of the arrow is over correcting flaws in your form. Either that or there is some major mismatch in the tune of bow and arrows.
If you have perfect form and perfectly tuned equipment a bare shaft arrow with no fletching at all should impact within an inch of fletched arrows at 20 -30 yards. Can you tell me the type of bow, draw weight, your draw length, type of arrow and arrow length?