it takes a severly underloaded slow powder to get into trouble, anything close to the book ought to be way safe.
WRONG!!!!!!!!!It depends on the powder. I don't know what you mean by 'slow' powder, as it's all relative.
One of my favorite powders, Hodgdon H110 (used for hot handgun magnum loads) is one example of powder that is sensitive to light loadings. Hodgdon recommends AGAINST light loads with this powder as even minor reductions in powder can cause dangerous high pressures.
To the original question about fillers, some people (myself included) use a small amount of dacron fiber as filler to keep the powder against the base of the case. Some loads with especially fast burning powders pushing all lead bullets require only 2-3 grains of powder or so. Bullets don't seat THAT deep. Powder burn rates vary based on the amount of powder surface area exposed when ignition occurs.
A filler can be added to gain reliable ignition and consistent performance.
My opinion based on my experiences.