I've seen photos of proper loading of lead ball, although I don't use them. In fact, the lead ball did slight have lines imprinted in it, through the patch, from the rifling. Upon ignition, the lead ball expands slightly, giving it even more grab on the rifling. The patch just keeps it from smearing lead into the rifling. A harder material would not form to the rifling when loading and shooting, and would not be as accurate. Of course a harder material would not expand well on impact either. Yes, it would be more expensive, I guess.
As far as the attitude that non-toxic rounds are not necessary, I've seen youtube videos, and read articles that demonstrate lead projectiles, both modern and black-powder, fragment and splatter lead into a carcass, far from the wound channel. In fact, samples of ground game meat tested positive for lead contamination about half the time (that's not a scientific reference; just something I read). Lead is bad for kids, they say, and some probably sensative adults also.
I also have a rifle with a slower twist rate that designed to shoot lead conicals. While I'm not gonna run out and sell it, my next rifle purchase will have a fast twist so I can shoot Barnes XTPs. I tried them in my rifle, and they tumble badly. Check it out!