I had someone using my ground blind this year too, but they didn't leave any garbage. They left 2 out of the 3 windows wide open, and the door in the back was halfway unzipped. I was done using it for the year when I found it like that, so it was no big deal, I just took it down.
If I were you I'd be looking for another location, hopefully a little more secluded where they won't find it.