Thanks for all the input guys. To follow this up, I went to another drainage I know well a few miles away the next day. There were 8 or 9 does and one spike in there. After an hour I noticed another deer cruising out of a timber pocket 300 yards from me, 24-25" wide 4x4 with a dropper. The guy I was with got to 80 yards from him, but couldn't quite get an arrow in him. He was just cruising the ridge looking for does and the group from the day before just didn't have a buck (that I could see) in their vicinity.
I have a camera in the canyon where that big buck was in, looking forward to seeing him come by my salt so I can see him up close!