It’s never a waste of time to elk hunt. If you aren’t going to elk hunt and just focus on deer then you have to weigh the disappointment of having a chance encounter without a tag or the disappointment of spending $400 or so and not even trying to use it.
If I wanted to deer hunt I would Not buy an elk tag, because if I had an elk tag I would start elk hunting and would forget about the deer.