Imagine taking a giant rubber band, then holding it like a bow and pulling it back.
Where does it go? Closer to your body because as you pull to full draw you get closer to that 180* angle. Work on your form, decrease draw length untill the problem goes away.
Id check DL first, usually easy to change unless your maxed out.