Defenders leaving receivers DURING the route...

I have this issue regardless of a man or zone call. The defender will start out running with a receiver but just as the QB throws the ball the defensive player will leave the guy and rush the passer. This of course leads to a completion and usually a first down. Anyone else having this issue?

thanks