Roughing the passer penalty

Anyone else having an issue with roughing the passer penalties?  I barely touch the QB after they throw the ball and it’s called a penalty.  The QB isn’t knocked down or tackled.  It’s frustrating.