It's nothing to with quitting or playing better players. If it was that would mean that the game is completely broken. It would be like Liverpool turning up to play a league 2 side who had lead boots on. The delay is there before my opponent even touches the ball.
And again how could it possibly be my connection if Rivals is working fine ATM. Both have the same ping.
Friendlies last year were by far the best mode. So to understand the connection if would be better asking someone from EA to know why the big difference this year.