Whoever sends the invite has a better chance of winning in Online friendlies

My friend and I played online friendlies , when I send the invite my players are faster and I win the game and when he does he's players become faster and mine lag and he wins the game , this is a really bad experience,  we had to test it out and we noticed that was the experience.