Sounds to me like you are experiencing lag, is this on Garden Op games you join or host?
If it is one you are hosting, make sure no one is downloading files while you are playing, it is also prudent to use a wired connection rather than Wi-Fi - These will affect the quality of your connection.
If it is one that someone is hosting, not a lot you can do, except leave the game.
