PGA Tour PS5 - Local multiplayer controllers randomly change

Yer, witnessed this today when playing with a couple of mates. I thought maybe it was down to if someone pressed a button on the controller first, but it didnt seem to be the case. Also was not happy that I couldn't use my career character and have my mates use there's even though all have logins on same ps5 with game data in sync from playstation cloud. Maybe there is a method for this.