This also happens when using the Xbox app on windows. I cast/play the game from my Xbox one over to my laptop/screen using the xbox app, and if the main controller disconnects (idle, batteries, whatever), when I get back into the game there is at least 1 ghost controller added and the main controller becomes a secondary one. Every time the controller disconnects, once you come back from idle status there is a new ghost controller added.
At that point one of the the ghost controllers becomes the main controller; if I'm on the pre-game screen, the ghost controller is auto assigned to my team and even if assign my controller also to my team there is no way to enter the game if not pressing Y as per a cpu vs cpu game. At that point, I have to go into the Select Sides menu and FINALLY I can assign my controller to my team, un-assign all the ghost controllers and start the game.
ok so, I might have found a solution on my own.
If I set up my Xboxone on Standby mode instead of Shutdown mode in the energy saving settings and cast my xboxone on my pc using the Xbox Console Companion app on windows instead of the regular Xbox app, I no longer get the ghost controllers once my main controller shuts down or goes idle.
Hope this helps others with this issue.
