Controller ID clash when disconnecting

I am playing on PC. I have a controller that I connect to the PC. However, if I leave my games on pause to do something and the controller turns off, the game doesn't let me continue my games anymore...