Unable to ready up after a game

Whenever I sit in a lobby for too long or leave a game to return to the lobby the game won't let me ready up unless I leave my current party and join again. Is there any solution to this?