Game in a limbo when entering a challenge.

Every now and then when playing in a group party, me, as a  leader select the challenge to play co-op. The moment that is selected, the entire game enters a limbo state that you can't do anything. Ca...