"Waiting for players..." period before challenges

I really dislike how most if not all challenges will have a short period to wait for players to join (when I have yet to see a single player actually do so) before allowing me to actually do it. Especially considering that some challenges amount to "Do a single trick & collect the coins" and are completed in less time than the waiting for players period took.