Next Match not starting

Half the time I play Sabotage the next match doesn't start. It just times out and then stays on that screen even if I press A when it prompts me to. It just stays on the screen until I quit to the menu and start a new match. This happens in any game mode too I just happen to play Sabotage the most. I am on Xbox Series X.