Timed out error (Xbox series X)

Every time I try to load into a math my game freezes and goes black then after about 20 or so seconds it shows me the title screen again and gives me the error saying

"TIMED OUT" "Yours session timed out from inactivity, and you've been disconnected."

I was able to play with out issue before this nothing seems to fix it I've already tried restarting the game, console, modem/router other games work just fine I haven't been able to find much help online either I'm not sure if this is a common occurrence or not