Stuck on loading screen when traveling with more than one sim

There seems to be an issue with traveling. I have a family of five and everytime I try to go to a community lot the game get stuck on the loading screen. After I while it seems like the lot has loaded as I can hear sims talking in the background. But the loading screen never goes away. If I only travel with one sim it works fine. What might be the issue?

I have no mods or CC