I'm running Big Sur on an early 2015 Macbook Pro. Although it did not initially occur to me that it might be related to the OS update (so I can't guarantee that it's cause and effect), since around November, the Sims has been extremely slow to load. From first clicking on "Play" or on the app icon to the point where the lot loaded would consistently take 30 minutes, the longest portion being the time to load the lot. Once the initial lot loaded, there were no excessive load times when travelling.
Disabling my mods and later removing the mods folder entirely had no effect. I've also tried repairing the game, unsuccessfully. Moving the Sims 4 folder to the desktop and reloading the game helped somewhat, bringing the final load time from ~20 minutes to 12, but that's still far longer than I would expect. This is for a save that I've been playing for about 6 sim weeks, but it seems to be happening for all of my saves except one that I created just for testing and have only played for a few seconds.