FROSTKILLER902 The game is very big to run on Xbox and requires some regular game maintenance to keep it running and avoid crashes and lagging. I completely cleared the library after advice from Xbox Support years ago and now keep it mostly empty. How many save games are you running? I never run more than 3 on my Xbox Series X, currently it's just 2 and one is a backup. In Other Households I clear some random Townies too as I've found excessive Townies cause massive issues. I clear them daily as there are a lot of new households being generated lately especially after travelling to public lots. It's also good practice to clear the personal and build inventories of previous Sims and households.
I'd also try powercycling the console regularly. Switch it off with the Xbox button on the front for around 10 secs until it's shut down completely and the light stops flashing. Unplug the power cable to completely reset the power supply. Wait at least 10 secs then plug it back in. Switch the Xbox on with the power button.
You can also clear your locally saved data and sync up with the cloud again by selecting Sims 4 on your console home screen using the menu button (3 lines) go to Manage Game and Add-ons>Saved Data. You'll see 2 sets of data, pick the set next to your gamertag/username and select delete from console only (this step should be carried out while the game is not running). Then immediately load up your game and it will sync up with the cloud and retrieve your data. That's another tip I got from Xbox Support and I actually do this daily.
I'm happier with how the game is running on Xbox Series X, I'd say it's way too big for the older models now. I had issues on Xbox one and Xbox Series S. But clearing space regularly is essential for Sims 4 on any Xbox. I have the full game with all add-ons installed so I'm really careful with my in game storage. Hope this helps!