DreamDwe11er are you using a lot of landscaping items on the lot? Like plants, or items from debug? That can also cause lagging and the occasional crash I've found. I was building on the 64x64 lot in Windenburg recently and added loads of landscaping. I built a lake and a boathouse and added a little rowing boat and a bridge. I used a lot of plants and trees, water decorations and debug items and it froze up quite a lot and I crashed. So I made sure to save around every 10 minutes or after I'd made a big change. I did notice it was fine in live mode and was surprised that the lot played smoothly, it was just build mode. After that I was put off 64x64 lots and did a small Oasis Springs lot, Dusty Turf (30x20) with absolutely no problem at all.
But I did have quite a bit of crashing on Series S, in fact I lost a few months playing time after the addition of My Wedding Stories. I've upgraded my WiFi since then to the second fastest, which is pretty fast. I did find that with the Series S the WiFi has to be spot on since it relies heavily on being online. I switched to Series X after a while for the increased power. I had some chats with Xbox support live chat too as they can be pretty helpful, especially if you get a Sims player.
Edit: another thought. Playing older saves can cause problems too. I just made new saves of my game and deleted the original saves as they were 3 years old. I run 1 game save with another as a back up and just been saving over the same files all this time. So now I've made new ones.
I've also found a bug report with players reporting a lag where they are trying everything to solve it but it's not helping and there are posts from Xbox players, you can add a reply here and upvote it on the first post:
Severe lag after 1 hour of gameplay : requires restart | EA Forums - 12344705