[HR] Chestnut Ridge FPS drop and simulation lag

So I've had this issue happen a few times in Chestnut Ridge, specifically in Galloping Gulch while playing on the largest lot. Sometimes the simulation will start lagging (sims not moving) and after a few seconds the fps starts to drop until the game completely freezes and I have to close it from task manager.

This was with mods so I did make a new save with NO mods/cc and placed a new household on that same lot and just waited to see if it happened again. It did occur again after waiting awhile

Some extra information if needed...
- If I pause the game, the FPS goes back to normal. When I resume it will drop again.
- Last time it happened on my main save, I saw a sim on a horse riding towards my lot and when he disappeared my fps returned to normal and my sims start moving again. 
- This may be a coincidence but at least two times it happened, it was when a family member of one of my sims walking up to the lot. 
- I haven't seen this happen anywhere else, only on this lot in this neighborhood

PC SPECS
Nvidia 4070 12gb
i7-10700F
32GB RAM
Windows 11