Sims frozen yet still moving?

For a while now, my game has done this thing where it would randomly freeze if I was in 2x speed for too long or just switching between speeds too frequently. The game would almost pause and the sims...