[XB1] Getting "Lag" on Sims executing tasks in a large household

It seems to get worse slowly, over time.  Though I do feel sometimes I come back and play and it had stepped up a bit more with the lag than other sessions. Once it gets bad, it just stays bad, session after session.  Only when I did thinks like clearing cache or moving the game files back and forth between external and internal HD does it get better.  It is not 100% better, but maybe like 70-80% better, then it starts to degrade again.

Here are the videos. 

1. Bath time lag (This one shows some of the stopping between queued tasks, and also the "popping" of the lag when something that is causing the lag (this time likely the notification that comes up at the end of the video), and then things progress for a time.)
2. Dinner Lag (This shows similar lag while eating, and again notifications pop at the end and let things progress)

Screenshots of the house:

1. Over all lot
2. First Floor looking from the front of the lot
3. First Floor looking from the back of the lot
4. Second Floor looking from the back of the lot
5. Backyard
6. Basement

Let me know if you need anything else.