Sims automatically leaves work when another sim in the household leaves the lot

One sim in my household will leave to go to work, and I'll decide to spend the day with the sim with the day off and take them off the residential lot, but as soon as I load into the park/cafe/whatever, the working sim automatically finishes work and goes home, even after being at work for like, an hour. 

I've updated all my mods, and then I removed them, to see if that was the issue but it's not. 

Basically I can't go anywhere with my sims unless they both have the day off, which much like real life, isn't very often