Sims return from work early

I have a few Sims in my household and they work at different times, so sometimes I go to a different lot for a while with one of them, but when I go back home with this Sim for some reason all household members go back home, including the ones that are at work. Because of this work is cut short and I only get a small amount of money, and I can't send them back to work and it affects their performance negatively (i had one sim almost getting promoted but because of this glitch her performance went down so she got no promotion). Is there anything I can do about this?