Sims keep dying of hunger in adjacent apartments

I will start playing as a family in an empty apartment building, and any sim that is occupying the apartments in the building other than mine keep dying of hunger. I can't control them unless I go to their household, so I'm not sure why I'm expected to keep sims that aren't even in my sim family fed. This definitely needs to be patched as I've already had three neighboring sims die on me.