Definitely test without mods.
If it still happens without mods: Are they getting routing errors? Can the infant/toddler no one is caring for be reached? Is the sim who won't leave the bathroom able to? What if you just do a "go here" outside the door?
If the first instance doesn't involve mods or a routing error - It's still somewhat normal for sims who are told to take care of an infant/toddler to end up canceling and doing something else if the infant/toddler either says no or is just too busy doing something else and they time out while waiting. Have you tried resetting the infant/toddler, or just monitoring the caregiver and telling them to try again when the action cancels?
