How do I stop my sims from picking up their toddlers for no reason?

I play with autonomy off and I expect my sims to sit there doing nothing until I queue up something for them. But ever since they got toddlers, the parents are constantly getting up to pick up the tod...