I don't always but the family I am currently playing I have. I haven't found it too difficult. I love the trait you get for maxing out their skills as it makes levelling skills up on children so much ...
If Skill Day is coming up or if the parent completed the Big Happy Family aspiration, then I definitely do it. No reason not to. Otherwise if I'm anxious to move the story along, I'll get all their skills to 3 instead.