Milestones resetting when switching families/rotational play

I don’t believe this bug happens for me, though there have been many opportunities in my game for this to happen. I play rotationally and the kids who are all growing up together will attend each others’ birthday parties, etc. The ones I’ve played as infants still have their milestones. Perhaps if you play through the entire infant lifestage before changing households, they will not be lost for you, either?