I had that happen with my Sims' infants, too, over and over again, and at one point, I got so annoyed that I aged all three infants up to toddlers. That made playing a lot easier and enjoyable because toddlers are a lot easier to handle - they can do stuff on their own, and if you hire a nanny she will take a lot of work off your hands. The only annoying things about nannies is that they will always give your kid something unhealthy to eat, and if you have more than one infant, forget about nannies altogether because they can't handle it and will just stand there doing nothing while the kids cry their eyes out.
At one point, I had three infants on one lot (don't ask...), and the game was completely overwhelmed with that. It had nannies spawn like crazy (I think I had four or five nannies), the infants kept glitching in and out of their cribs, the parent Sims would just stand around doing nothing while the babies were crying... very, very stressful, but it got a lot better once the kids aged up to toddlers.