My infant lives with his grandparents
I was doing an only child per gen thing in one of my saves to make the game easier for me to see and play through since I play on console and the size settings aren’t exactly visually impaired friendly with the font already being like 6pt and everything gets smaller for every sim added after 4. That being said my 3rd gen had twins. They got a house together, 1 got married, while the other ones bf died the night of their last prom. 3 sims in the house. Eventually baby 1 come by. Twin B gets married to an Ito descendant but I don’t move him in because I’m already at 4. They have a kid and I move him in with the grandparents. 2 days later grandma dies. What will happen to the infant when grandpa dies? Does he get taken and disappear? Does the manage household screen pop up and he has to get put in another household? Gramps has the long live trait but still, his time is coming.
First of all, sorry to hear you have trouble with the accessibility of the game. I'm glad you've found a workaround that works for you (for the most part).
That said, this happened to me once, and the newborn disappeared from the game. I wasn't playing with the household though (only the father). There isn't an in-game option to then move the baby. Once it is left alone, CPS swoops in (behind the scenes, presumably), and takes the baby away permanently.
For anyone on PC, MCCC (mod) has the ability to have the game reassign a family (I've tested it and had a baby "adopted" by ignoring it for days, lol). For console, however, your best bet is to have a younger sim in the household, at least teen, so that your risk is not so high. I know you are on console, but this is for anyone else who may have the same issue/question who are not on console.
Since you asked only about the grampa and the baby, perhaps move in a cousin/other relative or add a live-in "nanny" (not the game nanny, but just a household sim that you use as an extra caretaker). That would give you three sims in a household, and under your limit of four.
Good luck and happy simming!