Hi there !
I'm annoyed by this as well but I really don't think it's a glitch as it's been the case ever since they added toddlers to the game (and I believe it was the same for the Sims 3). In my opinion, as they are small children, the team probably thought it more realistic for toddlers not to interact with unknown sims. It would make sense IF family members didn't consider each other as strangers before they "meet" for the first time. Problem is that they do, so, as it has been said, toddlers can't interact with their grandparents or cousins if not in the same household.
To me it's uncanny, even beyond the toddler issue, that family members' relationships don't start off with a boost (similarly to sims in the same household) since their usually is a certain, undeniable bond, that can wear off obviously with time if not maintained. I would have loved some sort of setting taking under consideration the different links between sims, so for instance grandchildren/grandparents could start of at 40, cousins and uncles/aunts at 30, and then if sims don't see each other or do anything together, the relationship would wear off similarly to any relationship in the game. I feel like this way, there would be more appreciation for family ties, but still leave room for storytelling. I mean overall, the family aspect of the game is lacking... I hate that we don't have links such as son/daughter/brother/sister in law which is a pretty special relationship as well, or the fact that once sims are married, the don't become uncle/aunt to the other sims' niece/nephew. It's like sims are only married to each other and not the family... Sad!
Anyway, I'd love for it to change but I don't see that happening in the Sims 4, it probably is too much to work on/around as the game has been set as is... But if it CAN be altered, it would definitely make the gameplay more interesting as more family members would be involved. There is so much around family dynamics that is missing in the game, for instance I also hate that grandparents/aunts/uncles/cousins can't be called to babysit for instance. How come you can hire a nanny, a complete stranger, but not a family member? It doesn't make sense to me... A nanny that will, by the way, show up with a tremendous relationship boost with the children (maybe a similar setting could be added to family members so that when they encounter another member for the first time - find themselves in the same lot basically - they get a relationship boost CAUSE ITS FAMILY^^)...
But, I'm digressing here... To cope with the toddler and overall family issue, I try to first of all invite them as much as possible and include them in the toddler's activity without actually interacting (since they can't). So, for instance, i'll have the toddler play a game and have him ask for help/assistance (the cubes are a good one) to whoever I want him to build a relationship with, or i'll have him show a toy, or dance together. After a little while, once the relationship has increased a bit, you can start basic interactions such as chatting, ask to play, read a story and whatnot. It's annoying because obviously it could be much simpler, but it works... And when I really don't have the patience I'll just set all the children's relationships manually with MCC (does feel like cheating^^).
Hope it helps !