I don't think I could replace a sim of any age once I've played with it a few minutes. Delete and forget, yes, but not replace with a new actor.
To prevent that I hardly ever go through natural pregnancy. Most of my sims' offspring gets created via Play with genetics (as teens, so that I get a good impression of what they will look like, then aged down while still in CAS). I have a preference for sims who are a near-average mix of their parents, not fond of them looking too much like one of them only.
So, yes, I get where you are coming from!