How do you make peace with the randomly generated sims?

I'm not bothered about townie sims appearance or fashion, but about the fact that they get generated at all. Doesn't matter what you look like, dear pixel, you're stealing the spotlight from my ~100 curated custom townies, who could have appeared instead of you. 
I tend to go on deletion sprees either at the end or the beginning of a game session.