Random sims walking through house/apartment

Sometimes sims who are just walking through the neighborhood seem to cut through my sims’ backyard and house. My sims don’t know them, and haven’t invited them in. They just walk through and go? Ob...