Why are random sims coming to my sim's house and using my pool?-lock please-

I have the same thing. I put up the fence and now they stand there complaining the fence is in their way. The mailman does it every day. My sim got ready to leave for work and there is Geoffrey Landgraab sitting in her house like it was his. I have the fence as the front door and that usually works. Locking everything won't stop the complaining that they think they should be able to use my sims house.