Sims4 Visitors

I moved my sims into a new lot, and now sims are walking through the back door to knock at the front door. It was weird watching the sims practically phase through to house to ask to cone inside. I am not playing with any mods. The game isn't broken, this is just something I find odd/annoying.