Random Sims Keep Entering Home Without Invitations

While I'm playing with my sim, either a sim friend or a stranger sim keeps entering me home without knocking or letting my sim invite them in. They would just walk into the house, start doing things or walk up to my sim and chat with them.