Weird NPCs spawning in my house!

I can't play like that! Weird npcs spawning inside my living room at Sulani. I kicked one of the townies out of their house and took over their lot where I built my own stilt house. Don't know if this...