Random NPC's will invade and never leave...then become family ??

when you are in an apartment even if you make sure each space is designated, theres no crazy lot trait or challenge or mod. random npc will come in even if you lock the door and then its like all the communcation turns into as if theyre family not randoms, if you do get the chance to say leave they just get up and do the shoulder shrug, ive had to off a whole freaking town of them