Is there away to stop sims from trying to come into my house?

What lot is your house built on? Was it originally a community lot that had a food merchant? There may be some left over hidden marker from that lot.

I would turn on buydebug to see if there are any hidden markers. It may require you to evict your household, save them and then save the house. Then bulldoze the lot and look at the empty lot with buydebug to look for hidden markers. If any remove them then place your saved house and your saved household back.