Is there a way to prevent empty (townie) houses?

My favorite way of playing this game is via legacy challenges however there is a thorn in my side which ends up wrecking the fun a little, it's the random emptying of townie houses as time goes by. U...