I'm the same with the several rotational players here, households spanning from 1-8. Recently, several of my 8 person households have my elders dropping like flies, so the average of sims in my rotational houses are about 5-6.
I never really stay playing with only 1 sim. It gets too boring for me, especially if there aren't any children around. By minimum, I tend to feel more at ease with at least 3 sims in a house. I like 5 person households, too. Sometimes having too many gets overwhelming, which is when I usually rotate to my 1 sim houses. :)