I usually play with none up at all unless I am building. If so, I put them all the way up to do the wallpapers and put up wall decorations then get taken back down,
I prefer having them partially up. Though I recently built my family a house so huge I can actually leave the walls all the way up and still see what I'm doing. It's a nice change of pace.