I have all the bigger packs except Journey, Snowy Escape (and one more?). I have only 2-3 stuff packs.
I'm on a low-end computer and although adding one more pack hardly makes a huge impact on game performance, I keep thinking that al add-ons will sum up with more stuff and more calculations going on in the background. Another reason why I skip a pack every now an them (or get a new pack late like I got IL more than 1 year after release) is that I think packs come too often, and some times I will rather play my game than installing and exploring new content. With each new pack I tend to focus on a couple new households and of course some new features, meaning I'll take a long break from my rotation. If I get 2-3 new packs every year there is no time left for the rotation at all, LOL.