Honestly can't speak for 2 as I've never played it but as someone who grew up playing 3, the game sounds really fun in my head but that's just nostalgia speaking. I've tried going back to play it but I just can't get into it anymore.
I think as much as 4 has its faults you get used to playing a different game and how it works. It just becomes a massive shock going back to a game that looks and feels like 2008, and doesn't have a lot of the features we've got used to nowadays and just use all the time without thinking about it (like the gallery).
You get used to all these little things, like in 4 if I don't like a house - straight to the gallery. I'm familiar with the furniture and just know what I like so it's easy to find it, if I want a new sim - straight to the gallery to find a base sim I like and can make my own. Going back to a past game you just lose all those little things you become familiar with and it's hard to get used to not having them again.