Looking at the whole series, Sims 2 is the only one that felt like a 100% upgrade. The jump between Sims 1 and Sims 2 is MASSIVE. The Sims 3 added and improved a lot, but it sacrificed detail and I've never been able to overcome the optimization issues and bugs, even with patches and NRAAS. The content in the Sims 3 is a ton of fun, but the stress I have from knowing my game could turn into a BFBVFS at any moment puts me on edge too much to fully enjoy the game. The Sims 2 has problems due to its easy corruptibility, but these issues seem easier to tame from a user standpoint, especially since the game as been out for over a decade. I'm much more at ease playing the Sims 2. Sims 2 sims are also, in my opinion, some of the most realistic and "organic" sims in the whole series thus far.
It's hard to compare all of them side-by-side, since there's a two-decade gap (or 1.5, however you look at it) between 1 and 4. I think that if I look at them while accounting for their age and place in the series, I'd say 2>3>>4. Sims 1 I don't like to compare too much, because it was the first of the series and innovative across all fronts simply for being the beginner of it all.