Do I still need to remove or move my mods folder elsewhere whenever there's a patch update?

I have never bothered moving mine either. My perhaps flawed reasoning is that the actual game files are being patched and not the user files. However, I start my saves over so often that it would not be that upsetting to me if I lost them. I'd just shrug and reinvent my world again. So far though, nothing has ever been messed up during any of the patches and this has applied to Sims 2 and Sims 3 as well.

Still, the accepted wisdom is to move the files out to be safe and it only takes a few seconds really.