I forget the number of residents in each official world but if I remember correctly, ranged from 50 to 90 Sims. I think in most cases the population count is around 60 to 70 Sims.
This is why the developers say that TS3 is a mostly CPU intensive game because each Sim/Sim Pet (Cat, Dog and Horse) is an individual Sim A.I. that the CPU calculates for the most part when they are out within the open seamless neighborhood. And even while inside of houses and rabbitholes, the A.I. still runs but just with less impact because there aren't any visuals and the code seems to run in the background. There is an explanation of this by TS3 A.I. designer. You can find it yourself if you are interested to know more about how Sims are being processed.
Using NRass Tagger and NRass MC, it's easy to see where Sims are at and what they are doing and I can use the Open option on a closed house to see A.I. driven stories unfold that are what I call, in process behind closed doors. Though some A.I. in closed houses are in a dormant/idle state I notice but their status changes quite a bit depending on various factors.
Anyway, I feel like 100 Sims or more is pushing the game engine to the limit with what it can handle. Not so much the limit of what the CPU/RAM can handle but a 32-Bit application game that cannot use more than 4 GB of RAM is not going to handle too many residential Sims including controllable pets.
Also, the biggest culprit for lag in my personal experience are the use of mods that are CPU/GPU intensive. Especially, CPU because again TS3 is already a mostly CPU driven game, thus putting more work on the game engine than its original workload will create poor performance regardless of a gaming computer or a super computer. And again because it is the limitation of a 32-Bit application. The game engine can only handle but so much extra workload.
80-85 Sims max seems to be okay for me and will vary for each person since everyone has a different computer configuration. On top of Sim population is also object count in each house and the overall count in the whole neighborhood. CC with high poly counts will drag down performance as well and there are so many other things too. Like bad lot design causing route failure, broken objects that Sims keep trying to use and can't thus get stuck in a loop and so on. It is not just Sim population that can cause lag/camera stutter. The official world populations are all within good ranges in my personal experience.
My gameplay performance always drops when I start using certain mods compared to my other games without mods or gameplay altering mods.