Population increase

There are many factors that affect the population.  We may not even know all of them yet.  But specific to specializations, there are caps involved in the boost you can get.  So, for example:

Gambling, Landmark, and Entertainment specializations have a combined 100% cap. So if you have a residential zone with a Luxurious Casino Tower (60%) and a Washington Monument (also 60%) nearby, you'll still only get 100% boost even though you should be getting 120%. The other three (Parks, Education, Transportation) stack up better. You can get 100% boost for EACH type (as opposed to combined), up to a maximum of 300% boost.

From what you've said, it sounds like you may be running into these caps.  By carefully rearranging your city, it's possible for a given residential area to get a total boost of up to 400% based on the info above.

Hope that helps a little.