OP, I'm feeling much the same as you. A while ago I was excited about the idea of TS5, but that's rapidly fading as I just don't have much hope in TS5 being a robust improvement. I think TS5's build and CAS will probably be very similar because they both seem well received. I'd like to think the elements that came in patches (as you've listed) will be included in base game.
I have no hope that things like weather and pets will be included in base game even though these are things that I would like to be included in base game. I think EA would rather release these features for a cost right away and then get to the "exciting" stuff afterwards, similar to how they're doing now. I'm over it honestly.
I really don't know if the personalities will be improved. The team doesn't seem to find anything wrong with them or how players receive the way personalities operate in the game, so I don't know if they would think this is an area that needs improvement.
Part of the reason I want TS5 to be here soon is so I can just know what it's like, and then I can know if I'll stick with TS4 or move on. I know some people say you can play more than one version, but that only works for me with a game like Pokemon lol. I can't play more than one version of TS at the same time.
