I worry that if Maxis removes idle chat, then that would break the immersion even more because most sims does not know each other. The mod "Have some personality please" does remove idle chat, so I would use that.
I completely agree on the issue where sims want to idle chat with you from a mile away though... if closing the conversation by clicking on it in the lower left does not work, then if you play with testingcheats on, you can try shift-clicking your sim and click reset, that will solve most issues where the game locks your sim from doing anything.