I have some friends that I picked up (or vice verse) from the beginning that I have hidden, but if newer neighbors aren't active, don't leave footprints in my game or haven't progressed I delete them.
If they're someone from forum and I see them chatting here I keep them, a lot of them already said they don't visit all the time or do part of their list one day part another.
I do get attached to my friends list though, even if I've never actually talked to you, I get used to your nick in my list and feel kinda bad as I toss you out on your ass ;)