What to do with idle neighbors?

I am at the 100 limit and visit ALL my friends on a daily basis. I have replaced friends who are obviously not playing. (no V-day content or Duff blimp). I do not delete friends where I cannot tap any characters. I do not have a lot of characters myself as I have only been playing for 7 weeks and being in The Netherlands most people visit when I'm sleeping. So I'm sure I have friends who have nothing to tap in my town either, although I regularly clear the cards during the day. I hate it it when I open the game and all my characters have been tapped. That means that there must be friends who have missed out. Oh well...not much I can do about this.