Sims wont have private conversations

Im trying to get two sims to have a private conversation but every time they walk to the other side of the house to join two sims who are already in a conversation. It doesnt matter what I select. Every interactions causes this.

Why??