Townies interacting with selected sim

Whenever I take my Sim to a gym venue, random Sims and the gym trainer always start talking to my Sim while they’re working out, which I don’t want. Also, when they talk to my Sim, the friendship level increases rapidly, which is annoying because I don’t want my Sim associating with townies I haven’t interacted with myself. If this could be fixed, it would be very helpful. Thank you.