This bug happens when you invite a sim to the current lot (not a household member) or go with them using map, mobile... to someplace. Its easy to replicate, go with your sim to a bar, invite someone to current lot (not organizing an event, just inviting). The invited sim will come and stay for a little and then, they'll leave the place.
When you organize a date or something through the event planner menu, it works fine. In fact the workaround I'm using is organize a friendly hangout through that menu and not directly, which is kind of annoying
Thanks for the extra info.
To be sure, they get grouped together for you when someone arrives ? Do they leave while you interacting with them ?
