I kinda find the way to fix it by creating a private match and when the host leaves the room then the other will leave with you as a party.
(I just play this game for like 1 day, so am not sure there is the other way to do it but this should do it for now)
Example
1. Have one player create a private match
2. Then let everyone join your game
3. Host press Esc and leave the room
4. And then you should see you and your friend in the party together
I hope this will help with your problem and sorry for bad english 😉
