I've read a lot about this since the launch, and may i have an idea of how this occurs today.
It's possible that the matchmaking system doesn't allocate us correctly because it doesn't understand what's happaning on the server side. If you read the comments everywhere, especially with crossplay off, the matches usually only fills with 5 players and don't receive new players after they start.
If a new player starts searching for a match, i think that instead of grouping this player into the previous server that already has five players, the game creates a new server with this new player and maybe a few others who start searching at the same time.
It could be that this previous server is invisible to the game.
