Issue Loading into games with Friends

I have had issues for over a year when I play in a party with my friends, this applies to friends I play with on PC or on Playstation 5 (I play on PS5).

Almost always whenever we go through matchmaking, it loads me into a game on my own. If I join their lobby then I am the only one who does not load in.

The only minor fix to this I found has been to restart my game and ensure that they join and ready up ASAP, which still does not work 50% of the time.

sometimes it also does not display they are readying up, meaning the game does not even start matchmaking.

There are no issues when I re-queue directly after a game.

Is there any fix possible to this, as having to restart my game everytime and taking 15 minutes to start playing everytime is a nightmare…