Issues with the lobby

Lately me and my friend who live in the same neighborhood have been baving issues with teaming up with each other (game works fine when we aren't with each other). The issues are the following:

-Only one of us gets into the match, usually the person hosting.

-Party not ready despite both of us readying up

-Unable to join party and party leader quit

What we have to do is keep trying until we both get into a game, then the reque with party works for a while until it doesn't and we have to go back into the lobby where we face the same problems again.

Game's been unplayable for us and I'd greatly appreciate it if y'all help us. Also I am on PS5 and he's on PS4 if that matters.