Friend H2H Issue

So, I want to play with my friend in a H2H match; he can join my lobby and vice versa. But once we pick the teams and the loading screen appears, after about a minute, I just go back to the menu. My ...