Can't get into a game with a party

When I play a a game with my friends, 90% of the time a game is found, I don't load in with my squad. Instead I'm stuck on the main menu and have to either join the game via EA Connect (Only works sometimes) or get invited via Xbox (Only works when playing with an Xbox person). How do I fix this? Or is this a glitch that's been plaguing me since launch?