I have notifications disabled in-game, I only hear the sound. So the only way I have to accept a party invite, is by opening the origin menu in-game (Shift+F1), getting into my friends' list, finding the one who sent me an invite, open a chat window with him, and there you'll see a Join Game option. If you've been invited by this player, when you click the Join Game, you'll be added to the party, and if your friend is in a game, you'll be carried there, if not you'll just join his party.
If the person didn't invite you, or the invite expired, clicking the Join Game on Origin's chat window with him will only get you into his match.
Also this doesn't work if any of you are 'Appearing Offline'. To join a party or send party invites, you must be online.