It takes two connection troubles

I've been frustrated with this issue that I've been encountering wIth It Takes Two when trying to play online with my brother. We are both on PC and I have EA Play and he's using friend pass. Initially when booting up the game for the first time we had no problems but when we tried to play again the next day when I invited him it took a while to load on his side and eventually tells him that the player has left the session when I am still in that session. This has been super frustrating as we have tried over multiple weeks to find a solution from verifying game files, reinstall the game, moving the game to a different drive, switching between launching the game on steam vs EA, deleting appdata files but nothing works, thus was done on both PCs. There was only once it worked after I reinstalled the game but after a couple of hours we disconnected and the issue persisted. Another thing we tried after giving up on it takes two was to try and play the other Hazelight game, A Way Out, to still have the same issue in the other game. We don't have connection issues on any other game we play together online so it seems to be an underlying issue related to these game. Any help would be greatly appreciated.