Controller button input not working in Online games

Starting up the game, it's guessing if my controller will work or not. (PC, using PS5 controllers (tried 3))

In the game menu, squad games or any other offline mode, the controller works without issue. 
But when entering an online game (Rush, Rivals, Live events,...) It's always a suprise if the controller will keep working or not, it feels like flipping a coin.

In menu's the controller works, I start a squad battle game to warm up, still no issues.
Then I start a online game like Rivals and when the whistle goes, my controller stops working. After some time I get kicked from the game and get a DNF penalty. Back in the menu, the controller works again without issue. Entering a new online game results in the same issue. 
Rebooting the game usually fixes it.

When starting a game and the controller works, It's fine, I can game for hours. My PC can go in standby, controller can be switched or turned off and on... whatever you do, the controller keeps working as long as the game stays open, the issue doesn't occur.

Rebooting to then have a working game session that keeps working, I don't mind to much. I also don't expect much help on the issue itself because it's near the end of the game cycle. BUT the DNF penalties I'm piling up and the progress I lose in tournament modes or modes with limited attempts like Champions or Live events are very annoying.

Anyone tips/tricks to check if the controller will be fine before entering an online game session (without getting a DNF) would be much appreciated. 
Currently I'm joining Rush games as my first game, so worst that can happen is a DNF. Although I'm not happy with annoying 3 other players that have an inactive player in their team.