I understand the frustration @MikeBru2102 but if this were game related then everyone would be seeing it or at least a huge number of players.
If you are using an Xbox controller but not using the official dongle then you can run into issues like this and it is not something we can assist with.
The game is plug and play, I use my PS5 and Xbox controllers with it without issue but I keep them up to date and use the official support adaptors for them as Bluetooth devices often have issues. I have a certain lag when using my soundbox and PC over Bluetooth that does not exist when I have it wired.
That you have multiple Bluetooth devices connected can also cause issues. The more you have connected the more likely you will see things like this pop up.
You can blame the game all you want but it won't fix the issue. Our team can't push a fix for an issue that is on your system side. And as you said yourself, you don't have it all the time so that is further evidence that this is not a game issue but system related.
Darko