Controller issues when entering fights

Hi 

I've been playing the game for a couple of hours now and noticed whenever i enter a fight my controller doesn't work anymore it get's stuck and nothing happens.
When i'm running around in the open map no issues at all it only happens when i enter fights.
I always have to disconnect my controller while fighting and reconnect it for it to work.
When i'm in small fights i have time to do it but once i'm in a mob fight or boss fights or such i have no time to reconnect my controller , it's really annoying as hell.

I'm using a PS5 controller on steam (and steam input is disabled)
And of course in all my other games i have no such issues