Controller is fighting my mouse

I have Astro a40 headset that requires you to keep them plugged into a controller. I use mouse and keyboard for Battlefield 6, everytime a game starts the controller takes over as soon as I try to move. I can hear everything just fine. This doesn’t happen in any other part of the game.