Happens to me too. I'm also using switch pro controller, but had the same issue on PS4 controller. Playing with experienced control scheme fwiw. Tried restarting, tweaking what feels like relevant settings, switching control schemes. Some days this bug occurs, other days it's not an issue at all.
Edit: Managed to fix this. If you're on steam, open the steam overlay (home button on switch controller) -> controllers tab on the bottom -> edit layout -> right joystick behavior -> click on cogwheel -> set the stick response curve to aggressive. Not noticing any side effects atm. I also might have set the "current button layout" to the gamepad template before doing this, I can't remember.
Edit 2: Try setting it to relaxed if you're finding it hard to manual.
