Mouse acting weird

In-game the mouse just sometimes seem to "snap" to random positions and hold there, like an auto-aim kinda thing (except I don't think its actually aiming at anything). Obviously this is pretty game-breaking for me. My mouse works fine in the menu's, vehicles and any other game, just having this occur as I play infantry.

I do have a PS5 controller installed, but its not connected (since I thought the controller auto-aim might be interfering)