VR: Double Vision and Incorrect Rendering on HMDs with Non-Parallel Displays

Sadly, I can't even play the game in VR, everytime I pass a mission it just exits to desktop, I checked GPU VRAM usage, lowered resolution both in Pitools and SteamVR, I run the game on windowed, turned down ALL graphical settings, It just seems to work really well when it does but in between missions it just crashes unexpectedly, I tried running from desktop mode to VR mode and I'm all out of ideas. My gpu is updated to the latest driver and it's a RTX 2060 (even tried lowering the core clock in case of a GPU power/clock issue).