Some more info after further testing.
I was briefly able to get HDR in-game! I did this by having the TV as my only monitor. I then changed the colour settings to reference mode in NVCP, however I don't think that's what did it, I think it is the next thing.
After I changed that setting, the display blanked then came back much brighter than before. Usually, Windows will adjust SDR brightness down in HDR mode when you're on desktop or using regular apps. For some reason, this didn't happen (it usually gradually fades down in brightness). It was retina-searing and still bright, almost as if being locked/forced into HDR mode system-wide. I ran the game, turned HDR to auto, and it was much brighter and instantly noticeable vs. when it was off.
For that gaming session, it was HDR the whole time. I could even toggle it off then back on using Win+Alt+B and it would kick back and stay in glorious HDR (makes a great diff at night in the game).
Sadly, this was not repeatable. However, it seems that, if we could somehow force system-wide HDR (so, basically, make Win11 behave the exact way you usually do NOT want it to behave, and make it blast HDR on all parts of the desktop and Windows regardless of HDR content being played), we might be onto something here.
I have a feeling this is what @WhatAboutMeBoss stumbled upon and perhaps also why it's not repeatable (at least for me, and the YouTube trick as described also doesn't seem to work for me).
@WhatAboutMeBoss -- have you had success repeating this several times?
It feels like we might be making a bit of progress!