HDR mode on Xbox X LG C6, gets too dark and washed out

I'm having the same issue with screen brightness changing when I go into multiplayer (and then changing back when it goes to the menu screen between matches) I've been messing with my screen setting as well as the in-game settings and can't find a way to fix this.