Xbox One Headset adapter not muting in game chat

I have an Xbox One headset adapter and you I cannot mute in game chat audio while using the adapter.

When I'm in the game, I press and hold the button on the puck with the controller symbol (top left) until it beeps.  That's suppose to set game volume to 100% and chat to 0%.  However, I still hear people chatting.

This issue does not exists when I'm in a party chat. When I follow the procedure above while in party chat, the chat room will be muted and I get game audio only.

This is also not an issue with the included Xbox One chat headset.  If I press and hold the minus button till it beeps, I will no longer hear in game chat audio.