Unplugging Xbox one microphone from controller caused game to crash.

I have had this happen multiple times so I know this is the issue. Whenever i unplug my mic from my controller while in a game, the game crashes and takes me to the xbox one dashboard. The reason i u...