Can you remove CC/Mods mid-game?

Hey all, Just a quick question: can you remove a mod from the mods file while the game is open, or do you have to close it, remove the mod, and open it again? I've always assumed the second, but...