Can you remove CC/Mods mid-game?

Hey all,

Just a quick question: can you remove a mod from the mods file while the game is open, or do you have to close it, remove the mod, and open it again?

I've always assumed the second, but thought I'd ask if anyone actually knew.