I also have this problem: frames drop to half. I believe this is a nvidia driver issue. Gpu stays locked at half speed (MHz); does not return to "idle" (135 MHz, in my case). A cold boot solve this problem, cause gpu driver is reset back and eveything works good. But you can also restart (reset) gpu driver using devide manager (inside control panel), without restarting the whole pc. To do this I select display graphics (inside device manager), double click my gpu and then click disable (inside driver tab) and confirm. Wait a few seconds and enable it back. If you have sli, I think you should do this to every gpu: first disable all of them, then enable them back.