Fps drop from 100fps to less than 30fps

Generally my game runs very well all on ultra, at around 100fps, but sometimes when I play the game a little, for example, after replays, or end of missions (I believe when it runs a replay) my fps d...