Ideas
Frame limiter is not a sole culprit. It's a combination of DLSS Upscaling, Nvidia Reflex, V-Sync as well as frame limiter. Also when the framerate drops below the monitor's refresh rate, FG turns on. The more of these technologies you use the more this issue occurs. For now people find that the issue doesn't occur if Reflex, frame limiter and upscaling are all turned off.
But the frame limiter does in fact make the issue occur more often. On the other hand, I found that if I limit my game to 60 fps (in game frame limiter) I can toggle all of the aforementioned technologies and the issue doesn't occur. But I wish I didn't have to run my game at 60 fps.
Yeah this is odd then, because I didnt have an issue with DLSS or Reflex being on as long as the frame limiter was off. I did notice that when the limiter was on and close to my average, every time I hit a slight dip below 160, FG would stay on and take a while to stabilize and turn back off again, the game would have to have a less demanding scene to render for this to happen.
I tested setting it at 60 like yourself and since I average above that FPS I to stayed locked there with no FG assistance. It seems to me like FG is coming in to help smooth out any dips in FPS you may experience when your game is rendering just within the threshold of the limit you set. I stand by my theory however that the frame rate limiter is the main culprit in this issue, and for that matter Vsync as well if your monitors refresh rate is say 144, and your average is around there, any frame dips you hit would immediately trigger the FG to kick in and try to keep you around that 144 regardless of having the limiter set, because Vsync being on would be acting as a limiter.