Speed up slow down stuttering

After about 3-5 games of smooth gameplay, the game starts to have heavy input lag and speed up and slow down, which progressively gets worse the longer you play. The only way to fix it is to completely restart my pc. It is 100% a game issue and not a pc one.

7800x3d
5070
32gb ddr5 6000mhz