Ultra Speed will slow down to normal after about 10 seconds

I've been having this issue for awhile where when I click on ultra speed, it will only go fast for a few seconds and then slow down to normal. I then have to press play, then click ultra speed again ...