Ultra Speed will slow down to normal after about 10 seconds

I've been having this issue for awhile where when I click on ultra speed, it will only go fast for a few seconds and then slow down to normal. I then have to press play, then click ultra speed again and so on an so on. I don't know if it's suppose to be that way or what, but it's certainly annoying.