Ridiculous Lag

Every game I go in, I spike from 20ms to 100-300ms, and it doesn't get shown in the quality report, everything says good or great. What on earth is this, I don't get this on any other game like battlefield or cs.
Help would be appreciated.