Pit Stops ruining races

Every time I go for a pit stop, I get held up by the red light for about 15 to 20 seconds. I went from P1 to P16 because of this once. The very next race I was held again for the same amount of time.