Disconnected from ranked match but can't rejoin and get penalized

This has happened to me every time I have disconnected and tried to rejoin. Was on the verge of diamond in 4th place just now and it happened. Lost a ton of rp and got a timeout because I tried rejoining but it wouldn't let me. I was in a party with my teammate so I know the match was still going for about 5 minutes after. I had plenty of time to reconnect but the game wouldn't let me. Internet on my end was fine, since I was talking to my buddy the whole time.