Offline players during the game

I assume you're talking about unranked modes, and no, they have to quit penalty. Nor should they. It's the wild west.

What you're seeing is people who don't like the jump master's drop, so they quit. Or they didn't like the skill distribution curve in the lobby, so they quit. Or they didn't like their teammates' banners. Or they won't play with people who aren't on a mic. It can be a lot of things. The only way to get away from it is to stack or go to ranked.