Why isn't it fair? I'm making the assumption that the 4.9 mil guy has climbed (rather than all the others dropping due to inactivity). The 4.9 guy has shown that they're extremely adapt at the game, have climbed and getting better rewards because of it/better rewards than the average 5 mil player, thus progressing more.
Either, the 4.9 guy will survive in this environment or drop slightly until they find a level that is competitive for them. Not really a biggy if that happens, just a bad week. What isn't fair is allowing Mr/Mrs 4.9 to continue vs accounts of the same size and never lose. They then start to think 'Hey, if I never lose, and I really only care about GAC.....why do I need to grow my account. I can just sit here snapping on the all the fish that swim by'.
This is the beauty of the ladder, it'll put you where you have competitive match ups.....even if they don't look competitive on paper.