Re: Trying to understand SBMM - 20 match experiment

@Professor_Doggy 

Ty =)

I just think its bad for everyone when rumors are spread and taken as fact. Some guy says there are 20+ predators in all their lobbies, some other guy gets killed by the only pred in his lobby and then thinks the first guys claim is a fact and start to hate the "unfair SBMM" =)

@apostolateofDOOM 

I have been Diamond 2,3 or 4 every season up to this one, depending on how much I grind (I dont play that much anymore, and almost never ranked so I guess I am whatever you get reset from D2). I have 2.4 overall K/D but most of my matches earlier seasons are  with premade squads of equally good players. When playing solo with randoms, I think these matches represent my "casual with randoms K/D" pretty well, i.e. 38 kills on 20 matches = 1.9 k/d

And on the question what this experiment actually showed:

Like I said in the summery; I dont want to draw too many conclusions since there were so many other variables affecting those patterns I was looking to find (correlation between performance in previous matches and level of opponent/teammates etc), but all I can say is that it basically felt like pretty much the "same" lobby in all 20 matches. No "super sweaty, all predator-lobbies" that some people claim they always are seeded in etc. 

But I do think one result in this experiment was very consistent over all 20 games:  I was, regardless of individual performance in the previous game,  the big damage dealer in my team. 

I had the highest damage in 15/20 matches. In 7 of those 15 had I more damage then my teammates combined. I think this is some kind of empirical evidence that the teaming of solo randoms isnt 100% random, that its rather skill based to make even teams. Let me also remind you that these were 20 straight games in a row, and randomly or purposely picked matches.

Ps, 

for the sake of "scientific transparency", here are all the match summaries. I think they are in order but not sure.

https://ibb.co/CnDKCvN
https://ibb.co/Z1DYgnX
https://ibb.co/ZzWnBBb
https://ibb.co/SvZ4bwD
https://ibb.co/wsHgHNF
https://ibb.co/1qqc4c4
https://ibb.co/LpzTPk4
https://ibb.co/dG4wbLm
https://ibb.co/SxsqfSx
https://ibb.co/9vYd96K
https://ibb.co/9rDCWxL
https://ibb.co/sqKPWn5
https://ibb.co/Syy8DdW
https://ibb.co/bH2nJRz
https://ibb.co/f8f0qmC
https://ibb.co/xCbhp8b
https://ibb.co/z4CQYYc
https://ibb.co/k8QzMCF
https://ibb.co/k3M1ZsJ
https://ibb.co/xMn2W6Y

@Balladalidila In your reply to @Professor_Doggy that it's just rumours or misinformation that people are playing in lobbies filled with predators but you're the one (intentionally or not) spreading rumours trying to back up your claim with poorly designed research giving misleading conclusions.

I think firstly trying to judge a servers skill base on trails is flawed, especially when that player is seeking diamond or predators. Series 1-2 are the only seasons where you kept your diamond trails and in the results you posted you, on average and pretty consistently, had more than a handful of visible trails at that rank.  If that's just over a third of all diamonds in that match (giving a rough estimate) as Series 3-4 aren't visible, that would roughly be a third to half you're lobby at the same rank as you. You may have the odd predator here and there that have fallen from having bad rounds and others who have been doing reasonably well to fill some of the void but your matches seem to be approximately at your skill level throughout.

You won't be placed in full master/pred lobbies when you're diamond tier and not at that level. Not to criticize you but in the context of the topic at hand.

I do know from people who have smurfed that you can have at least 2 rounds after creating a new account before SBMM puts them in their respected lobbies so i doubt it's dependant on just the previous match but without any clear win streaks or bad rounds consecutively placing 15th with zero kills where you were clearly out matched i'd say it's hard to make a determination from the graph you provided.

SBMM has been discussed pretty extensively and multiple times in general chat. One of the conclusions and also verified by a post from someone at Respawn on twitter is matchmaking will try to balance teams out for solo que players. If you're diamond/master/pred solo queing, it will give you 2 pretty low level/skilled to balance against a gold/platinum pre made for example instead of stacking you with people of equivalent skill so on this you were right. I personally don't like this method because if your skilled and carrying 2 deadweight players then you're basically playing solo and if you're the low level you'll likely get beamed first as they're the most likely to wander from their team leaving them at a disadvantage = Not enjoyable for either but statistically balanced.

They also have this in ranked now which imo is a complete joke to competitive or aspiring professional players alike.

Server location and regional time, que time, SBMM are just some of the things the algorithm will consider when matching teams but also things like accuracy where we can't see the stat will also likely be used so while i applaud your efforts at trying to figure it out, i think the way in which you have tried to calculate it is fundamentally flawed in combination with such a small sample.

I can also vouch that some of my lobbies are up to 80% master/predator trails so i know how gruelling it can be to play like that often. It's mentally draining leaving no room for error and sometimes be tougher than ranked with those willing to try pull off riskier plays and i'm totally sympathetic towards those that want a more casual experience without feeling like their playing scrims in pubs when we have ranked. It's not about wanting to bash noobs but rewarding better players who have invested the time to improve instead of punishing them so lesser skilled players have a better chance and over time more likely to spend more. Why should a silver have the same win rate as i diamond? What's the point in improving if the end result is always the same?

Equal opportunities or equal outcomes and SBMM or EOMM.. I'm sure @Axs5626Sxa5001 can discuss this more with you. 😛 

Next time I log on, I'll add you. My gamer tag is Squid Lemonade.

@apostolateofDOOM 

"...but you're the one (intentionally or not) spreading rumours trying to back up your claim with poorly designed research giving misleading conclusions."

Then we have different meaning of the word "rumors". What I "spread" is the black and white stats of 20 games and my observations during these games. I have images of all the champion squads, all my squads, all the diving trails (you can see on the "player left on ship" approximately when the image was taken). If you want them I can post them as well.   

I never said that "no visible fire or lighting trails means for sure low skill player", did I? I said; "here are the drop trails I saw in 20 games, use this information to draw whatever correlation to the lobbies overall skill level, weak as it may be, as you want" 

I never said "As a player at my level (2.4 k/d), I will be placed with master pred lobbies and since I didnt get placed in these lobbies, they doesnt exist at all". 

I never even said anything about server location or time. I just stated, once again, black and white as a variable in my experiment, that I played on EU, 7pm to 9 pm with randoms. No conclusions even made at that topic.

What I did say that I didnt see much pattern in either competition or teammates, regardless of my performance. What I did say was that I almost in every game out damaged my two teammates combined. Those are NOT rumors, those are, once again BLACK AND WHITE STATS from said 20 games..

"I can also vouch that some of my lobbies are up to 80% master/predator trails so i know how gruelling it can be to play like that often"

If you claim that you have 80% master and predators in your public lobbies, and also say you are at a much higher skill level than me, well then THAT statement doesnt at all contradict what I saw in my experiment. On the contrary; if you better than me, presumably a master or better since Im high diamond, and actually have 80% masters or better in your lobbies, it just validates why I have so few, which was one of my observations, right? 

Here are all the images if you are interested. I think they are in order BUT as you can see on the table in my first post, I sometimes failed to take a screenshot so there are no correlation between the match number and the rows. You just have to use the table and see which matches I failed to take a screenshot in each category =)

My Squad                                 Match Champions                           Dive trails

@Balladalidila I think you're misinterpreting what i'm saying.

''The only real patterns I could find were:

1) My lobbies seemed to almost consistently have the same ratio of preds (0-2) and diamonds (8-12ish) regardless of how well or badly I played last game).''

SBMM is working as Respawn intended keeping your teams reasonably balanced while explaining that other variables you failed to mention will influence the results and also how trails shouldn't be used as a gauge to determine lobbies.

''So for those who claim they always get into lobbies with 20 preds, I dont know what you are talking about tbh''

I think i've explained quite clearly why and how does that validate your observations when you say you don't know? Maybe i misread or skipped something here?

''2) Me, as a 2+ k/d player almost ALWAYS out damaged both my random teammates, both in quick "low damage" games where we died early in the match, or games in which we survived in to late game. Very often, I did more dmg and both my randoms put together.''

''It also seems its trying to match "stronger" players in the same team with "weaker" players to make  "even" teams.''

I actually backed this point up with, ''One of the conclusions and also verified by a post from someone at Respawn on twitter is matchmaking will try to balance teams out for solo que players. If you're diamond/master/pred solo queing, it will give you 2 pretty low level/skilled to balance against a gold/platinum pre made for example instead of stacking you with people of equivalent skill so on this you were right.''

What i'm disputing is ''very few Pros, few highly skilled and many good-to-average players'' because if that's the case then why am i at times getting a majority of master/pred trails in my solo que matches? Maybe a premade squad and others of similar skill que at the same time so it will balance the teams with equivalent teammates but not giving any room for lower tiers to get in, being unable to give rise to the usual pick and mix teams you were describing.

This contradicts what i agreed with you on that point previously.

Also explaining your experience wasn't that different as you didn't have any drastic changes consistently over multiple matches.

@apostolateofDOOM 

Well, maybe I was a bit unclear how much I weighted the observed trails in my assessment of the skill level of my lobbies. The trails was just a part of it, and as you can see at the now linked images, my matches looked pretty much the same, trail-wise, which was not many pred or diamond trails. 

What I basically meant was that, regardless how well or badly I played in previous games, my lobbies seemed to be at quite the same overall skill level, which was considerably lower than high diamond, master, and predator lobbies. I compare with my ranked games. Last season I made it to D2 which meant pretty much only Preds masters and high diamonds, so I know how those lobbies feels..  I mean, I wouldn't have got an K/D of 1.9 over 20 games with randoms IF my lobbies had 20+ predators in it. As you now can see on my champions squad screen shots, those werent that hot either. I just dont think it would be very likely lobbies with "the majority masters and preds" that you are referring to, would have those kinds of Champs....

Hrm well assuming that its true you are getting matches with majority preds and masters just adds more information to the quest to figure out SBMM, right? That doesn't contradict my observations of very consistent lobby skill level of   "very few Pros, few highly skilled and many good-to-average players".  You made it sound like you were quite better than me, right? Maybe there simply is a threshold/line you passed and I have not, seeding me into "mid-to-highly" skilled players, me being among the "high" (which would explain me out damaging my teammates), whereas you get seeded into "very high-to-PRO" skilled players, you being "very high" .

Or do you play as a premade duo or trio when you get these lobbies?. That at least SHOULD be an significant variable in a matchmaking system since playing solo with randoms almost always means teammates leaving the match whenever they get killed. I wouldnt be surprised if you get seeded into harder lobbies if you play as a trio etc. Maybe thats the next step for me? Play 20 games with premades of different skill levels =) 

@apostolateofDOOM 

And lastly; you say this was a "poorly designed" experiment..

First of all; Im gonna tell you what it was; it was 20 consecutive public matches as a solo, in which I recorded the banners of the champions, my squad, the jump trails from the early part of the flight path, and lastly the dmg and kills and placement of my own squad.... Thats it. If you think I base the experiment on conditions like "no 3k banner = player isnt good"  then you misunderstood the entire purpose. 

Second of all; At least did I base my "summery" on some kind of empirical analysis, right? You are still arguing based on your "gut feelings", an approach that I wanted to move away from. You are "vouching" that your public games have 80% predators and masters and therefore you think my empirical observation is wrong, even though it happened (with photo evidence and all).

What would be interesting was if you did an exactly same study with your supposedly much harder public game. Then we can start trying to find the variables that actually affects SBMM. 

@Balladalidila I definitely get it more often in premade but also solo.

When i don't get the mixed teams it's noticeable where i have to sit up straight and focus as it's just going to be ''one of those matches''. ie really sweaty.

I'm pred from previous series and tried solo grinding master multiple times but best i got was mid D2 too. With work and kids my effort and ability fluctuates, combining that with randoms it's just to inconsistent especially with the time constraints, for me anyway. FYI i wasn't saying i'm better or having an e-peen contest with you in anyway as it makes no difference to me.. I'm just a casual who happens to be put with competitive players often.

The champions i get are mostly premade preds more often than not and i fall down after some bad rounds into similar ones you showed, which are basically plat+ pubs so i think you're on the right track in the way you've categorised the mid to high, very high ect.

''very few Pros, few highly skilled and many good-to-average players''

Having some lobbies that aren't mixed does contradict this. It's literally the opposite to what you're saying.  I'm not going by ''gut feeling'' but personal experience having witnessed it quite frequently and also many of my friends who play at that level experience the same thing. I have no need to try and figure out SBMM or post the results from my matches but i can point out to others who are doing so they're missing out key bits of information when trying to interpret their data.  If something is wrong or missing then the conclusion is most likely flawed.

Curious, what's your intention in trying to figure out SBMM and what rumors are you trying to debunk?

@apostolateofDOOM 

" FYI i wasn't saying i'm better or having an e-peen contest with you in anyway as it makes no difference to me"

Well that wasn't my point at all. I brought it up because I, early in the discussion, got the impression that you said I was at a lower skill level than you and therefore my observation incorrect. All I wanted to say when speaking about "your and my skill level" was that my observations from my experiment maybe was applicable for MY skill level, but not for yours. It had nothing to do with some kind of contest between you and me..

"Having some lobbies that aren't mixed does contradict this. It's literally the opposite to what you're saying"

Yes, it would IF a claimed that my observations, and therefore also my conclusion, from my 20 games should be applied for every every player of all skills and for solos, premade duos and trios alike, which I dont. 

"I'm not going by ''gut feeling'' but personal experience having witnessed it quite frequently and also many of my friends who play at that level experience the same thing"

Well, tbh that is kinda what I, somewhat jokingly, is calling "gut feeling" and that was the purpose of this experiment, i.e. to move away from what I experienced (in my gut) and actually statistically document the actual stats of matches. 99% of the discussion about SBMM is based on what someone feels about their matching, with almost no actual stats to back it up.

Curious, what's your intention in trying to figure out SBMM and what rumors are you trying to debunk?

Based on the general SBMM discussion on this forum, players at ALL skill levels seems to think that:

1) they always get teamed with low level, clueless noobs (i.e. worse than themselves)

2) they always get matched against many pros, in "stacked lobbies" of multiple predator or master squads

3) If they play well one match, they get seeded in a super stacked predator game the next

So in a nutshell, players from ALL skill levels, seems to "feel in their gut"  that the SBMM generally always is unfavorable towards them, which ofc just doesn't add up. 

And dont think that I, before this experiment, wanted to disprove them because I disagreed, I wanted to examine SBMM and maybe debunk OR confirm some of them. Before this experiment, I also "felt in my gut" that I got teamed with noobs the majority of my games. So this "gut feeling of mine" turned out to be true, i.e if you are a player at my specific skill level, you will more often than not be matched with worse teammates, both on paper (i.e banner level/high dmg badge etc) and more importantly, performance (dmg dealt)

I just wanted to clarify that was the case as some people seem to get touchy on the forum feeling it's a personal attack on them when it's not.

Guess i was looking at SBMM as a whole of system point of view as i assume it would be calculated for all of us in the same way, right? Not just the way it effected you who's in a certain skill range with a limited sample and analysis to draw conclusions from.

I honestly can't be bothered to validate what i'm saying with pics and such (not that i could if i wanted to as i'm currently temp banned) but i can tell you it's not a gut feeling.

I can also probably answer those question with reasonable accuracy without needing to experiment.

1) Those that have been playing for over a month will most likely experience this often. It's balancing the teams across the server. Say for example i'm level 500 and my team mates are 100 each. That's 700 per squad that SBMM will try and calculate throughout. Other things will be taken into consideration like que times, kdr/wlr, previous matches history (thinking along the lines of EOMM) ect..

2) Not the case. It's an inevitability that lobbies fluctuate and although those of higher tier lobbies will have to contend with this more, even those cream at the top of the milk players will get easier matches just down to que time excluding multiple other thing. If you're bottom of the barrel or a new player then everyone is pro who can climb plat/diamond or smurf with 20kill.4k badge ect.. these guys have and always will exaggerate. Like the new kid that thinks he's getting beamed by aimbot every round because he's not fluid in positioning, movement, strafing, weapons, bullet drop ect ect.. they just don't grasp it due to limited experience.

3) Pred premades after a win (champion) will still get put into plat pub lobbies and that's in large part just down to que times alone. Ranked matches, as you're aware, can take considerable time and they don't allow that in pubs to anywhere near that extent so top players feed through. For players who can't climb through plat (majority) that's a punishing experience they likely feel quite often.

4) For the reasons mentioned above, i can see why a lot people think that's the case. Perhaps it's not as bad as they make out but when one player can decimated their entire team and there's a handful of players in the match like that then it's understandable.

5) Consistently playing better will put you in tougher matches (on average) but you could be on a 10 win streak and only have sub plats queuing in your region (they throw you in servers not of your choosing to reduce que time anyway) so you'll likely have sub optimal SBMM leading to the grievances of those we touched on earlier.

@apostolateofDOOM 

I dont know if I was unclear with the purpose of this experiment. It was just ADD another approach to the SBMM discussion, which is  NOT to base it on what we experience but instead look at the actual hard stats....   So you say what you say is "not gut feeling".. Well it is your gut feeling, according to my definition.. Call it experiences/thoughts then if you think gut feeling is too derogatory. We can sit and debate our experiences forever and it wont add anything to the discussion.

20 unforgiving games in a row during "prime hours", 100% transparency about me as player, champ squads, my random squads, dive trails and the squads performance. Black and white data...No  "experiences" that are, like you said and which I strongly agree on, biased by our own skill. Every 0.5 k/d player will think the aim of a 2 k/d player is predator-level or even hacking. But that is exactly why this experienced based SBMM discussion is so pointless, and that we need empirical data.

@Balladalidila  ''gut feeling'' An intuition or instinct, as opposed to an opinion based on a logical analysis. Even by your other definitions of thoughts/experiences, do you not think people have tested SBMM and unlike your small sample, pretty extensively?

Not just in the way you have with 20 matches and getting roughly equal results throughout by playing the same but testing just kills over 20 games, then accuracy, placement, losing a player consistently, losses/wins ect.. over hundreds of games to see how this influences it. Someone had to throw 15-20 games just to figure out you'll be placed in a bot lobby, another would go to a different regions at different times to know the algorithm wouldn't have enough players to pull from giving them easier matches for example.

Me posting just 20 games isn't going to provide the inner workings of SBMM, not even close.

Giving out key metrics in how it's determined would just lead to more people exploiting it because our findings and discussions to improve it have proven to be ignored by Respawn for various reasons which i fully understand but also disagree with. Many are happy to flaunt the fact they have a way to bypass protections for new or less skilled players which i know there is at least a handful of that are generally publicly known and noobbashing isn't something i want to see promoted as it's unhealthy for the game.

The likely outcome of your data and findings, if posted publicly.

@apostolateofDOOM 

Seems to me that you just like to complain and find ways to poke holes in my findings, only because you dont have anything better to come with tbh. Ever heard of straw man arguments?

I've said 2-3 times now that this experiment was just a new approach of exploring SBMM, which I think is a much more constructive way than the usual way people talk about SBMM on this forum. No one except  you are talking about finding "the inner workings of SBMM". 

It is what it is; 20 games in a row with some stats from every match documented. And you immediately goes "but its a too small sample; you have to do 100 matches with every variable defined to provide the inner workings of SBMM".....  So once again; the experiment was what it was, and this is the last time I will repeat this but you seem to forget: 20 consecutive matches when I, a player of 2.4 k/d, jumped in in public lobbies as a solo where I documented my squads banners, the champions banners, the jump trails early in the flight path, the match summery screen. 

You seem to have the very childish approach "only because your methods wasnt 100% perfect to explain SBMM, then we should dismiss this method and keep to the "gut feeling" approach" (yeah I will call it gut feeling because that is what it is) . And, btw, thats how science usually work; small steps. I even stated in the summary that I didnt want to draw to many conclusion since so many other variables could not be considered, and that the only pattern I really could see was that I usually was the best player in the squad and that the visual observation of trails was pretty consistant. 

I will always think my approach, i.e of empirical evidence even with a small sample as 20 games with those stats I choose to document,  is a better way to discuss SBMM, compared tot how its discussed on this forum generally. You dont seem to think that so lets agree to disagree. 

@Balladalidila  I'll break this down as you appear to be going round in circles.

''I wanted, mostly for my own sake, see if I could debunk some of the "rumors" about SBMM''

To answer, from the horses mouth so to speak.

Based on the general SBMM discussion on this forum, players at ALL skill levels seems to think that:

1) they always get teamed with low level, clueless noobs (i.e. worse than themselves)

2) they always get matched against many pros, in "stacked lobbies" of multiple predator or master squads

3) If they play well one match, they get seeded in a super stacked predator game the next

After i explained why you won't be able to answer all those question with the results you have shown due to inadequate testing with a poor research method using such a tiny sample within a limited skill range...

I even gave you reasonable answers to these questions.

Your findings..

''I think these observations shows some kind of proof that the SKILL-based matchmaking does NOT sort 60 players of equal skill in the same lobby, but rather tries to build lobbies of a certain player skill distribtion.

This is true but not in the way you're describing it here.

''(very few Pros, few highly skilled and many good-to-average players). It also seems its trying to match "stronger" players  in the same team with "weaker" players to make  "even" teams''''

As i explained earlier, it will balance the team across the server (in most cases leaving the solo que of higher level or skill with 2 new or inexperienced players) but not always as some, especially those in the top bracket will get full squads at their skill level  giving rise to lobbies up to 80% master/predator that you said which in your findings..

''So for those who claim they always get into lobbies with 20 preds, I dont know what you are talking about''

Now you do.

The probability of this occurring drastically increases when premade but not exclusively.

''Yes, it would IF a claimed that my observations, and therefore also my conclusion, from my 20 games should be applied for every every player of all skills and for solos, premade duos and trios alike, which I dont''

Obviously the rules will change with solo or premade but that can be clarified too and if SBMM is the same for everyone then wouldn't your findings be applicable to all?

In this case solo.

''debunk OR confirm''

You have done neither and i explained why. Except 1 (already common knowledge). 

Now you're saying ''I've said 2-3 times now that this experiment was just a new approach of exploring SBMM''

Your research is like taking a baths worth of water out the sea and saying that on average in the past 20 tubs the fish are roughly the same size to contribute towards the discussion of what the average fish size/type are in the ocean with your ''empirical evidence''..

''Ever heard of straw man arguments?''

Most of your replies, failing to address the issues i'm highlighting for you.

''Seems to me that you just like to complain and find ways to poke holes in my findings, only because you dont have anything better to come with tbh.'' 

When people post their research and findings publicly it's up to others to pick apart, scrutinise and add criticism so how it can be changed or improved upon in future tests and research. The way you're going about this is fundamentally flawed and the way you're responding to this input/criticism is less than constructive. I think you're wasting your time and i know you're wasting mine so i wish you the best of luck.

I have forwarded this thread to someone who has a reasonably good understanding of SBMM as we're obviously not seeing eye to eye on this so maybe a third party can shed light on it, even if it's me missing something here.