Drop rates, loss runs and transparency

This thread is great. Keep up the good work.

It's like when I was getting pretty angry about the lack of slave 1 shards. Until I realized that with a rotation chance of 17%, there is actually a 2% chance of going without shards for 2 weeks. So it wasn't that strange to have a long dry spell like that. Still made me upset ;) Just as my 9 sims I just did with 0 stun gun salvage is frustrating.

Btw, no real reason to assume the 17% rotation rate, except it seemed that tie fighter appeared about twice as often as anything else, so splitting the chance of scimitar/slave 1 down the middle gives 17%.

"Acrofales;864793" wrote:
This thread is great. Keep up the good work.
Btw, no real reason to assume the 17% rotation rate, except it seemed that tie fighter appeared about twice as often as anything else, so splitting the chance of scimitar/slave 1 down the middle gives 17%.

Yeah, we need to collect more data to get a bette idea - that 17% could be actually be between 5% and 40% depending on sample size. Collecting the data is a pain to do though.

Bollocks. Those 9 runs without salvage I was talking about? They just become 15.

But to put that in perspective, assuming a 0.2 chance of a drop, there's still a 3.5% chance of such a run happening, and with the amount of sims we all do for gear, such a run is bound to happen. And be noticeable. Whereas the almost equivalent chance (still assuming a 0.2 chance) of two salvage dropping in a row (4%) seems completely unremarkable.

Psychology sucks :P

E: actually, I'm not quite sure they are equivalent. My statistics is a bit shakey here, but given a long chain of actions you'd probably also have to take into account how many sequences may occur. And there are far far far more 2-sequences than 15-sequences. So starting from 0, your chance of hitting 2 salvages in a row is only slightly higher than higher than hitting 15 shots of nada, but in a long chain of actions, it should be easier to find a sequence (any sequence) of 2 salvages in a row than a 15-sequence of nothing?

If I got this right, then given a chain of X actions, the number of 2-sequences are X - 1, whereas the number of 15-sequences is X-15. So your chance of finding a 2-sequence is: 1 - 0.96^(X - 1), whereas your chance of finding a 15-sequence of no salvage is 1 - 0.965^(X-15). So it clearly depends on the size of X. If X is large enough, then the influence of X is negligible (the chance trends to 1, of course, in the infinite), but if we look for something like X=100, then we have a 98% chance of getting a 2-sequence with 2 stun gun salvages, and a 95% of having any 15-sequence with 0 salvages, which is a noticeable difference (albeit not really to the human running those 100 sims, because human perception of chance sucks).

"Acrofales;864863" wrote:
Bollocks. Those 9 runs without salvage I was talking about? They just become 15.
E: actually, I'm not quite sure they are equivalent. My statistics is a bit shakey here, but given a long chain of actions you'd probably also have to take into account how many sequences may occur. And there are far far far more 2-sequences than 15-sequences. So starting from 0, your chance of hitting 2 salvages in a row is only slightly higher than higher than hitting 15 shots of nada, but in a long chain of actions, it should be easier to find a sequence (any sequence) of 2 salvages in a row than a 15-sequence of nothing?

The sequence of win/loss trials shouldn't 'remember' previous trials, so the probability of an arbitrary number of losses before a wins should be a geometrically distributed random variable (and so far as I can tell, both loss and win run distributions are distributed this way).

"Acrofales;864863" wrote:

If I got this right, then given a chain of X actions, the number of 2-sequences are X - 1, whereas the number of 15-sequences is X-15. So your chance of finding a 2-sequence is: 1 - 0.96^(X - 1), whereas your chance of finding a 15-sequence of no salvage is 1 - 0.965^(X-15). So it clearly depends on the size of X. If X is large enough, then the influence of X is negligible (the chance trends to 1, of course, in the infinite), but if we look for something like X=100, then we have a 98% chance of getting a 2-sequence with 2 stun gun salvages, and a 95% of having any 15-sequence with 0 salvages, which is a noticeable difference (albeit not really to the human running those 100 sims, because human perception of chance sucks).


If the drop rate is denoted p and the probability of missing a drop is 1 - p then the probability of 2 losses before a win is just:

(1 - p) * (1 - p) * p 

If the drop rate is 0.2, then that gives a 0.128 probability of two losses before a win (three trials in all).

The probability of 15 losses before a win is

(1 - p)^15 * p = 0.007036874

- not very likely at all.

Give it some thought (or check the wiki page) and you can work backwards from probability to loss runs. I think the most interesting thing to monitor is the expected maximum loss sequence for an arbitrary number of runs -- if you think about it, the expected maximum loss sequence is more useful to know ahead of time.

"kobe_juan_canobie;88883" wrote:
None of what follows is new, but I hope it encourages EA to have a think about how they handle artificially stochastic functions in the game, and gives some of you a bit more info about how to gain dropped gear strategically.

People spend money on this game. It's like a strategy game with gambling attached -- eg. shard and gear drop battles. Unlike reputable gambling sites however SWGOH does not tell you the probability of a win, or even make the drops Provably fair. An example of a provably fair online gambling site which provides clear odds and the house edge is the well known cryptocoin casino Just Dice

So, because of this lack of transparency I see lots of posts about unfair drop rates. This is probably partly due to an unknown drop rate, cognitive bias and unfamilirarity with expected win and loss runs in a set of geometrically distributed trials. It takes a great many trials to estimate a drop rate with reasonable accuracy, and even at higher drop rates you can hit quite long runs of losses - for example for a drop rate of 0.2 and in a series of a hundred wins you can expect your longest run of losses (before a win) to be between 14 and 37 (some easy papers on run length distributions here and here).

Those loss runs are pretty long and could be disheartening -- and experienced by a large number of players, given the amount of gear with drop rates of 0.2.

I think people either assume that the drop rates are much worse than they are, or believe that EA are playing silly buggers and varying the drop rates (eg giving a large number of successes just before you run out of crystals, and then making up for it with a large number of losses just after you purchase them). These beliefs are all probably due to cognitive bias and lack of transparency, but EA leave themselves open to accusations of cheating by not using any of available methods of proving fairness and by not providing accurate drop rates.

As my guild (BaseDeltaZero rocks wooo!) knows I have a new baby and my brain is reeling. I don't have time or brain power to do the usual sorts of non-swgoh analysis I did pre-baby at my blog, but I have just about enough IQ left to post an analysis of my drop rates, loss runs and win runs for the past fortnight. I also checked that in each case distribution of number of losses before a win fit the expected geometric distribution well enough to not reject that assumption as the null, and in each case they did. From this you could just as easily believe that EA is running a fair game, or they're being very subtle about cheating us. Make no mistake - our disbelief is their fault.

I'll post my actual results in other posts - this one is too long if I include them.

+1 So eloquently said. Great op

"kobe_juan_canobie;864913" wrote:

"Acrofales;864863" wrote:
Bollocks. Those 9 runs without salvage I was talking about? They just become 15.
E: actually, I'm not quite sure they are equivalent. My statistics is a bit shakey here, but given a long chain of actions you'd probably also have to take into account how many sequences may occur. And there are far far far more 2-sequences than 15-sequences. So starting from 0, your chance of hitting 2 salvages in a row is only slightly higher than higher than hitting 15 shots of nada, but in a long chain of actions, it should be easier to find a sequence (any sequence) of 2 salvages in a row than a 15-sequence of nothing?

The sequence of win/loss trials shouldn't 'remember' previous trials, so the probability of an arbitrary number of losses before a wins should be a geometrically distributed random variable (and so far as I can tell, both loss and win run distributions are distributed this way).

"Acrofales;864863" wrote:

If I got this right, then given a chain of X actions, the number of 2-sequences are X - 1, whereas the number of 15-sequences is X-15. So your chance of finding a 2-sequence is: 1 - 0.96^(X - 1), whereas your chance of finding a 15-sequence of no salvage is 1 - 0.965^(X-15). So it clearly depends on the size of X. If X is large enough, then the influence of X is negligible (the chance trends to 1, of course, in the infinite), but if we look for something like X=100, then we have a 98% chance of getting a 2-sequence with 2 stun gun salvages, and a 95% of having any 15-sequence with 0 salvages, which is a noticeable difference (albeit not really to the human running those 100 sims, because human perception of chance sucks).

If the drop rate is denoted p and the probability of missing a drop is 1 - p then the probability of 2 losses before a win is just:

(1 - p) * (1 - p) * p 

If the drop rate is 0.2, then that gives a 0.128 probability of two losses before a win (three trials in all).

The probability of 15 losses before a win is

(1 - p)^15 * p = 0.007036874

- not very likely at all.

Give it some thought (or check the wiki page and you can work backwards from probability to loss runs. I think the most interesting thing to monitor is the expected maximum loss sequence for an arbitrary number of runs -- if you think about it, the expected maximum loss sequence is more useful to know ahead of time.

I don't think we're disagreeing, just calculating 2 different things. I was calculating (1) the probability of me going on a 15-run dry spell, and then putting it into perspective by calculating there being a dry spell of at least 15 sims in a sequence of 100 sims (pretty sure my 95% on that is correct).

I also agree that being able to compute the expected maximum loss sequence for a number of runs is far more useful (and will let you put it into perspective). I saw you did that for stun guns, and my 15 run sequence is still within the bounds.

Now I accumulated enough energy for another 3 sims. So lets go see what I get!

---

And. Zip. Nada. Squat.

So we're up to an 18-run. Which is out of bounds for maximum expected dry spell (was that a 95% confidence interval?) in 100 sims. Of course, you computed it for a sequence of 100 sims, which is an arbitrary number of sims you chose because it's a pretty round number. And I don't really know how many I have done. I stopped somewhere around 30 stun guns to do light side battles for the daily. I then got up to 46 stun guns before my dry spell started. So how do we count that sequence? In total it's clearly more than 100 sims, but since the light side battles ended, I spent 4 refreshes, so we're only up to 48 sims + natural regen (probably another 30 or so?)

"Acrofales;864969" wrote:

I don't think we're disagreeing, just calculating 2 different things. I was calculating (1) the probability of me going on a 15-run dry spell, and then putting it into perspective by calculating there being a dry spell of at least 15 sims in a sequence of 100 sims (pretty sure my 95% on that is correct).

I also agree that being able to compute the expected maximum loss sequence for a number of runs is far more useful (and will let you put it into perspective). I saw you did that for stun guns, and my 15 run sequence is still within the bounds.

Now I accumulated enough energy for another 3 sims. So lets go see what I get!

---

And. Zip. Nada. Squat.

So we're up to an 18-run. Which is out of bounds for maximum expected dry spell (was that a 95% confidence interval?) in 100 sims. Of course, you computed it for a sequence of 100 sims, which is an arbitrary number of sims you chose because it's a pretty round number. And I don't really know how many I have done. I stopped somewhere around 30 stun guns to do light side battles for the daily. I then got up to 46 stun guns before my dry spell started. So how do we count that sequence? In total it's clearly more than 100 sims, but since the light side battles ended, I spent 4 refreshes, so we're only up to 48 sims + natural regen (probably another 30 or so?)

Aha! You're counting each simulation as a sample, but i was actually measuring the number of runs rather than the number of trials. I can't think of a way to calculate run length probability in terms of trials except as it was done in one of the papers i linked to in the OP, so I really ant to know: how do you derive your equation 1 - 0.96^(X - 1)?


Here's some precomputed values for the expected maximum run length (in number of trials) for a given number of runs (not trials) and drop rates:

    drop_rate loss_runs E_max_length_loss_run
  1:      0.05        20                    68
  2:      0.10        20                    32
  3:      0.15        20                    20
  4:      0.20        20                    14
  5:      0.25        20                    11
  6:      0.30        20                     8
  7:      0.35        20                     6
  8:      0.40        20                     5
  9:      0.45        20                     4
 10:      0.50        20                     3
 11:      0.05        40                    81
 12:      0.10        40                    39
 13:      0.15        40                    24
 14:      0.20        40                    17
 15:      0.25        40                    13
 16:      0.30        40                    10
 17:      0.35        40                     8
 18:      0.40        40                     6
 19:      0.45        40                     5
 20:      0.50        40                     4
 21:      0.05        60                    89
 22:      0.10        60                    42
 23:      0.15        60                    27
 24:      0.20        60                    19
 25:      0.25        60                    14
 26:      0.30        60                    11
 27:      0.35        60                     9
 28:      0.40        60                     7
 29:      0.45        60                     6
 30:      0.50        60                     5
 31:      0.05        80                    95
 32:      0.10        80                    45
 33:      0.15        80                    29
 34:      0.20        80                    20
 35:      0.25        80                    15
 36:      0.30        80                    12
 37:      0.35        80                    10
 38:      0.40        80                     8
 39:      0.45        80                     6
 40:      0.50        80                     5
 41:      0.05       100                    99
 42:      0.10       100                    47
 43:      0.15       100                    30
 44:      0.20       100                    21
 45:      0.25       100                    16
 46:      0.30       100                    13
 47:      0.35       100                    10
 48:      0.40       100                     8
 49:      0.45       100                     7
 50:      0.50       100                     5
 51:      0.05       120                   103
 52:      0.10       120                    49
 53:      0.15       120                    31
 54:      0.20       120                    22
 55:      0.25       120                    17
 56:      0.30       120                    13
 57:      0.35       120                    10
 58:      0.40       120                     9
 59:      0.45       120                     7
 60:      0.50       120                     6
 61:      0.05       140                   106
 62:      0.10       140                    50
 63:      0.15       140                    32
 64:      0.20       140                    23
 65:      0.25       140                    17
 66:      0.30       140                    13
 67:      0.35       140                    11
 68:      0.40       140                     9
 69:      0.45       140                     7
 70:      0.50       140                     6
 71:      0.05       160                   108
 72:      0.10       160                    52
 73:      0.15       160                    33
 74:      0.20       160                    23
 75:      0.25       160                    18
 76:      0.30       160                    14
 77:      0.35       160                    11
 78:      0.40       160                     9
 79:      0.45       160                     7
 80:      0.50       160                     6
 81:      0.05       180                   111
 82:      0.10       180                    53
 83:      0.15       180                    34
 84:      0.20       180                    24
 85:      0.25       180                    18
 86:      0.30       180                    14
 87:      0.35       180                    11
 88:      0.40       180                     9
 89:      0.45       180                     8
 90:      0.50       180                     6
 91:      0.05       200                   113
 92:      0.10       200                    54
 93:      0.15       200                    34
 94:      0.20       200                    24
 95:      0.25       200                    18
 96:      0.30       200                    14
 97:      0.35       200                    12
 98:      0.40       200                    10
 99:      0.45       200                     8
100:      0.50       200                     6
     drop_rate loss_runs E_max_length_loss_run

"Silleck;864998" wrote:
From what I noticed it's extremely streaky...In all parts of the game, including drop rates

That's randomness for you - clumpy as hell. In fact, you can tell a truly random sequence of 1s and 0s from a human generated one - the human generated "random" string is usually more evenly distributed, less clumpy and fewer long runs than the really random sequences.

In short: If you think something looks random, it probably isn't. If it looks more streaky, it probably is.

I spend most of my crystals on refreshing battle energy, so that is absolutely gambling with money.

"Allenb60;864575" wrote:

Also congratulations, I wish you the best of luck

Cheers bud!



I meant to say crystals are a free currency, even if they are available for purchase

I don't think I understand your terminology.

A trial = a sim? Or 100 sims? Given what you have said so far, it seems like a trial is 100 sims, and you simply look at what is the maximum loss-run (which would be the longest series of sims within a single trial without a win?)

Monte Carlo doesn't seem like a particularly useful approach to determine this, because repeating 100-sim trials and recording the maximum is going to trend towards 100 as you repeat more and more trials. Such an empirical approach seems useful to find the average (or expected) length of a dry streak.

Anyway, the way I was doing it (and it may be wrong, I admit) is:

Assume you have a drop rate of p, then it's pretty trivial to compute the probability of getting a streak of at least X losses, it's:

P = (1 - p)^X

Now assume X <= 100, and a single trial is 100 sims. Then the probability of encountering a streak of at least length X losses in that trial depends on the number of sequences of length X you can get. In a trial of length 100, that is (100 - X). Now, the chance of none of those trials being a sequence of all losses is

(1 - P)^(100 - X)

So the chance of getting a streak, is simply the opposite, all written out in the original probabilities:

1 - (1 - (1 - p)^X)^(100 - X)

Fill in a droprate p=0.2 and a streak of X=15, and we get the abovementioned 95% probability of a trial of 100 containing a dry streak of 15 sims.

Note that in the earlier post, I was working the other way round: X was the trial size, and not the streak size.

The problem is that in reality, we do not run "trials" of 100 sims. We simply sim a lot until we get all the salvage we need. Thus we actually have:

1 - (1 - (1 - p)^X)^(T - X)

Where T is the variable trial size. What would be more useful than a trial size of 100, is to compute the average number of sims you need to get a full gear and use that as your trial size. But you'd still only have an average run. If you have "bad luck" then your piece of gear will take more sims than the average trial, and incidentally also increase the probability of a long streak of losses. However, given a drop rate p, and a number N of salvages you need, the formula becomes:

1 - (1 - (1 - p)^X)^((N/p) - X)

And (I think) that to compute the largest X within a 95 confidence interval, you simply have to do:

0.95 <= 1 - (1 - (1 - p)^X)^((N/p) - X)

And compute X given p, and N. For stun guns (N=50 and assume p=0.2):

0.05 >= (1 - (0.8)^X)^(250 - X)

I don't know how to solve this exactly, but we want integers anyway, so then it's pretty easy to numerically find the largest integer X for which this holds:

0.05 > (1 - (0.8)^X)^(250 - X)

Which is X=19. So on an average farm for stun guns, you can expect with 95% confidence a streak of at least 19 sims with no salvage dropping. That is pretty fucking horrible. I should know, I'm in it! :P

Oh, I figured I should add the plot:
https://www.wolframalpha.com/input/?i=plot+Y%3D+(1+-+(0.8)%5EX)%5E(250+-+X)+for+X+from+0+to+50

On the Y-axis, the inverted chance and on the X-axis the streak length.