Another disappointing mod
I was like, "Oh sweet ! From a mod challenge! If speed rolls once I'm happy! Wut I get ? 5 protection% rolls.
Forum Discussion
not a stats person to know how big of a sample I need and too lazy to figure it out.
To answer this part specifically there is no sample size that is too small. This is a common misconception. You can only have a sample size that is "too small to show that the hypothesis .... can be rejected with a confidence level of ...."
To show that the drop rates in this game are not equal to 99.99999% can be done with a sample size of about 5 sims with an extremely high confidence level. To show that the drop rate in this game is different from 33% by more than 5% with a confidence level over 99% you need about 500 sims. To show that the drop rate in this game is different from 33% by more than 1% with a confidence level over 99% you probably need at least 10,000 sims.
That is, if your given your aim is to reject the hypothesis that the drop rate is 33%. Things are different if you just wish to report the value that you measured with appropriate confidence intervals in which case the confidence interval is dependent on sample size and other assumptions made.
