Kickstarter Stats 101: The Meaning of Statistical “Significance”

I will use the word “significant” so many times in this blog that you will become sick of it! But let me take a second to explain a little more about this particular word, and what statisticians really mean when they use it.

Oversimplifying  "Significance"

For the sake of simplicity and time, I will overgeneralize the definition. When I say significant, I mean that it is highly unlikely that something occurred by chance alone. How unlikely? Usually it is assumed that there is at most a 5% chance that the result could have occurred by randomness alone.

So, when a statistician says that the difference between two variables (for example, the difference between the success rates of projects launched during January versus July) is “significant”, they mean that this difference is very unlikely to have occurred by chance. More specifically, they reject the hypothesis that this difference was due to chance alone!

To determine whether something is significant or not, we perform a statistical test. There are many types of tests, but they are all trying to achieve this similar goal – determining how likely it is that the results would have occurred by chance alone. More specifically, the probability that this could have occurred by chance alone. If this probability is less than say… 5% (or some other pre-established threshold) then we say that the result is significant, because something is influencing it other than randomness or chance.

Since we reject the hypothesis that the differences we see are due to chance, we by default accept the hypothesis that the differences we see are due to some type of influence - something is influencing the data and causing the difference that we see.

Can we be 100% Certain? 

Can we prove this? Well no, but at some level we can’t prove anything with 100% certainty.

We can, however, show that most other plausible options are so statistically improbable, that we are left to conclude that the variables we do have at hand are the major influencers.

Man I love Statistics!

Please feel free to add to this conversation by commenting below!