Significance Level

The significance level (AKA p-value) is the probability of obtaining a sample statistic (e.g., mean, proportion, etc) with a particular value or one that is more extreme, given that the null hypothesis is true. The calculated significance level is used when deciding to retain or reject the null hypothesis. When the significance level, or p-value, reduces, it becomes increasingly unlikely that the sample was drawn from the population where the null hypothesis is true. Accordingly, when the p-value falls below a particular value (typically, 0.05, which is termed the α-level) the null hypothesis is rejected and the alternative hypothesis accepted. When this happens we say the finding is significant. However, a more informative way of reporting a significance level is as follows:

1         >=  p-value >= 0.1      :- little evidence against the null hypothesis
0.1       >  p-value >= 0.05    :- weak evidence against the null hypothesis
0.05    > p-value >= 0.01      :- moderate evidence against the null hypothesis
0.01    > p-value >= 0.001    :- strong evidence against the null hypothesis
0.001 > p-value >= 0             :- very strong evidence against the null hypothesis

The value of the significance level will vary depending whether a directional or non-direction hypothesis is being evaluated. When the null hypothesis is rejected, there is always a remote chance that the null hypothesis was in fact true and a mistake has been made, that is, the null hypothesis has been falsely rejected, and the α-value indicates the likelihood of this, as this is the threshold used for making the decision.