The P-value is a probability, with a value ranging from zero to one. It is measure of how much evidence we have against the null hypothesis. P-value is a way to express the likelihood that $H_0$ is not true. The smaller the p-value, the more evidence we have against $H_0$.
p-value can be defined as
The largest significance level at which we would accept the null hypothesis. It enables us to test hypothesis without first specifying a value for $\alpha$. OR
The probability of observing a sample value as extreme as, or more extreme than, the value observed, given that the null hypothesis is true.
If the P-value is smaller then the chosen significance level then $H_0$ (null hypothesis) is rejected even when it is true. If it is larger than the significance level $H_0$ is not rejected.
If the P-value is less than
- 0.10, we have some evidence that $H_0$ is not true
- 0.05, strong evidence that $H_0$ is not true
- 0.01, Very strong evidence that $H_0$ is not true
- 0.001, extremely strong evidence that $H_0$ is not true
Misinterpretation of a P-value
Many people misunderstand P-values. For example, if the P-value is 0.03 then it means that there is a 3% chance of observing a difference as large as you observed even if the two population means are same (i.e. the null hypothesis is true). It is tempting to conclude, therefore, that there is a 97% chance that the difference you observed reflects a real difference between populations and a 3% chance that the difference is due to chance. However, this would be an incorrect conclusion. What you can say is that random sampling from identical populations would lead to a difference smaller than you observed in 97% of experiments and larger than you observed in 3% of experiments.
Download pdf file: