p-value interpretation, definition, introduction and examples
The p-value also known as observed level of significance or exact level of significance or the exact probability of committing a type-I error (probability of rejecting H0, when it is true), helps to determine the significance of results from hypothesis. The p-value is the probability of obtaining the observed sample results or a more extreme result when the null hypothesis (a statement about population) is actually true.
In technical words, one can define p-value as the lowest level of significance at which a null hypothesis can be rejected. If p-value is very small or less than the threshold value (chosen level of significance), then the observed data is considered as inconsistent with the assumption that the null hypothesis is true and thus null hypothesis must be rejected while the alternative hypothesis should be accepted. The p-value is a number between 0 and 1 and in literature it is usually interpreted in the following way:
- A small p-value (<0.05) indicates strong evidence against the null hypothesis
- A large p-value (>0.05) indicates weak evidence against the null hypothesis.
- p-value very close to the cutoff (say 0.05) are considered to be marginal.
Let the p-value of a certain test statistic is 0.002 then it means that the probability of committing a type-I error (making a wrong decision) is about 0.2 percent, that is only about 2 in 1,000. For a given sample size, as | t | (or any test statistic) increases the p-value decreases, so one can reject the null hypothesis with increasing confidence.
Fixing the level of significance ($\alpha$) (i.e. type-I error) equal to the p-value of a test statistic then there is no conflict between the two values, in other words, it is better to give up fixing up (significance level) arbitrary at some level of significance such as (5%, 10% etc.) and simply choose the p-value of the test statistic. For example, if the p-value of test statistic is about 0.145 then one can reject the null hypothesis at this exact significance level as nothing wrong with taking a chance of being wrong 14.5% of the time of someone reject the null hypothesis.
p-value addresses only one question: how likely are your data, assuming a true null hypothesis? It does not measure support for the alternative hypothesis.
p-value is usually incorrectly interpreted as it is usually interpreted as the probability of making a mistake by rejecting a true null hypothesis (a Type-I error). p-value cannot be error rate because:
p-value is calculated based on the assumption that the null hypothesis is true and that the difference in the sample by random chances. Consequently, p-value cannot tell about the probability that the null hypothesis is true or false because it is 100% true from the perspective of the calculations.
Download pdf file: