Effect Size and Statistical Significance
Statistical significance is important but not only the most important consideration in evaluating the results. Because statistical significance tells only the likelihood (probability) that the observed results are due to chance alone. It is important to consider the effect size when you obtain statistically significant results.
Effect size is a quantitative measure of some phenomenon. For example,
- Correlation between two variables
- The regression coefficients ($\beta_0, \beta_1, \beta_2$) for the regression model, for example, coefficients $\beta_1, \beta_2, \cdots$
- The mean difference between two or more groups
- The risk with which something happens
The effect size play an important role in power analysis, sample size planning and in meta-analysis.
Since effect size is an indicator of how strong (or how important) our results are. Therefore, when you are reporting results about statistical significant for an inferential test, the effect size should also be reported.
For the difference in means, the pooled standard deviation (also called combined standard deviation, obtained from pooled variance) is used to indicate the effect size. The effect size ($d$) for the difference in means by Cohens’s is
$d=\frac{mean\, of\, group\,1 – mean\,of\,group\,2}{SD_{pooled}}$
Cohen’s provided the rough guidelines for interpreting the effect size.
If $d=0.2$ the effect size will be considered as small.
For $d=0.5$ the effect size will be medium.
and if $d=0.8$ the effect size is considered as large.
Note that statistical significance is not the same as the effect size. The statistical significance tells how likely it is that the
Also note that the statistical significance is not equal to economic, human, or scientific significance.
For effect size of dependent sample $t$-test, see the post effect size for dependent sample t-test
See the a short video on Effect Size and Statistical Significance
