Cohen Effect Size and Statistical Significance

Statistical significance is important but not the most important consideration in evaluating the results. Because statistical significance tells only the likelihood (probability) that the observed results are due to chance alone. Considering the effect size when obtaining statistically significant results is important.

Effect size is a quantitative measure of some phenomenon. For example,

Correlation between two variables
The regression coefficients ($\beta_0, \beta_1, \beta_2$) for the regression model, for example, coefficients $\beta_1, \beta_2, \cdots$
The mean difference between two or more groups
The risk with which something happens

The effect size plays an important role in power analysis, sample size planning, and meta-analysis.

Since effect size indicates how strong (or important) our results are. Therefore, when you are reporting results about statistical significance for an inferential test, the effect size should also be reported.

For the difference in means, the pooled standard deviation (also called combined standard deviation, obtained from pooled variance) is used to indicate the effect size.

The Cohen Effect Size for the Difference in Means

The effect size ($d$) for the difference in means by Cohen is

$d=\frac{mean\, of\, group\,1 – mean\,of\,group\,2}{SD_{pooled}}$

Cohen provided rough guidelines for interpreting the effect size.

If $d=0.2$, the effect size will be considered as small.

For $d=0.5$, the effect size will be medium.

and if $d=0.8$, the effect size is considered as large.

Note that statistical significance is not the same as the effect size. The statistical significance tells how likely it is that the result is due to chance, while effect size tells how important the result is.

Also note that the statistical-significance is not equal to economic, human, or scientific significance.

For the effect size of the dependent sample $t$-test, see the post-effect size for the dependent sample t-test