Effect Size for Dependent Sample t test
An effect size is a measure of the strength of a phenomenon, conveying the estimated magnitude of a relationship without making any statement about the true relationship. Effect size measure(s) play an important role in meta-analysis and statistical power analyses. So reporting effect size in thesis, reports or research reports can be considered as a good practice, especially when presenting some empirical results/ findings, because it measures the practical importance of a significant finding. In a simple way, we can say that effect size is a way of quantifying the size of the difference between the two groups.
Effect size is usually computed after rejecting the null hypothesis in statistical hypothesis testing procedure. So if the null hypothesis is not rejected (i.e. accepted) then effect size has little meaning.
There are different formulas for different statistical tests to measure the effect size. In general, the effect size can be computed in two ways.
- As the standardized difference between two means
- As the effect size correlation (correlation between the independent variables classification and the individual scores on the dependent variable).
The effect size for dependent sample t-test
The effect size of paired sample t-test (dependent sample t-test) known as Cohen’s d (effect size) ranging from $-\infty$ to $\infty$ evaluated the degree measured in standard deviation units that the mean of the difference scores is equal to zero. If the value of d equals 0, then it means that the difference scores is equal to zero. However larger the d value from 0, more is the effect size.
The effect size for dependent sample t-test can be computed by using
\[d=\frac{\overline{D}-\mu_D}{SD_D}\]
Note that both the Pooled Mean (D) and standard deviation are reported in SPSS output under paired differences.
Let the effect size, d = 2.56 which means that the sample means difference and the population mean difference is 2.56 standard deviations apart. The sign has no effect on the size of an effect i.e. -2.56 and 2.56 are equivalent effect sizes.
The d statistics can also be computed from obtained t value and the number of paired observation by Ray and Shadish’s (1996) such as
\[d=\frac{t}{\sqrt{N}}\]
The value of d is usually categorized as small, medium and large. With cohen’s d:
- d=0.2 to 0.5 small effect
- d=0.5 to 0.8, medium effect
- d= 0.8 and higher, large effect.
Computing Effect Size from $R^2$
Another method of computing the effect size is with r-squared ($r^2$), i.e.
\[r^2=\frac{t^2}{t^2+df}\]
It can be categorized in small, medium and large effect as
- $r^2=0.01$, small effect
- $r^2=0.09$, medium effect
- $r^2=0.25$, large effect.
The non‐significant results of t-test indicate that we fail to reject the hypothesis that the two conditions have equal means in the population. The larger the value of $r^2$ indicates the larger effect (effect size), while a large effect size with a non‐significant result suggests that the study should be replicated with a larger sample size.
So larger the value of effect size computed from either method indicates a very large effect, meaning that means are likely very different.
References:
- Ray, J. W., & Shadish, W. R. (1996). How interchangeable are different estimators of effect size? Journal of Consulting and Clinical Psychology, 64, 1316-1325. (see also “Correction to Ray and Shadish (1996)”, Journal of Consulting and Clinical Psychology, 66, 532, 1998)
- Kelley, Ken; Preacher, Kristopher J. (2012). “On Effect Size”. Psychological Methods 17 (2): 137–152. doi:10.1037/a0028086.
