Statistics for Data Science & Analytics

Understand the sampling distribution of differences between means—what it is, why it matters, and how to apply it in hypothesis testing (with examples). Perfect for students, data scientists, and analysts! Ever wondered how statisticians compare two groups (e.g., test scores, sales performance, or medical treatments)? The key lies in the sampling distribution of differences between means—a fundamental concept for hypothesis testing, confidence intervals, and A/B testing.

Sampling Distribution of Differences Between Means

The Sampling Distribution of Differences Between Means is the probability distribution of differences between two sample means (e.g., $Mean_A – Mean_B$) if you repeatedly sampled from two populations.

Let there are two populations of size $N_1$ and $N_2$ having means $\mu_1$ and $\mu_2$ with variances $\sigma_1^2$ and $\sigma_2^2$. We need to draw all possible samples of size $n_1$ from the first population and $n_2$ from the second population, with or without replacement.

Let $\overline{x}_1$ be the means/averages of samples of the first population and $\overline{x}_2$ be the means/averages of the samples of the second population. After this, we will determine all possible differences between means/averages denoted by
$$d =\overline{x}_1 – \overline{x}_2$$

We call the frequency distribution differences as frequency distribution, while the probability distribution of the differences is the sampling distribution of differences between means.

Notations for Sampling Distribution of Differences between Means

Notation	Short Description
$\mu_1$	Mean of the first population
$\mu_2$	Mean of the second population
$\sigma_1^2$	Variance of the first population
$\sigma_2^2$	Variance of the second population
$\sigma_1$	Standard deviation of the first population
$\sigma_2$	Standard deviation of the second population
$\mu_{\overline{x}_1 – \overline{x}_2}$	Mean of the sampling distribution of difference between means
$\sigma^2_{\overline{x}_1 – \overline{x}_2}$	Variance of the sampling distribution of difference between means
$\sigma_{\overline{x}_1 – \overline{x}_2}$	Standard deviation of the sampling distribution of difference between means

Some Formulas for Sampling with/without Replacement

Sr. No.	Sampling with Replacement	Sampling without Replacement
1.	$\mu_{\overline{x}_1 -\overline{x}_2} = \mu_1-\mu_2$	$\mu_{\overline{x}_1 -\overline{x}_2} = \mu_1-\mu_2$
2.	$\sigma^2_{\overline{x}_1 -\overline{x}_2}=\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}$	$\sigma^2_{\overline{x}_1 -\overline{x}_2}=\frac{\sigma_1^2}{n_1}\left(\frac{N-1-n_2}{N_1-1}\right) + \frac{\sigma_2^2}{n_2}\left(\frac{N_2-n_2}{N_2-1}\right)$
3.	$\sigma_{\overline{x}_1 -\overline{x}_2}=\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}$	$\sigma_{\overline{x}_1 -\overline{x}_2}=\sqrt{\frac{\sigma_1^2}{n_1}\left(\frac{N-1-n_2}{N_1-1}\right) + \frac{\sigma_2^2}{n_2}\left(\frac{N_2-n_2}{N_2-1}\right)}$

Example

Let $\overline{x}$ represent the mean of a sample of size $n_1=2$ selected at random with replacement from a finite population consisting of values 5, 7, and 9. Similarly, let $\overline{x}_2$ represent the mean of a sample of size $n_2=2$ selected at random from another finite population consisting of values 4, 6, and 8. Form the sampling distribution of the random variable $\overline{x}_1 – \overline{x}_2$ and verify that

• $\mu_{\overline{x}_1 – \overline{x}_2} = \mu_1 – \mu_2$
• $\sigma^2_{\overline{x}_1 – \overline{x}_2} = \frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}

Solution

Population I	Population II
5, 7, 9 $N_1=3$ $n_1=2$	4, 6, 8 $N_2=3$ $n_2=2$
Possible samples with Replacement are $N_1^{n_1}=3^2 =9$	Possible samples with Replacement are $N_2^{n_2} = 3^2 = 9$

All Possible Samples

All possible differences between samples means from both of the population is ($d=\overline{x}_1 – \overline{x}_2$).

$d=\overline{x}_1 =-\overline{x}_2$	4	5	5	6	6	6	7	7	8
5	5-4= 1	0	0	-1	-1	-1	-2	-2	-3
6	2	1	1	0	0	0	-1	-1	-2
6	2	1	1	0	0	0	-1	-1	-2
7	3	2	2	1	1	1	0	0	-1
7	3	2	2	1	1	1	0	0	-1
7	3	2	2	1	1	1	0	0	-1
8	4	3	3	2	2	2	1	1	0
8	4	3	3	2	2	2	1	1	0
9	5	4	4	3	3	3	2	2	1

The Sampling Distribution of Differences Between Means

$d=\overline{x}_1 – \overline{x}_2$	$f$	$P(d)$	$d\cdot P(d)$	$d^2$	$d^2 \cdot P(d)$
-3	1	1/81	$-3 * 1/81 = -3/81$	9	9/81
-2	4	4/81	-8/81	4	16/81
-1	10	10/81	-10/81	1	10/81
0	16	16/81	0/81	0	0/81
1	19	19/81	19/81	1	19/81
2	16	16/81	31/81	4	64/81
3	10	10/81	30/81	9	90/81
4	4	4/81	16/81	16	64/81
5	1	1/81	5/81	25	125/81
Total	81		81/81=1		297/81=3.67

\begin{align*}
\mu_{\overline{x}_1 – \overline{x}_2} &= E(d) = \Sigma(d\cdot P(d)) = \frac{81}{81}=1\\
\sigma^2_{\overline{x}_1 – \overline{x}_2} &= E(d^2) – [E(d)]^2\\
&=\Sigma d^2 P(d) – \left[\Sigma (d\cdot P(d))\right]^2\\
&= 3.67 – 1^2 = 2.67
\end{align*}

Verification

• $\mu_{\overline{x}_1 – \overline{x}_2} = \mu_1 – \mu_2 \Rightarrow 7-6 = 1$
• $\sigma_{\overline{x}_1 – \overline{x}_2}^2 = \frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2} = \frac{2.66}{2} + \frac{2.66}{2}\Rightarrow 2.66$

Sampling in R Language