Muhammad Imdad Ullah - Statistics for Data Analyst

Student t-test Comparison Test (2015)

Jun 15, 2024Sep 20, 2015 by Muhammad Imdad Ullah

In 1908, William Sealy Gosset published his work under the pseudonym “Student” to solve problems associated with inference based on sample(s) drawn from a normally distributed population when the population standard deviation is unknown. He developed the Student t-test and t-distribution, which can be used to compare two small sets of quantitative data collected independently of one another, in this case, this t-test is called independent samples t-test or also called unpaired samples t-test.

The Student t-test is the most commonly used statistical technique in testing of hypothesis based on the difference between sample means. The student t-test can be computed just by knowing the means, standard deviations, and number of data points in both samples by using the following formula

\[t=\frac{\overline{X}_1-\overline{X}_2 }{\sqrt{s_p^2 (\frac{1}{n_1}+\frac{1}{n_2})}}\]

where $s_p^2$ is the pooled (combined) variance and can be computed as

\[s_p^2=\frac{(n_1-1)s_1^2 + (n_2-2)s_2^2}{n_1+n_2-2}\]

Using this test statistic, we test the null hypothesis $H_0:\mu_1=\mu_2$ which means that both samples came from the same population under the given “level of significance” or “level of risk”.

If the computed t-statistics from the above formula is greater than the critical value (value from t-table with $n_1+n_2-2$ degrees of freedom and given a level of significance, say $\alpha=0.05$), the null hypothesis will be rejected, otherwise, the null hypothesis will be accepted.

Note that the t-distribution is a family of curves depending on the degree of freedom (the number of independent observations in the sample minus the number of parameters). As the sample size increases, the t-distribution approaches a bell shape i.e. normal distribution.

Student t-test Example

The production manager wants to compare the number of defective products produced on the day shift with the number on the afternoon shift. A sample of the production from 6-day and 8-afternoon shifts revealed the following defects. The production manager wants to check at the 0.05 significance level, is there a significant difference in the mean number of defects per shits?

Day shift	5	8	7	6	9	7
Afternoon Shit	8	10	7	11	9	12	14	9

Some required calculations for the Student t-test are:

The mean of samples:

$\overline{X}_1=7$, $\overline{X}_2=10$,

Standard Deviation of samples

$s_1=1.4142$, $s_2=2.2678$ and $s_p^2=\frac{(6-1) (1.4142)^2+(8-1)(2.2678)^2}{6+8-2}=3.8333$

Step 1: Null and alternative hypothesis are: $H_0:\mu_1=\mu_2$ vs $H_1:\mu_1 \ne \mu_2$

Step 2: Level of significance: $\alpha=0.05$

Step 3: Test Statistics

$\begin{aligned}
t&=\frac{\overline{X}_1-\overline{X}_2 }{\sqrt{s_p^2 (\frac{1}{n_1}+\frac{1}{n_2})}}\\
&=\frac{7-10}{\sqrt{3.8333(\frac{1}{6}+\frac{1}{8})}}=-2.837
\end{aligned}$

Step 4: Critical value or rejection region (Reject $H_0$ if the absolute value of t-calculated in step 3 is greater than the absolute table value i.e. $|t_{calculated}|\ge t_{tabulated}|$). In this example t-tabulated is -2.179 with 12 degrees of freedom at a significance level of 5%.

Step 5: Conclusion: As computed value $|2.837| > |2.179|$, the number of defects is not the same on the two shifts.

Different Types of Comparison Tests

Independent Samples t-test: This compares the means of two independent groups. For example, you might use this to see if a new fertilizer increases plant growth compared to a control group.
Paired Samples t-test: This compares the means from the same group at different times or under various conditions. Imagine testing the same group’s performance on a task before and after training.
One-Sample t-test: This compares the mean of a single group to a hypothesized value. For instance, you could use this to see if students’ average exam scores significantly differ from 75%.

The summary of key differences between the comparison tests

	Independent Samples	Paired Samples	One-Sample
Groups	Independent	Same group at different times	Single group
Hypothesis	Means are different	Means are different	Mean is different from a hypothesized value
Assumptions	Normally distributed data, equal variances (testable)	Normally distributed differences	Normally distributed data

Regardless of the type of t-test, all the above comparison tests assess the significance of a difference between means. These tests tell the research if the observed difference is likely due to random chance or reflects a true underlying difference in the populations.

https://rfaqs.com

https://gmstat.com

The sum of Squared Deviations from Mean (2015)

Jun 25, 2024Sep 11, 2015 by Muhammad Imdad Ullah

Introduction of Sum Square Deviations

In statistics, the sum of squared deviations (also known as the sum of squares) is a measure of the total variability (Measure of spread or variation) within a data set. In other words, the sum of squares is a measure of deviation or variation from the mean (average) value of the given data set.

Computation of Sum of Squared Deviations

A sum of squares is calculated by first computing the differences between each data point (observation) and the mean of the data set, i.e. $x=X-\overline{X}$. The computed $x$ is known as the deviation score for the given data set. Squaring each of these deviation scores and then adding these squared deviation scores gave us the sum of squared deviation (SS), which is represented mathematically as

\[SS=\sum(x^2)=\sum(X-\overline{X})^2\]

Note that the small letter $x$ usually represents the deviation of each observation from the mean value, while the capital letter $X$ represents the variable of interest in statistics.

The Sum of Squared Deviations Example

Consider the following data set {5, 6, 7, 10, 12}. To compute the sum of squares of this data set, follow these steps

Calculate the average of the given data by summing all the values in the data set and then divide this sum of numbers by the total number of observations in the data set. Mathematically, it is $\frac{\sum X_i}{n}=\frac{40}{5}=8$, where 40 is the sum of all numbers $5+6+7+10+12$ and there are 5 observations in number.
Calculate the difference of each observation in the data set from the average computed in step 1, for the given data. The differences are
$5 – 8 = –3$; $6 – 8 = –2$; $7 – 8 = –1$; $10 – 8 =2$ and $12 – 8 = 4$
Note that the sum of these differences should be zero. $(–3 + –2 + –1 + 2 +4 = 0)$
Now square each of the differences obtained in step 2. The square of these differences are
9, 4, 1, 4 and 16
Now add the squared number obtained in step 3. The sum of these squared quantities will be $9 + 4 + 1 + 4 + 16 = 34$, which is the sum of the square of the given data set.

Sums of Squares in Different Context

In statistics, the sum of squares occurs in different contexts such as

Partitioning of Variance (Partition of Sums of Squares)
The sum of Squared Deviations (Least Squares)
The sum of Squared Differences (Mean Squared Error)
The sum of Squared Error (Residual Sum of Squares)
The sum of Squares due to Lack of Fit (Lack of Fit Sum of Squares)
The sum of Squares for Model Predictions (Explained Sum of Squares)
The sum of Squares for Observations (Total Sum of Squares)
The sum of Squared Deviation (Squared Deviations)
Modeling involving the Sum of Squares (Analysis of Variance)
Multivariate Generalization of Sum of Square (Multivariate Analysis of Variance)

As previously discussed, the Sum of Squares is a measure of the Total Variability of a set of scores around a specific number.

Summary

A higher sum of squares indicates that your data points are further away from the mean on average, signifying greater spread or variability in the data. Conversely, a lower sum of squares suggests the data points are clustered closer to the mean, indicating less variability.
The sum of squares plays a crucial role in calculating other important statistics like variance and standard deviation. These concepts help us understand the distribution of data and make comparisons between different datasets.

Online MCQs Test Website

R Faqs

Randomized Complete Block Design (RCBD)

Apr 12, 2024Aug 25, 2015 by Muhammad Imdad Ullah

The Randomized Complete Block Design may be defined as the design in which the experimental material is divided into blocks/groups of homogeneous experimental units (experimental units have same characteristics) and each block/group contains a complete set of treatments which are assigned at random to the experimental units.

In Randomized Complete Design (CRD), there is no restriction on the allocation of the treatments to experimental units. But in practical life there are situations where there is relatively large variability in the experimental material, it is possible to make blocks (in a simpler sense groups) of the relatively homogeneous experimental material or units. The design applied in such situations is called a Randomized Complete Block Design (RCBD).

Randomized Complete Block Design

RCBD is a one-restriction design, used to control a variable that influences the response variable. The main aim of the restriction is to control the variable causing the variability in response. Efforts of blocking are made to create a situation of homogeneity within the block. Blocking is a source of variability. An example of a blocking factor might be the gender of a patient (by blocking on gender), this is a source of variability controlled for, leading to greater accuracy. RCBD is a mixed model in which one factor is fixed and the other is random. The main assumption of the design is that there is no contact between the treatment and block effect.

Randomized Complete Block design is said to be a complete design because in this design the experimental units and number of treatments are equal. Each treatment occurs in each block.

The general model is defined as

\[Y_{ij}=\mu+\eta_i+\xi_j+e_{ij}\]

where $i=1,2,3\cdots, t$ and $j=1,2,\cdots, b$ with $t$ treatments and $b$ blocks. $\mu$ is the overall mean based on all observations, $\eta_i$ is the effect of the ith treatment response, $\xi$ is the effect of the jth block, and $e_{ij}$ is the corresponding error term which is assumed to be independent and normally distributed with mean zero and constant variance.

The main objective of blocking is to reduce the variability among experimental units within a block as much as possible and to maximize the variation among blocks; the design would not contribute to improving the precision in detecting treatment differences.

Randomized Complete Block Design Experimental Layout

Suppose there are $t$ treatments and $r$ blocks in a randomized complete block design, then each block contains homogeneous plots for one of each treatment. An experimental layout for such a design using four treatments in three blocks is as follows.

Block 1	Block 2	Block 3
A	B	C
B	C	D
C	D	A
D	A	B

From the RCBD layout, we can see that

The treatments are assigned at random within blocks of adjacent subjects and each of the treatments appears once in a block.
The number of blocks represents the number of replications
Any treatment can be adjacent to any other treatment, but not to the same treatment within the block.
Variation in an experiment is controlled by accounting for spatial effects.

MCQs about Sequence and Series (Intermediate Math Part – 1)

Learn R Programming

Covariance and Correlation (2015)

Aug 2, 2024Aug 20, 2015 by Muhammad Imdad Ullah

Introduction to Covariance and Correlation

Covariance and correlation are very important terminologies in statistics. Covariance measures the degree to which two variables co-vary (i.e. vary/change together). If the greater values of one variable (say, $X_i$) correspond with the greater values of the other variable (say, $X_j$), i.e. if the variables tend to show similar behavior, then the covariance between two variables ($X_i$, $X_j$) will be positive.

Similarly, if the smaller values of one variable correspond with the smaller values of the other variable, then the covariance between two variables will be positive. In contrast, if the greater values of one variable (say, $X_i$) mainly correspond to the smaller values of the other variables (say, $X_j$), i.e. both of the variables tend to show opposite behavior, then the covariance will be negative.

In other words, positive covariance between two variables means they (both of the variables) vary/change together in the same direction relative to their expected values (averages). It means that if one variable moves above its average value, the other variable tends to be above its average value.

Similarly, if covariance is negative between the two variables, then one variable tends to be above its expected value, while the other variable tends to be below its expected value. If covariance is zero then it means that there is no linear dependency between the two variables.

Mathematical Representation of Covariance

Mathematically covariance between two random variables $X_i$ and $X_j$ can be represented as
\[COV(X_i, X_j)=E[(X_i-\mu_i)(X_j-\mu_j)]\]
where
$\mu_i=E(X_i)$ is the average of the first variable
$\mu_j=E(X_j)$ is the average of the second variable

\begin{aligned}
COV(X_i, X_j)&=E[(X_i-\mu_i)(X_j-\mu_j)]\\
&=E[X_i X_j – X_i E(X_j)-X_j E(X_i)+E(X_i)E(X_j)]\\
&=E(X_i X_j)-E(X_i)E(X_j) – E(X_j)E(X_i)+E(X_i)E(X_j)\\
&=E(X_i X_j)-E(X_i)E(X_j)
\end{aligned}

Note that, the covariance of a random variable with itself is the variance of the random variable, i.e. $COV(X_i, X_i)=VAR(X)$. If $X_i$ and $X_j$ are independent, then $E(X_i X_j)=E(X_i)E(X_j)$ and $COV(X_i, X_j)=E(X_i X_j)-E(X_i) E(X_j)=0$.

Covariance and Correlation

Correlation and covariance are related measures but not equivalent statistical measures.

Equation of Correlation (Normalized Covariance

The correlation between two variables (Let, $X_i$ and $X_j$) is their normalized covariance, defined as
\begin{aligned}
\rho_{i,j}&=\frac{E[(X_i-\mu_i)(X_j-\mu_j)]}{\sigma_i \sigma_j}\\
&=\frac{n \sum XY – \sum X \sum Y}{\sqrt{(n \sum X^2 -(\sum X)^2)(n \sum Y^2 – (\sum Y)^2)}}
\end{aligned}
where $\sigma_i$ is the standard deviation of $X_i$ and $\sigma_j$ is the standard deviation of $X_j$.

Note that correlation is dimensionless, i.e. a number that is free of the measurement unit and its values lie between -1 and +1 inclusive. In contrast, covariance has a unit of measure–the product of the units of two variables.

For further reading about Correlation follow these posts

R Frequently Asked Questions