Muhammad Imdad Ullah

Currently working as Assistant Professor of Statistics in Ghazi University, Dera Ghazi Khan. Completed my Ph.D. in Statistics from the Department of Statistics, Bahauddin Zakariya University, Multan, Pakistan. l like Applied Statistics, Mathematics, and Statistical Computing. Statistical and Mathematical software used is SAS, STATA, Python, GRETL, EVIEWS, R, SPSS, VBA in MS-Excel. Like to use type-setting LaTeX for composing Articles, thesis, etc.

Formal Hypothesis Test

by

A formal hypothesis test in statistics is a structured method used to determine whether there is enough evidence in a sample of data to infer that a certain condition holds for the entire population. It involves making an initial assumption (the null hypothesis) and then evaluating whether the observed data provides sufficient evidence to reject that assumption in favor of an alternative hypothesis.

Null and Alternative Hypotheses

In a formal hypothesis test, the null hypotheses are denoted by $H_o$ and the alternative hypotheses are denoted by $H_a$. The null and alternative hypotheses need to be assigned as follows:

Null Hypothesis

The null hypothesis is the hypothesis being tested. $H_o$ must

  • be the hypothesis we want to reject
  • contain the condition of equality (=, $\ge$, or $\le$)

Alternative Hypothesis

The alternative hypothesis is always the opposite of the null hypothesis, $H_o$. $H_a$ must

  • be the hypothesis we want to support
  • not contain the condition of equality (<, >, $\ne$)

A formal hypothesis test will always conclude with a decision to reject $H_o$ based on sample data or the decision that there is not strong enough evidence to reject $H_o$.

Formal Hypothesis Test, Hypothesis Testing

Components of a Formal Hypothesis Test

The following are key components of a formal hypothesis test.

  • Null Hypothesis ($H_o$)
    It is a statement of “No Effect” or “No Difference”. For example, $H_o:\mu=\mi_o$ (population mean $\mu$ equals a specified value $\mu_o$
  • Alternative Hypothesis ($H_1$)
    It is a statement that contradicts the null hypothesis. An alternative hypothesis can be one-tailed (for example, $H_1:\mu> \mu_o$, or $H_1:\mu<\mu_o$) or two-tailed (for instance, $H_1:\mu\ne\mu_o$).
  • Test Statistic (Test Formula)
  • A numerical value is calculated from sample data by using an appropriate t-statistic, z-score, f-statistic, or $\chi^2$ statistic.
  • Significance Level ($\alfha$)
    The maximum acceptable probability is typically chosen at the outset of the hypothesis test and is referred to as the level of significance or significance level for the test. The level of significance is denoted by $\alpha$, and the most commonly used values are $\alpha = 0.10, 0.05, and 0.01$.
    Note that once $\alpha$ (level of significance) is determined, the value of $\beta$ is also fixed; the probability of making a type-II error in a hypothesis test.
  • P-value
    The probability of observing the test statistic (or more extreme) if $H_o$ is true. If $p\le\alpha$, reject $H_o$; otherwise, accept it.
  • Decision Rule
    Reject $H_o$ if the test statistic falls in the critical region or if $p\le\alpha$
  • Conclusion
    State whether there is sufficient evidence to reject $H_o$ in favour of $H_1$.

Hypothetical Example: One-Sample t-test

  • Null Hypothesis: The population mean $\mu=50$
  • Alternative Hypothesis: The population mean $\mu \ne 50$ (two-tailed test)
  • Test Statistic: $t=\frac{\overline{x}-\mu}{\frac{s}{\sqrt{n}}}$, where $\overline{x}$ is the sample mean, $s4 is the sample standard deviation, $n$ is the sample size
  • Decision: if $|t| > t_{\alpha/2, n-1}$ or $p<\alpha$, reject $H_o$

Real Life Examples of Formal Hypothesis Tests

The following are a few real-life examples of formal hypothesis tests used in various fields.

  • Medical Testing (Drug Efficacy): Consider a pharmaceutical company that tests whether a new drug lowers blood pressure more effectively than a placebo. It is a real case used in clinical trials for hypertension medications. The hypotheses will be
    • $H_o$: The drug has no effect ($\mu_{drug} = \mu_{placebo}$.
    • $H_1: The drug reduces blood pressure ($\mu_{drug}<\mu_{placebo}$). It is a one-tailed test.
    • Test Statistic Used: Two-sample t-test will be used for comparing the means of two groups.
  • Social Science (Opinion Polls): Consider a pollster who tests whether support for a political party candidate differs between men and women. The hypothesis may be
    • $H_o:$: No gender difference in support ($p_{men} = p_{women}$).
    • $H_1: Support differs by gender ($p_{men}\ne p_{p_{women}$). It is a two-tailed test.
    • Test Statistic Used: Chi-Square test for independence (categorical data) will be used.
  • Economics (Policy Impact): A government tests whether a tax incentive increased small business growth. The hypotheses will be
    • $H_o$: The policy had no effect ($\mu_{after} – \mu_{before}=0$).
    • Test Statistic use: Regression analysis with a dummy variable or difference-in-differences test.
  • Business and Marketing (A/B Testing): An e-commerce company tests whether a redesigned website increases sales compared to the old version. The hypotheses will be:
    • $H_o$:The new design has no impact on sales ($p_{new}=p_{old}$)
    • $H_1$: The new design increases sales ($p_{new}>p_{old}$). It is a one-tailed test.
    • Test Statistic: For comparing conversion rates, a two-proportion z-test can be used.
  • Manufacturing (Quality Control): Suppose a factory checks if the average weight of cereal boxes meets the advertised weight of 500g. The hypotheses are:
    • $H_o$: The mean weight is 500g ($\mu=500$)
    • $H_1$: The mean weight differs from 500g ($\mu\ne 500$). It is a two-tailed test.
    • Test Statistic: A sample t-test can be used for testing against a known standard.
  • Environmental Science (Pollution Levels): Researchers are interested in testing if a river’s pollution level exceeds the safe limit (e.g., lead concentration > 15ppm). The hypotheses may be:
    • $H_o$: Mean lead concentration $\le$ 15 ppm ($\mu\le 15$)
    • $H_1$: Mean lead concentration > 15 ppm ($\mu > 15$). It is a two-tailed test.
    • Test Statistic: One-sample t-test (or non-parametric Wilcoxon test, if data is skewed) can be used
  • Education (Test Score Improvement): A school may be interested in testing whether a new teaching method improves students’ math scores. The hypothesis may be
    • $H_o$: The new method has no effect ($\mu_{after} – \mu_{before}=0$)
    • $H_1$: The new method improves scores ($\mu_{after} > \mu_{before}$). It is a one-tailed test.
    • Test Statistic: A paired sample t-test can be used.
  • Psychology (Behavioural Studies): A researcher may test whether sleep deprivation affects reaction time. The hypotheses are
    • $H_o$: Sleep deprivation has no effect ($\mu_{sleep\,deprived} > u_{normal\,sleep})
    • $H_1$: Sleep deprivation increases reaction time ($\mu_{sleep\,deprived}>\mu_{normal}$)
    • Test Statistic: An Independent two-sample t-test can be used for comparing two groups.

Exploratory Data Analysis in R

Understanding Ridge Regression

by

Discover the fundamentals of Ridge Regression, a powerful biased regression technique for handling multicollinearity and overfitting. Learn its canonical form, key differences from Lasso Regression (L1 vs L2 regularization), and why it’s essential for robust predictive modeling. Perfect for ML beginners and data scientists!

Introduction

In cases of near multicollinearity, the Ordinary Least Squares (OLS) estimator may perform worse compared to non-linear or biased estimators. For near multicollinearity, the variance of regression coefficients ($\beta$’s, where $\beta=(X’X)^{-1}X’Y$), given by $\sigma^2(X’X)^{-1}$ can be very large. While in terms of the Mean Squared Error (MSE) criterion, a biased estimator with less dispersion may be more efficient.

Ridge Regression, Bias Variance Trade off

Understanding Ridge Regression

Ridge regression (RR) is a popular biased regression technique used to address multicollinearity and overfitting in linear regression models. Unlike ordinary least squares (OLS), RR introduces a regularization term (L2 penalty) to shrink coefficients, improving model stability and generalization.

Addition of the matrix $KI_p$ (where $K$ is a scalar to $X’X$ yields a more stable matrix $(X’X+KI_p)$. The ridge estimator of $\beta$ ($(X’X+KI_p)^{-1}X’Y$) should have a smaller dispersion than the OLS estimator.

Why Use Ridge Regression

OLS regression can produce high variance when predictors are highly correlated (multicollinearity). Ridge regression helps by:

  • Reducing overfitting by penalizing large coefficients
  • Improving model stability in the presence of multicollinearity
  • Providing better predictions when data has many predictors

Canonical Form

Let $P$ denote the orthogonal matrix whose elements are the eigenvectors of $X’X$ and let $\Lambda$ be the (diagonal) matrix containing the eigenvalues. Consider the spectral decomposition;

\begin{align*}
X’X &= P\Lambda P’\\
\alpha = P’\beta\\
X^* &= XP\\
C &= X’^*Y
\end{align*}

The mode $Y=X\beta + \varepsilon$ can be written as

$$Y = X^*\alpha + \varepsilon$$

The OLS estimator of $\alpha$ is

\begin{align*}
\hat{\alpha} &= (X’^*X*)^{-1}X’^* Y\\
&=(P’X’ XP)^{-1}C = \Lambda^{-1}C
\end{align*}

In scalar notation $$\hat{\alpha}_i=\frac{C_i}{\lambda_i},\quad i=1,2,\cdots,P_i\tag{(A)}$$

From $\hat{\beta}_R = (X’X+KI_p)^{-1}X’Y$, it follows that the principle of RR is to add a constant $K$ to the denominator of ($A$), to obtain:

$$\hat{\alpha}_i^R = \frac{C_i}{\lambda_i + K}$$

Grob criticized this approach, that all eigenvalues of $X’X$ are equal, while for the purpose of stabilization, it would be reasonable to add rather large values to small eigenvalues but small values to large eigenvalues. This is the general ridge (GR) estimator. it is

$$\hat{\alpha}_i^R = \frac{C_i}{\lambda_i+K_i}$$

Ridge Regression vs Lasso Regression

Both are regularized regression techniques, but:

FeatureL2L1
ShrinkageShrinks coefficients evenlyCan shrink coefficients to zero
Use CaseMulticollinearity, many predictorsFeature selection, sparse models

Ridge regression is a powerful biased regression method that improves prediction accuracy by adding L2 regularization. It’s especially useful when dealing with multicollinearity and high-dimensional data.

Learn R Programming Language

Basic Statistics MCQs Test 25

by

Test your knowledge of fundamental statistics concepts with this 20-question multiple-choice quiz! This Basic Statistics MCQs Test is perfect for students, statisticians, data analysts, and data scientists. This Basic Statistics MCQs Test Quiz covers key topics like:

Online Basic Statistics MCQs Test with Answers
  • Measures of central tendency (mean, median, mode)
  • Measures of dispersion (range, variance, standard deviation)
  • Frequency distributions (class width, relative & cumulative frequency)
  • Data summarization (five-number summary, quartiles)
  • Statistical inference (sample vs. population, descriptive vs. inferential stats)

Sharpen your skills for exams, job interviews, and competitive tests with these practical Basic Statistics MCQs Test. Whether you’re preparing for university tests, certifications, or data-related job roles, Basic Statistics MCQs Test Quiz helps reinforce core statistical concepts. Let us start with the Online Basic Statistics MCQs Test now.

Online Basic Statistics Quiz with Answers

1. The following data shows the number of hours worked by 200 statistics students:
frequency distribution mcqs


The cumulative relative frequency for the class of 10 — 19

 
 
 
 

2. The sum of the percentage frequencies for all classes will always equal ————?

 
 
 
 

3. The difference between the largest and the smallest data values is the

 
 
 
 

4. If the variance of a dataset is correctly computed with the formula using $n-1$ in the denominator, which of the following is true?

 
 
 
 

5. The following data shows the number of hours worked by 200 statistics students:
frequency distribution mcqs


The class width for this distribution is

 
 
 
 

6. The value that has half of the observations above it and half the observations below it is called the

 
 
 
 

7. A researcher has collected the following sample data

5  12  6  8  5  6  7  5  12  4

The mode is

 
 
 
 

8. A statistics professor asked students in a class their ages. On the basis of this information, the professor states that the average age of all the students in the university is 21 years. This is an example of

 
 
 
 

9. The following data shows the number of hours worked by 200 statistics students:
frequency distribution mcqs


The relative frequency of students working 9 hours or less

 
 
 
 

10. If a dataset has an even number of observations, the median

 
 
 
 

11. A numerical value used as a summary measure for a sample, such as sample mean, is known as a

 
 
 
 

12. The following data shows the number of hours worked by 200 statistics students:
frequency distribution mcqs


The number of students working 19 hours or less

 
 
 
 

13. The standard deviation of a sample of 100 observations is 64. The variance of the sample equals

 
 
 
 

14. In a five-number summary, which of the following is not used for data summarization?

 
 
 
 

15. A tabular summary of a set of data showing the fraction of the total number of items in several classes is a

 
 
 
 

16. A researcher has collected the following sample data

5  12  6  8  5  6  7  5  12  4

The mean is

 
 
 
 

17. A researcher has collected the following sample data

5  12  6  8  5  6  7  5  12  4

The median is

 
 
 
 

18. The sum of deviations of the individual data elements from their mean is

 
 
 
 

19. In a sample of 800 students in a university, 160 or 20% are Business majors. Based on this information, the school’s University reported that “20% of all the students at the university are Business majors”. This report is an example of

 
 
 
 

20. $\mu$ is an example of

 
 
 
 

Question 1 of 20

Online Basic Statistics MCQs Test with Answers

  • A numerical value used as a summary measure for a sample, such as sample mean, is known as a
  • $\mu$ is an example of
  • The sum of the percentage frequencies for all classes will always equal ————?
  • In a five-number summary, which of the following is not used for data summarization?
  • The following data shows the number of hours worked by 200 statistics students: The class width for this distribution is
  • The following data shows the number of hours worked by 200 statistics students: The number of students working 19 hours or less
  • The following data shows the number of hours worked by 200 statistics students: The relative frequency of students working 9 hours or less
  • The following data shows the number of hours worked by 200 statistics students: The cumulative relative frequency for the class of 10 — 19
  • The difference between the largest and the smallest data values is the
  • If a dataset has an even number of observations, the median
  • The sum of deviations of the individual data elements from their mean is
  • The value that has half of the observations above it and half the observations below it is called the
  • In a sample of 800 students in a university, 160 or 20% are Business majors. Based on this information, the school’s University reported that “20% of all the students at the university are Business majors”. This report is an example of
  • A statistics professor asked students in a class their ages. On the basis of this information, the professor states that the average age of all the students in the university is 21 years. This is an example of
  • A tabular summary of a set of data showing the fraction of the total number of items in several classes is a
  • The standard deviation of a sample of 100 observations is 64. The variance of the sample equals
  • A researcher has collected the following sample data 5  12  6  8  5  6  7  5  12  4. The median is
  • A researcher has collected the following sample data 5  12  6  8  5  6  7  5  12  4. The mode is
  • A researcher has collected the following sample data 5  12  6  8  5  6  7  5  12  4. The mean is
  • If the variance of a dataset is correctly computed with the formula using $n-1$ in the denominator, which of the following is true?

Learn R Programming Language