Type I and II Errors

In this post, we will discuss Type I Type II error examples from real-life situations. Whenever sample data is used to estimate a population parameter, there is always a probability of error due to drawing an unusual sample. Two main types of error occur in hypothesis tests, namely type I and type II Errors.

Type I Error (False Positive)

It is rejecting the null hypothesis ($H_0$) when it is actually true. The probability of Type I Error is denoted by $\alpha$ (alpha). The most common values for type I error are: 0.10, 0.05, and 0.01, etc. The example of Type I Error: A medical test indicates a person has a disease when they actually do not.

Type II Error (False Negative)

Type II Error is failing to reject the null hypothesis ($H_0$) when it is actually false. The probability of Type II Error is denoted by $\beta$ (beta). The power of the test is denoted by $1-\beta$, which is the probability of correctly rejecting a false null hypothesis. The example of a Type II error is: A medical test fails to detect a disease when the person actually has it.

Comparison Table

Error Type	What Happens	Reality	Risk Symbol
Type I	Reject H₀ when it is true	$H_0$ is true	$\alpha$
Type II	Fail to reject $H_0$ when it is false	$H_1$ (alternative) is true	$\beta$

	$H_0$ True	$H_0$ False
$H_0$ Rejected	Type I Error	Correct Decision
$H_0$ Not Rejected	Correct Decision	Type II Error

Type I Type II Error Example (Real-Life Examples)

1. Medical Testing
   • Type I Error (False Positive): A healthy person is diagnosed with a disease. It may lead to unnecessary stress, further tests, or even treatment.
   • Type II Error (False Negative): A person with a serious disease is told they are healthy. It may delay treatment and worsen health outcomes.
     In this case, the more severe error is a Type II error, because missing a true disease can be life-threatening.
2. Court Trial (Justice System)
   • Type I Error: An innocent person is found guilty. It leads to punishing someone who did nothing wrong.
   • Type II Error: A guilty person is found not guilty. It led to the criminal going free.
     In this example, the more severe is often Type I, because the justice system typically aims to avoid punishing innocent people.
3. Fire Alarm System
   • Type I Error: The alarm goes off, but there’s no fire. Therefore, a false alarm causes panic and interruption.
   • Type II Error: There is a fire, but the alarm does not go off. It can cause loss of life or property.
     The more severe error is Type II error, due to the potential deadly consequences.
4. Spam Email Filter
   • Type I Error: A legitimate email is marked as spam. It means one will miss important messages.
   • Type II Error: A spam email is not caught and lands in your inbox. The spam email may be a minor annoyance or a potential phishing risk.
     The more severe error in this case is usually Type I, especially if it causes loss of critical communication (like job offers, invoices, etc.).
5. Quality Control in Manufacturing
   • A factory tests whether its products meet safety standards. The null hypothesis ($H_0$) states that the product meets requirements, while the alternative ($H_1$) claims it is defective.
   • Type I Error (False Rejection): If a good product is mistakenly labeled defective, the company rejects a true null hypothesis ($H_0$), leading to unnecessary waste and financial loss.
   • Type II Error (False Acceptance): If a defective product passes inspection, the company fails to reject a false null hypothesis ($H_0$). This could result in unsafe products reaching customers, damaging the brand’s reputation.

Which Error is More Severe?

• It depends on the context.
• In healthcare or safety, Type II errors are often more dangerous.
• In justice or decision-making, Type I errors can be more ethically concerning.

Designing a good hypothesis test involves balancing both types of errors based on what’s at stake.

Learn about Generic Functions in R

The post covers the Type I and Type II Errors examples.

Hypothesis testing helps us to determine whether the results are statistically significant or occurred by chance. Hypothesis testing is based on probability, therefore, there is always a chance of making the wrong decision about the null hypothesis (a hypothesis about population). It means that there are two types of errors (Type I and Type II errors) that can be made when drawing a conclusion or decision.

Errors in Statistical Decision-Making

To understand the errors in statistical decision-making, we first need to see the step-by-step process of hypothesis testing:

1. State the null hypothesis and the alternative hypothesis.
2. Choose a level of significance (also called type-I error).
3. Compute the required test statistics
4. Find the critical value or p-value
5. Reject or fail to reject the null hypothesis.

When you decide to reject or fail to reject the null hypothesis, there are four possible outcomes–two represent correct choices, and two represent errors. You can:
• Reject the null hypothesis when it is actually true (Type-I error)
• Reject the null hypothesis when it is actually false (Correct)
• Fail to reject the null hypothesis when it is actually true (Correct)
• Fail to reject the null hypothesis when it is actually false (Type-II error)

These four possibilities can be presented in the truth table.

Type I and Type II Errors Examples: Clinical Trial

To understand Type I and Type II errors, consider the example from clinical trials. In clinical trials, Hypothesis tests are often used to determine whether a new medicine leads to better outcomes in patients. Imagine you are a data professional and working in a pharmaceutical company. The company invents a new medicine to treat the common cold. The company tests a random sample of 200 people with cold symptoms. Without medicine, the typical person experiences cold symptoms for 7.5 days. The average recovery time for people who take the medicine is 6.2 days.

You conduct a hypothesis test to determine if the effect of the medicine on recovery time is statistically significant, or due to chance.

In this case:

• Your null hypothesis ($H_0$) is that the medicine has no effect.
• Your alternative hypothesis ($H_a$) is that the medicine is effective.

Type I Error

A Type-I error (also known as a false positive) occurs when a true null hypothesis is rejected. In other words, one can conclude that the result is statistically significant when in fact the results occurred by chance. To understand this, let in your clinical trial, the results indicate that the null hypothesis is true, which means that the medicine has no effect. In case, you make a Type-I error and reject the null hypothesis, it means that you incorrectly conclude that the medicine relieves cold symptoms while the medicine was (actually) ineffective.

The probability of making a Type I error is represented by $\alpha$ (the level of significance. Typically, a 0.05 (or 5%) significance level is used. A significance level of 5% means you are willing to accept a 5% chance you are wrong when you reject the null hypothesis.

Reduce the risk of Type I error

To reduce your chances of making Type I errors, it is advised to choose a lower significance level. For example, one can choose the significance level of 1% instead of the standard 5%. It will reduce the chances of making a Type I error from 5% to 1%.

Type II Error

A Type II error occurs when we fail to reject a null hypothesis when it is false. In other words, one may conclude that the result occurred by chance, however, in fact, it didn’t. For example, in a clinical study, if the null hypothesis is false, it means that the medicine is effective. In case you make a Type II Error and fail to reject the null hypothesis, it means that you incorrectly conclude that the medicine is ineffective while in reality, the medicine relieves cold symptoms.

The probability of making a Type II error is represented by $\beta$ and it is related to the power of a hypothesis test (power = $1- \beta$). Power refers to the likelihood that a test can correctly detect a real effect when there is one.

Note that reducing the risk of making a Type I error means that it is more likely to make a Type II error or false negative.

Reduce your risk of making Type II Error

One can reduce the risk of making a Type II error by ensuring that the test has enough power. In data work, power is usually set at 0.80 or 80%. The higher the statistical power, the lower the probability of making a Type II error. To increase power, you can increase your sample size or your significance level.

Potential Risks of Type I and Type II Errors

As a data professional, it is important to be aware of the potential risks involved in making the two types of errors.

• A Type I error means rejecting a true null hypothesis. In general, making a Type I error often leads to implementing changes that are unnecessary and ineffective, and which waste valuable time and resources.
  For example, if you make a Type I error in your clinical trial, the new medicine will be considered effective even though it is ineffective. Based on this incorrect conclusion, ineffective medication may be prescribed to a large number of people. While other treatment options may be rejected in favor of the new medicine.

• A Type II error means failing to reject a false null hypothesis. In general, making a Type II error may result in missed opportunities for positive change and innovation. A lack of innovation can be costly for people and organizations.
  For example, if you make a Type II error in your clinical trial, the new medicine will be considered ineffective even though it’s effective. This means that a useful medication may not reach a large number of people who could benefit from it.

In summary, as a data professional, it helps to be aware of the potential errors built into hypothesis testing and how they can affect the final decisions. Depending on the certain situation, one may choose to minimize the risk of either a Type I or Type II error. Ultimately, it is the responsibility of a data professional to determine which type of error is riskier based on the goals of your analysis.

R Language Quick Reference