Basic Statistics and Data Analysis

Statistics Lecture Notes, Online MCQs

Category: Type I error

Level of Significance / Testing of Hypothesis / Type I error

 by 

Difference between a Probability value and the Significance level

Difference between a probability value and the significance level?

Basically in hypothesis testing the goal is to see if the probability value is less than or equal to the significance level (i.e., is p ≤ alpha). It is also called the size of the test or size of the critical region. It is generally specified before any samples are drawn so that the results obtained will not influence our choice.

  • The probability value (also called the p-value) is the probability of the observed result found in your research study of occurring (or an even more extreme result occurring), under the assumption that the null hypothesis is true (i.e., if the null were true).
  • In hypothesis testing, the researcher assumes that the null hypothesis is true and then sees how often the observed finding would occur if this assumption were true (i.e., the researcher determines the p-value).
  • The significance level (also called the alpha level) is the cutoff value the researcher selects and then uses to decide when to reject the null hypothesis.
  • Most researchers select the significance or alpha level of .05 to use in their research; hence, they reject the null hypothesis when the p-value is less than or equal to .05.
  • The key idea of hypothesis testing it that you reject the null hypothesis when the p-value is less than or equal to the significance level of.05.

Level of Significance / Type I error / Type II Error

 by 

Type I error, Type II error and minimizing the risk of both of these types of errors

Type I and Type II Errors

In hypothesis testing there are two possible errors we can make: Type I and Type II errors.

  • A Type I error occurs when your reject a true null hypothesis (remember that when the null hypothesis is true you hope to retain it).
    α=P(type I error)=P(Rejecting the null hypothesis when it is true)
    Type I error is more serious than type II error and therefore more important to avoid that a type II error.
  • A Type II error occurs when you fail to reject a false null hypothesis (remember that when the null hypothesis is false you hope to reject it).
    β=P(type II error) = P(accepting null hypothesis when alternative hypothesis is true)
  • The best way to allow yourself to set a low alpha level (i.e., to have a small chance of making a Type I error) and to have a good chance of rejecting the null when it is false (i.e., to have a small chance of making a Type II error) is to increase the sample size.
  • The key in hypothesis testing is to use a large sample in your research study rather than a small sample!

If you do reject your null hypothesis, then it is also essential that you determine whether the size of the relationship is practically significant.
The hypothesis test procedure is therefore adjusted so that there is a guaranteed “low” probability of rejecting the null hypothesis wrongly; this probability is never zero.

Estimate and Estimation / Short Questions / Testing of Hypothesis / Type I error

 by 

Why do educational researchers usually use .05 as their significance level?

Type I Error

It has become part of the statistical hypothesis testing culture.

  • It is a longstanding convention.
  • It reflects a concern over making type I errors (i.e., wanting to avoid the situation where you reject the null when it is true, that is, wanting to avoid “false positive” errors).
  • If you set the significance level at .05, then you will only reject a true null hypothesis 5% or the time (i.e., you will only make a type I error 5% of the time) in the long run.