Type I And Type II Errors In Hypothesis Testing: Examples

Introduction:

In statistics, a Type I error is also known as a false positive conclusion (example: "an innocent person is convicted"), while a Type II error is known as false negative conclusion (example: "a guilty person is not convicted").

Much of statistical theory revolves around the minimization of one or both of these errors, though the complete elimination of either is a statistical impossibility if the outcome is not determined by a known, observable causal process.

Making a statistical finding includes uncertainties, so we have to face  these errors in hypothesis testing.

The probability of making a Type I error is the significance level, or alpha (α)

 probability of making a Type II error is beta (β). Which depends on sample size and variance.

Type I error

A Type I error means rejecting the null hypothesis while it’s actually true. It means concluding that results are statistically significant when, in reality, they came due to purely by chance or because of unrelated factors. Type I errors can be thought of as errors of commission.

The first kind of error is the mistaken rejection of a null hypothesis as the result of a test procedure. This kind of error is called a type I error (false positive) and is sometimes called an error of the first kind.

In terms of the courtroom example, a type I error corresponds to convicting an innocent defendant.

Generally, a higher Type I error triggers eyebrows because this indicates that there is evidence against the default state of being. This essentially means that unexpected outcomes or alternate hypotheses can be true. Thus, it is recommended that one should aim to keep Type I errors as small as possible.

The probability of making this error is the significance level (alpha or α) . Significance level is chosen by researchers. The significance level is generally set at 0.05 or 5%. This means that your conclusions  only have a 5% chance of occurring, or less, if the null hypothesis is actually true.

If the p value of your test is lower than the significance level, it shows your results are statistically significant and  accept the alternative hypothesis. If your p value is higher than the significance level, then you can concludes it is non-significant.

Type II error

A Type II error means fail to rejecting the null hypothesis while it’s really false.

Instead, a Type II error means failing to conclude there was an any effect when there actually was. Actually your study may not have had enough statistical power to detect an effect of a certain sample.

When doing a hypothesis testing, one fails to reject the null hypothesis when he should actually have rejected it, this error or mistake is termed as Type II error.

The second kind of error is the mistaken acceptance of the null hypothesis as the result of a test procedure. This sort of error is called a type II error (false negative) and is also referred to as an error of the second kind.

In terms of the courtroom example, a type II error corresponds to acquitting a criminal.

Power is the amount to which a test can correctly detect a real effect when there is one. A power level of 80% or higher is usually considered acceptable.

The probability of a Type II error is inversely related to the statistical power of a study.

 

Is a Type I or Type II error worse?

For statisticians, a Type I error is usually worse. In practical terms, however, either type of error could be worse depending on your research context.

A Type I error means mistakenly going against the main statistical assumption of a null hypothesis. This may lead to new policies, practices or treatments that are inadequate or a waste of resources.

Example 1: You decide to get tested for COVID-19 based on mild symptoms. There are two errors that could frequently occur:

· Type I error (false positive): the test conclusion tells you have tested covid positive, but you actually don’t have corona viruses.

· Type II error (false negative): the test conclusion tells you don’t have corona virus, but you actually have corona virus.

Example 2: Drug intervention

· Type I error: A Type I error happens when you get false positive results: you understood that the drug Consumption enhance sign and symptoms when it actually didn’t. These developments could have arisen from different random factors or measurement errors.

· Type II error: A Type II error happens when you get false negative results: you conclude that the drug intervention didn’t enhance sign and  symptoms when it actually did. Your study may have missed key indicators of improvements or attributed any improvements to different factors instead.

Example 3: House On Fire

Let’s take the example of smoke coming out of a home. Let’s state the null hypothesis, H0, that the house is not on the fire and the smoke is due to some food getting cooked. alternate hypothesis is that the house is on fire.

· Type I error: A person passing by the house thought that the house is actually burning with fire and thus called the firefighters. However, firefighters after arriving at the spot found that the smoke was actually due to the food being cooked.  The person passing by rejected the null hypothesis that the smoke is due to the food getting cooked and called the firefighters.

· Type II error:  let’s say the passer-by ignored the smoke from the house thinking that the smoke is coming out due to food being cooked. After some time, it was found that house actually burnt. In statistical sense, the passer-by failed to reject the null hypothesis that the house is not on fire and the smoke is coming due to food being cooked. Actually, the alternative hypothesis that the house is on fire was true.

Example 4-

Biometric matching, which for fingerprint recognition, facial recognition or iris recognition. Alternative hypothesis is The input does not identify someone in the searched list of people.  Null hypothesis is The input does identify someone in the searched list of people

· Type I error : The true fact is that the person is someone in the searched list but the system show that the person is not according to the data.

· Type II error : The true fact is that the person is not someone in the searched list but the system shows that the person is someone whom we are looking for according to the data.

 

Frequently asked questions (FAQs)

 

1.  What are type 1 and type 2 errors?

ANS: In statistics, a Type I error means rejecting the null hypothesis when it’s actually true, while a Type II error means failing to reject the null hypothesis when it’s actually false.

2. How do you reduce the risk of making a type 1 error?

ANS: The risk of making a Type I error is the significance level (Or alpha) that you choose. That’s a value that you set at the beginning of your study to assess the statistical probability of obtaining your results (p value).

The significance level is usually set at 0.05 or 5%. This means that your results only have a 5% chance of occurring, or less, if the null hypothesis is actually true.

To reduce the Type I error probability, you can set a lower significance level.

3. How do you reduce the risk of making a type 1 error?

ANS: The risk of making a Type II error is inversely related to the statistical power of a test. Power is the extent to which a test can correctly detect a real effect when there is one.

To (indirectly) reduce the risk of a Type II error, you can increase the sample size or the significance level to increase statistical power.

4. What is statistical power?

ANS: In statistics, power refers to the likelihood of a hypothesis test detecting a true effect if there is one. A statistically powerful test is more likely to reject a false negative (a Type II error).

If you don’t ensure enough power in your study, you may not be able to detect a statistically significant result even when it has practical significance. Your study might not have the ability to answer your research question.

Blog by:

1. Mohinee Jadhav

2. Ishita Kadam

3. Neha Kadam

4. Varsha Kadam