Understanding Type 1 and Type 2 Errors

Thank you everyone for joining me once again on Crunchy Econometrics. Today I'll be teaching on type 1 and type 2 errors. To give a clearer picture of type 1 and type 2 errors, I'm using this table which I adapted from Gujarati and Porter. Here is when the null hypothesis is true and here is when the null hypothesis is false. If you reject the results, when the null hypothesis is true, you have incurred a type 1 error. If you fail to reject the null hypothesis when it is false, you have incurred a type 2 error. So try to look at those tables and know the differences between type 1 error and a type 2 error. Now let's take an example of a type 1 error. I'll be borrowing a leaf from the crime literature. When the person who has committed a crime is given a not guilty verdict based on the facts presented. before the court of law. Remember, not guilty is not the same as saying you are free. The former statement is simply due to insufficient evidence to convict the accused. The judge in this case has committed a type 1 error by letting a criminal off the hook. Due to insufficient evidence, a person who committed a crime has been let off. So in this case, a type 1 error is committed. How about a type 2 error still borrowing a lead from the crime literature when an innocent man is convicted of a crime when he is in fact innocent the judge in this case has committed a type 2 error by sending an innocent man to jail also an example from medicine when you tell patients that drug b is no more harmful than drug a when it is actually harmful then such will have serious consequences and a type 2 error has been committed. So these are simple examples often used in textbooks of how you can distinguish between a type 1 error and a type 2 error. So now let's dig further on type 1 error. It is when the true null hypothesis is rejected when it is true. It is represented by alpha also called the size of the statistical test. It is the probability of rejecting the true null hypothesis. It is also when a false positive scenario has occurred. Classical statistics generally concentrate on the type 1 error. How about the type 2 error in this case? It is the failure to reject the null hypothesis when it is false. It is represented by 1-alpha called the power of the test. It is the probability of accepting the false Null hypothesis it is a false negative Scenario before you start testing your hypothesis These are the basic things that must be done even though most times we don't do it. I also fall Foul of this but it's always good for you to note it somewhere on a piece of paper The type of test do you rules guiding your test before you begin number one you have to state both the null and alternative hypothesis decide on the type of test is it a one-sided test is it going to be a two-sided test most of their corrective packages these days are programmed on the two-sided test set your sample size always make sure that at least you have 30 observations or more then decide on the level of alpha that is the statistical significance level Are you going to set alpha at 0.01 or 0.05 or 0.10? That is at 1%, 5% or 10%. After that, interpret your findings. So always endeavor to scribble these basic steps on a piece of paper before you begin your analysis. I'm going to show you ways by which you can state your hypothesis. For instance, I want to test the relationship between income level and consumer expenditure. Option 1, I could state my hypothesis in this form. You can see the null and the alternative. Option 2 is the most common way by which people state their hypothesis. Look at the differences between option 1 and option 2. Let's take a look at option 3. I can state it in a directional form that is income level has a negative impact against the alternative that income level has a positive impact. But like I said option two is the most common way which is a two-sided test by which most people state their hypothesis. Now let's look at the causes of type 1 and type 2 errors. Type 1 error occurs when the outcome of the analysis lead to the rejection of a true null hypothesis. If the p-value is lower than 0.05, if that is the level by which you set your alpha, then the true null hypothesis is rejected in favor of the alternative hypothesis. This happens mostly when results are statistically significant. Researchers always like to have their results to be statistically significant. We are happy when this happens. For type 2 error, it occurs when the outcome of the analysis leads to the failure to reject a false null hypothesis. If the p-value is higher than 0.05, then the false null hypothesis is accepted. This also happens mostly when your result turn out to be statistically not significant. We don't like this when it happens. I'm going to talk about choosing alpha the level of significance. Whether we reject or do not reject the null hypothesis depends critically on the level by which you have set your alpha which is the probability of you committing a type 1 error which is also the probability of rejecting the null hypothesis. when it is true but why is alpha commonly fixed at one five or at most ten percent level as a matter of fact there is nothing sacrosanct about these values any other values will do just as well depending on the nature of your research for most people in life sciences they often fix alpha at one percent because they are dealing with living things but People in social sciences that often use maybe some forms of secondary data can fix alpha at most at 10%. But most times the conventional level for those in social sciences is often 5%. Let's take a look at the T statistic. What is the T statistic? If the null hypothesis is rejected, it shows that the test result is statistically significant. On the other hand, If the null hypothesis is not rejected, then it shows that those coefficients are not significant. That implies that a large T statistic will be sufficient evidence against the null, all things being equal. In the language of significant test, a statistic is said to be statistically significant if the value of the test statistic lies within the critical region. I'm going to show you... a picture or a graph to explain exactly what I just said. In this case, the null hypothesis is rejected. By the same token, a test is said to be statistically insignificant if the value of the test statistic lies in the acceptance region. Let's take a look at the p-value. The p-value is also known as the observed or exact level of significance or the exact probability of committing a type 1 error more technically you can say that the p value is defined as the lowest significance level at which you can reject a null hypothesis it is also the p value that gives statistical relevance to the t statistic if you take a look at this table you can see the point at which the p value will give strong or weak evidence against the null hypothesis if the p value is higher than 0.10 it means evidence against the null is weak or does not exist if the p value lies between five and ten percent it shows that the evidence against the null hypothesis is moderate if the p value lies within one and five percent then there is strong evidence against the null and if the p-value is lower than one percent or equal to one percent then there is a very strong evidence to reject the null hypothesis so the p-value gives you the smallest evidence by which the null hypothesis can be rejected now let's take a look at those plots to your left is a one tail test using alpha at five percent and to your right is a two tail test Fixing alpha at 0.05. Now here is a rejection region for the one-tail test if after performing your analysis and the test statistics falls in this region then you are going to reject the null hypothesis for the one-tail test if it's for a two-tail test which is going to be the outcome from all these econometric packages and your result happens to fall in the rejection regions then you are rejecting the null hypothesis. Let me emphasize again for this plot alpha is at 0.05 you can see it up here. I was able to pull up these plots from Woodridge textbook. So these plots are based on a 5% significance level. Now how can type 1 and type 2 error occur using these plots? If alpha is increased to 10% the rejection region will expand that means the true beta will now fall into the rejection region and that increases the likelihood of committing a type 1 error remember this plot is based on 0.05 if alpha is increased to 0.10 the rejection region will expand perhaps maybe to something of this points and when that happens the test statistic will now fall into the rejection region and your likelihood of committing a type 1 error increases how about a type 2 error if alpha is reduced to one percent the rejection area contracts the true beta now falls into the acceptance region thereby increasing the likelihood of committing a type 2 error so the more you reduce alpha the likelihood of committing type 2 error increases. So that tells you that alpha plays a role in whether you commit a type 1 error or a type 2 error. So let's take a clear example from my e-views output. Any output from any of these statistical packages or economic packages will do just fine. Whether it's data or R or microfeet, it doesn't matter. It's the same explanation. If my alpha is fixed at 5%, so that means any statistic or probability value higher than 0.05, I won't be able to reject that null hypothesis. So that means for C2, which shows that the p-value is higher than 0.05, I cannot reject the null hypothesis. If I take a look at C9, where the p-value is also 0.0979, higher than 0.05, I cannot reject the null hypothesis. Now, what if another researcher decides to carry out this same test and chose alpha at 10%? So that means C2 for that researcher is significant, thereby the null hypothesis is rejected. Same thing C9 for that researcher is statistically significant, and at that point, the null hypothesis is rejected. So from here you can see the role alpha plays in swinging you from committing either a type 1 error or a type 2 error. Remember that if alpha increases, the likelihood of committing a type 1 error increases. And if alpha reduces, the likelihood of committing a type 2 error increases. Now let's talk about the T-statistics and the P-value. If the data do not support the null hypothesis, the t-statistic obtained under that null hypothesis will be large and therefore the p-value of obtaining such a t-statistic will be small. In other words, for any given sample size, as your t-statistic increases, you are going to have small p-value which now increases the confidence of rejecting the null hypothesis. Let's take a look at the p-value and relationship with alpha. the level of significance. If we make the habit of fixing alpha to be equal to the p-value of any given test statistic, then there is no conflict between the two values. To put it differently, it is better to give up fixing alpha arbitrarily at some level and simply choose the p-value for the test statistic. It is also preferable to leave it to the reader or to your audience to decide whether to reject the null hypothesis at that given p value just the way i showed you from the eviews apple earlier on. If in an application the p-value of a test statistic happens to be say for example 0.125 which is 12.5 percent and if your reader decides to reject the null hypothesis at that level of significance so be it. There is nothing wrong with taking a chance of being wrong 12.5 percent of the time if you reject the true null hypothesis. So what is the trade-off? in committing a type 1 and a type 2 error. To minimize both errors, a trade-off is required. But for any given sample size, it is not possible for you to minimize both errors at the same time. So keep the probability of committing a type 1 error at a fairly low level, say at 1% or 5%. Anything higher than that will lead you to commit a type 1 error. Likewise, Minimize the probability of committing a type 2 error as much as you can. What will be the decision consequences? This is actually important for research involving medicine and life sciences. For type 1 error, when the true null hypothesis is rejected, it will lead to grave consequences. For type 2 error, failure to reject the null hypothesis when it is indeed false will have dire consequences in my opinion committing a type 2 error is worse but i leave you to my audience to take their own positions thank you for staying with me i've come to the end of the topic on type 1 and 2 errors most of my tutorials and on lecture notes i got it from gujarati and woodridge i will recommend you to read further or pick up any basic econometric test book to support the understanding you got from my video Please like I always say video tutorials cannot be replaced with reading. Read up to deepen your understanding. Thank you for watching. Thank you for supporting my channel. Thank you for making me to hit the 1000 subscribers. That was good and now I need it to make me hit the 4000 hours. I appreciate those who have shared my videos. Please don't go away. Crunchy Econometrics is dedicated to beginners and those who are willing to deepen the understanding of the application of econometric tools. I'll be right back with more interesting videos.

If you fail to reject the null hypothesis when it is false, you have incurred a type 2 error. So try to look at those tables and know the differences between type 1 error and a type 2 error. Now let's take an example of a type 1 error.

I'll be borrowing a leaf from the crime literature. When the person who has committed a crime is given a not guilty verdict based on the facts presented. before the court of law. Remember, not guilty is not the same as saying you are free.

The former statement is simply due to insufficient evidence to convict the accused. The judge in this case has committed a type 1 error by letting a criminal off the hook. Due to insufficient evidence, a person who committed a crime has been let off. So in this case, a type 1 error is committed.

How about a type 2 error still borrowing a lead from the crime literature when an innocent man is convicted of a crime when he is in fact innocent the judge in this case has committed a type 2 error by sending an innocent man to jail also an example from medicine when you tell patients that drug b is no more harmful than drug a when it is actually harmful then such will have serious consequences and a type 2 error has been committed. So these are simple examples often used in textbooks of how you can distinguish between a type 1 error and a type 2 error. So now let's dig further on type 1 error. It is when the true null hypothesis is rejected when it is true. It is represented by alpha also called the size of the statistical test.

It is the probability of rejecting the true null hypothesis. It is also when a false positive scenario has occurred. Classical statistics generally concentrate on the type 1 error.

How about the type 2 error in this case? It is the failure to reject the null hypothesis when it is false. It is represented by 1-alpha called the power of the test. It is the probability of accepting the false Null hypothesis it is a false negative Scenario before you start testing your hypothesis These are the basic things that must be done even though most times we don't do it. I also fall Foul of this but it's always good for you to note it somewhere on a piece of paper The type of test do you rules guiding your test before you begin number one you have to state both the null and alternative hypothesis decide on the type of test is it a one-sided test is it going to be a two-sided test most of their corrective packages these days are programmed on the two-sided test set your sample size always make sure that at least you have 30 observations or more then decide on the level of alpha that is the statistical significance level Are you going to set alpha at 0.01 or 0.05 or 0.10?

That is at 1%, 5% or 10%. After that, interpret your findings. So always endeavor to scribble these basic steps on a piece of paper before you begin your analysis.

I'm going to show you ways by which you can state your hypothesis. For instance, I want to test the relationship between income level and consumer expenditure. Option 1, I could state my hypothesis in this form. You can see the null and the alternative. Option 2 is the most common way by which people state their hypothesis.

Look at the differences between option 1 and option 2. Let's take a look at option 3. I can state it in a directional form that is income level has a negative impact against the alternative that income level has a positive impact. But like I said option two is the most common way which is a two-sided test by which most people state their hypothesis. Now let's look at the causes of type 1 and type 2 errors. Type 1 error occurs when the outcome of the analysis lead to the rejection of a true null hypothesis. If the p-value is lower than 0.05, if that is the level by which you set your alpha, then the true null hypothesis is rejected in favor of the alternative hypothesis.

This happens mostly when results are statistically significant. Researchers always like to have their results to be statistically significant. We are happy when this happens.

For type 2 error, it occurs when the outcome of the analysis leads to the failure to reject a false null hypothesis. If the p-value is higher than 0.05, then the false null hypothesis is accepted. This also happens mostly when your result turn out to be statistically not significant.

We don't like this when it happens. I'm going to talk about choosing alpha the level of significance. Whether we reject or do not reject the null hypothesis depends critically on the level by which you have set your alpha which is the probability of you committing a type 1 error which is also the probability of rejecting the null hypothesis.

when it is true but why is alpha commonly fixed at one five or at most ten percent level as a matter of fact there is nothing sacrosanct about these values any other values will do just as well depending on the nature of your research for most people in life sciences they often fix alpha at one percent because they are dealing with living things but People in social sciences that often use maybe some forms of secondary data can fix alpha at most at 10%. But most times the conventional level for those in social sciences is often 5%. Let's take a look at the T statistic. What is the T statistic? If the null hypothesis is rejected, it shows that the test result is statistically significant.

On the other hand, If the null hypothesis is not rejected, then it shows that those coefficients are not significant. That implies that a large T statistic will be sufficient evidence against the null, all things being equal. In the language of significant test, a statistic is said to be statistically significant if the value of the test statistic lies within the critical region.

I'm going to show you... a picture or a graph to explain exactly what I just said. In this case, the null hypothesis is rejected. By the same token, a test is said to be statistically insignificant if the value of the test statistic lies in the acceptance region.

Let's take a look at the p-value. The p-value is also known as the observed or exact level of significance or the exact probability of committing a type 1 error more technically you can say that the p value is defined as the lowest significance level at which you can reject a null hypothesis it is also the p value that gives statistical relevance to the t statistic if you take a look at this table you can see the point at which the p value will give strong or weak evidence against the null hypothesis if the p value is higher than 0.10 it means evidence against the null is weak or does not exist if the p value lies between five and ten percent it shows that the evidence against the null hypothesis is moderate if the p value lies within one and five percent then there is strong evidence against the null and if the p-value is lower than one percent or equal to one percent then there is a very strong evidence to reject the null hypothesis so the p-value gives you the smallest evidence by which the null hypothesis can be rejected now let's take a look at those plots to your left is a one tail test using alpha at five percent and to your right is a two tail test Fixing alpha at 0.05. Now here is a rejection region for the one-tail test if after performing your analysis and the test statistics falls in this region then you are going to reject the null hypothesis for the one-tail test if it's for a two-tail test which is going to be the outcome from all these econometric packages and your result happens to fall in the rejection regions then you are rejecting the null hypothesis. Let me emphasize again for this plot alpha is at 0.05 you can see it up here. I was able to pull up these plots from Woodridge textbook.

So these plots are based on a 5% significance level. Now how can type 1 and type 2 error occur using these plots? If alpha is increased to 10% the rejection region will expand that means the true beta will now fall into the rejection region and that increases the likelihood of committing a type 1 error remember this plot is based on 0.05 if alpha is increased to 0.10 the rejection region will expand perhaps maybe to something of this points and when that happens the test statistic will now fall into the rejection region and your likelihood of committing a type 1 error increases how about a type 2 error if alpha is reduced to one percent the rejection area contracts the true beta now falls into the acceptance region thereby increasing the likelihood of committing a type 2 error so the more you reduce alpha the likelihood of committing type 2 error increases.

So that tells you that alpha plays a role in whether you commit a type 1 error or a type 2 error. So let's take a clear example from my e-views output. Any output from any of these statistical packages or economic packages will do just fine. Whether it's data or R or microfeet, it doesn't matter. It's the same explanation.

If my alpha is fixed at 5%, so that means any statistic or probability value higher than 0.05, I won't be able to reject that null hypothesis. So that means for C2, which shows that the p-value is higher than 0.05, I cannot reject the null hypothesis. If I take a look at C9, where the p-value is also 0.0979, higher than 0.05, I cannot reject the null hypothesis.

Now, what if another researcher decides to carry out this same test and chose alpha at 10%? So that means C2 for that researcher is significant, thereby the null hypothesis is rejected. Same thing C9 for that researcher is statistically significant, and at that point, the null hypothesis is rejected. So from here you can see the role alpha plays in swinging you from committing either a type 1 error or a type 2 error.

Remember that if alpha increases, the likelihood of committing a type 1 error increases. And if alpha reduces, the likelihood of committing a type 2 error increases. Now let's talk about the T-statistics and the P-value.

If the data do not support the null hypothesis, the t-statistic obtained under that null hypothesis will be large and therefore the p-value of obtaining such a t-statistic will be small. In other words, for any given sample size, as your t-statistic increases, you are going to have small p-value which now increases the confidence of rejecting the null hypothesis. Let's take a look at the p-value and relationship with alpha. the level of significance. If we make the habit of fixing alpha to be equal to the p-value of any given test statistic, then there is no conflict between the two values.

To put it differently, it is better to give up fixing alpha arbitrarily at some level and simply choose the p-value for the test statistic. It is also preferable to leave it to the reader or to your audience to decide whether to reject the null hypothesis at that given p value just the way i showed you from the eviews apple earlier on. If in an application the p-value of a test statistic happens to be say for example 0.125 which is 12.5 percent and if your reader decides to reject the null hypothesis at that level of significance so be it. There is nothing wrong with taking a chance of being wrong 12.5 percent of the time if you reject the true null hypothesis. So what is the trade-off?

in committing a type 1 and a type 2 error. To minimize both errors, a trade-off is required. But for any given sample size, it is not possible for you to minimize both errors at the same time. So keep the probability of committing a type 1 error at a fairly low level, say at 1% or 5%.

Anything higher than that will lead you to commit a type 1 error. Likewise, Minimize the probability of committing a type 2 error as much as you can. What will be the decision consequences? This is actually important for research involving medicine and life sciences. For type 1 error, when the true null hypothesis is rejected, it will lead to grave consequences.

For type 2 error, failure to reject the null hypothesis when it is indeed false will have dire consequences in my opinion committing a type 2 error is worse but i leave you to my audience to take their own positions thank you for staying with me i've come to the end of the topic on type 1 and 2 errors most of my tutorials and on lecture notes i got it from gujarati and woodridge i will recommend you to read further or pick up any basic econometric test book to support the understanding you got from my video Please like I always say video tutorials cannot be replaced with reading. Read up to deepen your understanding. Thank you for watching. Thank you for supporting my channel.

Thank you for making me to hit the 1000 subscribers. That was good and now I need it to make me hit the 4000 hours. I appreciate those who have shared my videos. Please don't go away. Crunchy Econometrics is dedicated to beginners and those who are willing to deepen the understanding of the application of econometric tools.

I'll be right back with more interesting videos.

Transcript for:Understanding Type 1 and Type 2 Errors

Transcript for:
Understanding Type 1 and Type 2 Errors