so in this module we're going to introduce confidence intervals so performing statistical tests and reporting p-values is important and useful but it's not enough to come to a conclusion about our analysis we usually want to know an estimate of the population parameter based upon the sample that we've collected from that population and we choose a single number or a statistic as our best guess of the unknown quantity and this is usually called a point estimate so for example the sample statistic for our point estimate for the population mean is our sample average so x x bar is a point estimate of mu because there's variability in how we sample from the population there's variability randomness and uncertainty in our estimate of the population parameter we could simply report this by giving a point estimate of the standard deviation as well as for example the sample average even better we usually report a confidence interval usually abbreviated as ci which gives a range of plausible values of the population parameter so let's look at an example what's the mean pulse rate of students taking a midterm for pm 510 so we designed an experiment to look at this let's say we have 50 students in the class and we have two tas who will each measure the pulse rate of 10 students taking the midterm in the class after one hour and each tas selects 10 students at random so let's say the first ta grabs their 10 students and these are the pulse rates that they measured in beats per minute and if we take the sample means we take the mean of these 10 observations we get an average pulse rate of 75.2 beats per minute and then the second ta does the same so samples 10 students and takes the average and gets 76.1 beats per minute now the estimates of 75.2 and 76.1 are not incompatible but they're they're definitely different right so 75.2 definitely does not equal 76.1 but they're in the same neighborhood and the difference between those two point estimates those two sample means is partly because the two tas sampled different students out of the 50 students that were available so a confidence interval is a range of plausible values for the value of a population parameter and the majority of confidence intervals have the following form you take the point estimate and either add or subtract some constant times the standard error of that point estimate so at the top of the slide here we're just recapitulating that equation again and here the standard error typically depends on the sample size n the variability of the raw data in the sample so the standard deviation and for the sample mean x bar the standard error is usually the standard deviation divided by the square root of n now the constant is a theoretical value that depends on the probability distribution of the point estimate and the chosen confidence level alpha so remember with every statistical test we've chosen an alpha level to compare our statistic to to decide whether we have statistical significance or not and that's going to have an impact here on our confidence interval so the general confidence interval for the population mean takes two different forms if the population variance or the population standard deviation are known then the formula is our point estimate of the mean or our sample mean plus or minus a z value that's based on 1 minus alpha over 2 times the standard error of the mean as calculated using the known standard deviation if the standard deviation or the variance of the population is unknown then the confidence interval is computed as the same point estimate right our sample mean plus or minus now instead of a z a t value that's going to be based on n minus one our total sample size minus one and one minus alpha over two and then multiplying by the sample standard error so the sample standard deviation divided by the square root of n so again for known standard deviation or known variance in the population we're going to use a z value and for unknown variance or standard deviation we're going to use a t value and what these two equations describe is with 100 times one minus alpha percent confidence where the true population mean mu lies based upon our data so under the standard normal curve which is the notation we use is n for normal and the two values in the parentheses basically are the mean and the variance so we're standard normal distribution right so zero mean and a variance or standard deviation of one the quantity z of 1 minus alpha over 2 is called a critical value for the normal distribution and remember under these population standard deviation known situation we're going to be multiplying our standard error by z and the critical value represents the point for which there is an area 1 minus alpha divided by 2 to the left and an area of alpha divided by 2 to the right of this particular z value so graphically speaking basically here the little blue part is where our z value is hypothetically so in this case our z value is 1.282 and that's where the blue part isn't denoting and so 1 minus alpha divided by 2 is this area under the curve and if alpha divided by 2 is going to be this area under the curve so critical values for confidence intervals for a mean again based on a known population variance or standard deviation can be computed let's just go back here again like using the bognar website or other software you can compute the areas so an eighty percent confidence level ninety percent confidence overall so on and so forth with different alpha levels alpha over two and then the actual z value that's associated with each of those and so knowing these z values for these different combinations of interval and alpha you can compute the area under the curve so let's look at an example let's say we want to compute a 95 confidence interval for a mean again under the known population variance or standard deviation situation and so let's say our question is the iq for a sample of la residents we collect seven individuals we compute the sample mean so there is the average iq is 99.6 and we know that the population standard deviation is 15. so the 95 confidence interval for the mean is 99.6 plus or minus 1.95996 times our standard population standard deviation divided by the square root of n which gives us 99.6 plus or minus 11.11193 now this 1.95996 let's go back to this table here basically comes from the fact that we want to do a 95 confidence interval which means our alpha level is 0.05 and so the corresponding z value is 1.95996 so when we do that we get 88.4807 110.7119 right so that's subtracting 11.11193 from 99.6 and adding that value to 99.6 so the interpretation of so this is our confidence interval and the interpretation is that we are 95 confident the true population mean iq of los angeles residents is between 88.5 and 110.7 with our estimate being 99.6 so again the sample mean our z value our critical value which is one minus alpha over two the known population standard deviation and the number of patients or our sample size now confidence interval values when the population standard deviation is unknown utilizes the t distribution instead of the z distribution so instead of z of 1 minus alpha over 2 we're going to use a t of n minus 1 and 1 minus alpha over 2 for the critical value and this t value is a critical value for the t distribution that breaks the distribution into areas that are again 1 minus alpha over 2 and alpha over 2. remember the shape of the t distribution changes based on the degrees of freedom which is why we have this n minus 1 parameter as part of our t value therefore for any given critical value the area is going to change based on the degrees of freedom so what's the mean pulse rate of students taking a midterm for pm 510 and let's say again we design the same experiment we have 50 students in the class and each ta selects 10 students at random and measures the pulses so here what i've done i've taken our data and ran it through spss using one of the descriptive statistics options and you can see here here are our two data for our two tas n of 10 for each and here we're getting the minimum and the maximum so the first ta the smallest pulse rate was 71 and the largest was 80 and for the second ta the smallest was 7d with the largest being 81. so here are the means for those two samples the standard error for both the standard deviation and the variance so this is all sample statistics based on that sample of 10 students so here are the general parameters we're working with so our sample size is 10 so our degrees of freedom is n minus 1 which is 9. so we want to look up a critical value for t that's got 9 degrees of freedom and 0.975 so here we're assuming that alpha equals 0.05 so we're doing again a 95 percent confidence interval so if we go back to our table or we go to like the bognar website and we plug in our 0.975 for right so here's 0.975 for 9 degrees of freedom and we want to look at the left side of the distribution we get our z value of 2.26216 so now we can compute for each of the samples the sample collected by ta1 and the sample collected by ta2 we can compute the 95 percent confidence intervals so let's just walk through carefully the one for ta1 and you can do ta2 on your own so again our equation our sample mean our point estimate plus or minus our t value that's based on our degrees of freedom n minus 1 and 1 minus alpha over 2 times the standard error so the standard deviation of the sample divided by the square root of n so our point estimate was 75.2 plus or minus 2.26216 which comes from our distribution over here our t distribution and then again 2.7 was our standard deviation and dividing by the square root of the sample size 10. so that gives us 75.2 plus or minus 1.931466 so our confidence interval is 73.26853 for the lower bound 77.13147 for the upper bound and we can just round those two single-digit single decimal points so 73.3 uh to 77.1 so what we're saying is again is that we're 95 confident that the true mean pulse rate for pm 510 students taking a midterm is between 73.3 and 77.1 beats per minute with our estimate being 75 75.2 beats per minute so again i'm going to let you walk through for ta2 that same calculation but that same calculation basically says the 95 confidence interval goes from 73.7 to 78.5 now notice that these two confidence intervals overlap so if i graph this out basically and let me just erase all my little chicken scratch to make this a little clearer so if you look at what we what the graph looks like basically the point in the middle is our point estimate so this is the average okay so actually let's do this that's x bar and then the two dots on the other side are the lower and upper bound for the 95 percent confidence interval so you can see clearly here that the two point estimates are very close to each other right so that distance is pretty small but the 95 confidence intervals overlap and basically what that tells you then is that the two-point estimates are statistically not different from each other because again the 95 remember the 95 confidence interval says we're 95 percent sure that the true value the true sample the population mean excuse me falls between these two values for the first sample and falls between these two values for the second sample and because these overlap it basically means that we cannot differentiate between these two samples in terms of the point estimate for the mean now there's a fundamental connection between confidence intervals and hypothesis tests a two-sided hypothesis test at a significance level of alpha will be statistically significant if and only if the two-sided 100 times one minus alpha confidence interval does not contain the hypothesized value of that parameter so a similar statement holds for the one-sided tests and confidence intervals in this class we're only going to use two-sided confidence intervals but you can do a one-sided confidence interval if you wanted to so let's go back and look at our iq data from los angeles and let's say we want to compare it to the national average iq so the national average is 100 and let's say we know that the population standard deviation is 15. and again our los angeles sample we grab seven individuals we get a sample mean of 99.6 and this population standard deviation of 15. so the 95 confidence interval so again this is a population standard deviation known situation so we want to use the sample mean and the z value so this is our z value times the standard error and we get our confidence interval here 88.5 to 110.7 and because the 95 confidence interval includes 100 which was the estimate for the national average the mean of the data is not statistically significantly different from 100 at the 0.05 level using a one sample z test as we've concluded above so this is one way to use confidence intervals to do statistical hypothesis testing basically what we just did was we compared our sample estimate of 99.6 to the population as a parameter of 100 to see if this 99.6 was statistically significantly different from the 100 but because 100 falls within our confidence bound we can cut say at the 0.05 level because we did a 95 percent confidence interval we can say that 99.6 is not statistically significantly different from 100 now suppose you calculate a 90 confidence interval for a population mean and it comes out to 3.56 and 5.29 so what does this mean so a common thing that people write is that there's a 90 percent chance that the population mean is between these two values this is absolutely wrong to say so there's a big difference between saying there's a 90 chance that the population mean is between those two values versus saying that we're 90 percent confident that the true population mean falls between those two values never ever say that there's a 90 percent chance or a 95 chance or whatever for a confidence level it's not telling you about chance and we can perform a small simulation to illustrate what a ninety percent confidence interval really means so using spss we can generate random data from a normal distribution that has let's the following characteristics a mean of 0 and a standard deviation of 9.6 and that's just some arbitr arbitrary values i've chosen and what we do is we're going to generate 56 data sets each of sample size 23 that we're so basically we're going to take samples of 23 from this normal distribution and we're going to do that 56 times and for each sample of 23 that we draw we're going to calculate the sample mean and the sample standard deviation and compute the 90 confidence interval for each result so we're going to end up with 56 sample means with 56 90 confidence intervals so 96 percent confidence interval is based on randomly generated data from that distribution and here i've plotted them all out so you can see here at the bottom this is our mean of zero and then the the confidence interval and basically the i just plotted them across two graphs if i because if i put 56 lines in a single graph it would be very hard to see so you notice that most of the confidence intervals overlap with zero except in the four cases that are shown in red so basically 52 out of our 56 simulated data sets had confidence intervals that contained zero so that's 92.9 percent now if the statement that there was you know let's go back to our statement so if the statement that that there's a 90 chance that the population mean falls between two values if that were true then we should have seen 90 percent of these confidence intervals should have contained zero and instead here we have basically 93 and you can repeat this simulation over and over again and you'll get slightly different numbers here but you won't see an estimate or the estimate of 90 is going to be extremely rare so why is that we're essentially testing the null hypothesis that mu equals zero the population mean equals zero versus the alternative hypothesis that the population mean does not equal zero mu does not equal zero so each individual confidence interval either contains zero or it doesn't there's no probability involved it's not saying that the interval ninety percent of the time the interval is going to include zero it either contains zero or it doesn't contain zero so what percentage of our confidence intervals contain zero if that ninety percent probability statement were correct we should have seen about ninety percent of our confidence involves containing zero and that wasn't the case so if the null hypothesis is true and you repeat the experiment many times then approximately ninety percent of these confidence intervals should contain the actual parameter of interest in this case the population parameter so that's why we say we're 95 we're 90 percent confident the true population parameter lies within the confidence interval okay so i hope that makes it clear the difference between saying there's a 90 chance that the true value lies within the confidence interval as opposed to saying that we're 90 confident that the true value lies within those two uh intervals