eternal validity refers to how well an experiment is done especially whether it avoids confounding more than one possible independent variable cause acting at the same time the confounding variable and the when the more internal validity we have we can say that there is a higher level of internal validity and so the less chance for a confound the confounding variable entering into our study the higher its internal validity is so therefore what can be an easy working definition for internal validity well internal validity refers to how well a piece of research allows you to choose among alternate explanations of something a research study with high internal validity lets you choose one explanation over another with a lot of confidence because it avoids many possible confounds so when we consider the first threats to internal validity we have to think of this in terms of a longitudinal study or a study where data is collected at multiple different time points from the same participant so attrition threat refers to when a participant or participants drops out of or leaves a study and this causes problems as it reduces available data in particular when data is different across groups so it can be thought of as quote the differential loss of individuals from treatment and/or comparison groups this is often a problem when research participants are volunteers volunteers may drop out of a study if they find it is consuming too much of their time or if they're not getting enough of a reward or any kind of compensation that might be involved others may drop out if they find the task to be too arduous it can also be thought of as referring to the drop out of subjects in an experiment if people drop out of an experiment they do so for a reason a problem with either a noxious or inadequate independent variable as in his 1988 article put it so how do we best fix attrition threat well the best way here as with many is to to account for this is to randomly assign participants into two groups one treatment group and one control group and one of the things to take home with this is that researchers can expect similar effects in treatment group versus control groups over time the next threat to internal validity is treatment diffusion this occurs when a comparison group learns about the program either directly or indirectly from program group participants in a school context children from different groups within the same school might share experiences during their lunch hour or comparison group students seeing what the program group is getting might set up their own experience to try to imitate that of the program group in either case if the diffusion of imitation effects the post-test performance of comparison group it can have and/or jeopardize your ability to assess whether your program is causing the outcome so here if we're in an in-school setting and we want to create a program for improving test scores right so there's a certain type of intervention taking place in the classroom and we want to study it we want to make sure that that program is in fact responsible for any increase of test scores that we might see right so notice that this threat to validity tends to equalize the outcomes between groups and minimize the chance of seeing a program effect if there even is one so how do we fix this well one of the ways to do this is by isolating one group of participants from the other this mitigates the likelihood that demand characteristics of the contrast between treatment conditions and control groups could interfere with the results and threaten the internal validity of the study in this way another threat is testing threat that changes in a test score due to taking it more than once through famila could affect the validity of the study so if you take the same test on two different occasions the fact that you're simply familiar with the study from previously familiar with the test I should say from previously taking that test could get in the way could interfere with results rather than the the result of maybe an intervention on how one is to perform in a classroom setting this is also referred to as pretest sensitization and this really refers to the effects of taking a test upon performance on a second testing merely having been exposed to the pretest may influence performance on a post-test and testing becomes a more viable threat to internal validity as the time between pretest and post-test is shortened because familiarity with that pretest will be heightened so this can be thought of the as the impact of quote taking a test and whether or not any special circumstances existed which may have influenced the scores on a subsequent test taking a test may cause anxiety stress or self-consciousness that the first time one takes it but such emotions may be absent at the post-test time causing an improvement that is not simply due to the treatment or intervention or independent variable as cross again put it in 1998 a 1988 article so how do we address this again random assignment becomes key what about history threat well history threat refers to events outside the experimental manipulation influencing the collective or individual participation so many change producing events may have occurred between the first observation and the second observation so history is more likely the longer the lapse in time between the pretest and the post-test takes place so things happen between a pretest and post-test especially when there may be six months to a year between the to these events or outside influences also known as confounding variables here they are again may cause changes we see between pretest time and post-test time and the the changes that we see may be due to these outside factors not the independent variable so how do we fix this well consistent methods and measurements and longitudinal studies from one time point to the next becomes key and also to the extent that you can shorten the duration between the first observation and later observations could also be helpful as well then we have instrumentation threat well this is when changes in the measurement or procedure over the course of a study such as with observations with coding systems over time drift systematically so changes in the way a test or other measuring instruments are calibrated that could account for the result of the research study could actually create a difference that we observe and not what we're trying to measure so if between the first observation and the second observation you say I don't think that some of the wording of these test questions was really that accurate so I'm going to change the wording and improve upon that test and then you have a second measurement and you see significant differences the difference is that you see may not be because of your intervention that you're trying to measure but it may be because you simply change the test so people are responding to a different understanding of what you're asking so how do we fix this again random assignments becomes key and also other than rather than using tests one could use direct observation have an observational study and have ratings based on these or ratings based on interviews so it becomes a quantitative coating of a more qualitative type assessment also just not changing the test from one time point to the next could be very helpful another threat is so action threat so this is where differences in groups that exist before treatment or intervention begins threaten the study this is especially problematic when participants selected for belonging to a specific group are implemented as a category in and of themselves differences may be due to this and these initial differences in groups and not the treatment or intervention so we can think of this as an issue of comparing apples and oranges so this can occur when intact groups are compared and the groups may have been different to begin with and if so for example if three different classrooms are each exposed to a different intervention the classroom performances may differ only because the groups were different to begin with and the end as cross put it the biases which result from comparing one group of complex and unique people to another group of complex and unique subjects comprises selection threat so how do we fix this again random assignment because these natural group differences will be mitigated by randomly selecting from each one of these groups what about maturation threat well this is where normal developmental changes and participants between pre and post-tests can be thought of as the as the changes we see based on our intervention this just isn't the case so gains and losses over time may be just due to normal maturation within individuals a popular example of this in psychology is personality disorder assessment so oftentimes individuals who have personality disorders present with very severe psychopathology well when measured over time especially in some of the most severe personality disorders or cluster beer access to very dramatic personality disorders over time as we assess individuals whether they have received therapy or not their symptoms severity goes down over time as their part as their personality solidifies doesn't mean they don't have the personality disorder it doesn't mean it isn't causing tremendous discord in their lives emotional disc or emotional dysphoria anxiety depression etc but it just means that it's less significant and impact on the whole over time and this can be thought of as a normal maturation of individuals over time as their personalities solidify so changes in physical intellectual or emotional characteristics that occur naturally over time can influence the results of a research study and that's really what this threat is referring to so as croft put it internal changes occurring between the pretest and posttest which can cause changes in the dependent variable can be thought of as maturation threat these are internal changes like becoming tired or heavier or weaker or less anxious or more inattentive with the passage of time these have nothing to do with the intervention that you're trying to study or measure so how do we fix this again assigning participants to two groups one treatment group one control group another threat is inequitable treatments so this is when participants in one group outperform or underperform relative to another group as a result of study expectation of a certain performance so I'll give you guys a commonly used example in statistics classes that for those of you who have taken statistics classes or who are in your statistics class your instructor might use a popular study was performed on children based on this particular threat this an equitable treatment threat and the demand characteristics of individuals in a study and two groups of children were randomly selected they had equivalent IQs they had equivalent intellectual functioning overall none had learning disabilities right they were all around average okay so in one group an instructor was told that they were getting a classroom full of very gifted students that they had all above average IQs they were extreme intelligent and they would likely perform well and so the class should be taught at a higher level of academic rigor in the other group the instructor was told that there were intellectually delayed students that they needed more hand-holding that they were performing worse in their previous classes and so these students that were divided into two groups went through the instructor presentations of the same material right so one instructor thinking that they could get more rigor and that the students were more intelligent than the other instructor believing that the that their students were to be lower performing so what do you guys think happened at the outcome of this study well upon final exams in the first group the gifted group the students performed astronomically well as though as though they were in fact gifted students and in the second group the students performed below average so this speaks to what was likely inequitable treatments taking place you had the same material that was being presented from one group to the next and that material was being presented likely in very different ways with very different demand characteristics from one classroom to the next the teachers who believed in their students had students who believed in themselves whereas the teachers who didn't believe as much in their students had students who didn't really believe in themselves or at least didn't perform at the top level and so what we can say to this is is that inequitable treatment causes a tremendous tremendous amount of negative influence on students performance and on study outcomes so how do we deal with this well first off by isolating the groups so one group doesn't know about the other group and also by blind studies so when we say blind studies we have two distinct into two types of blind studies and I would recommend writing this down because it's not on the slides but it might be on your test the first one is a single blind study in a single blind study the participant doesn't know whether they are assigned to a treatment group where they're actually receiving an intervention or a control group where they're receiving no intervention the group that is the comparison group if you will in a double-blind study neither the researchers who are administering the treatment and the researchers who are administering the control know whether their participants are in the control group or the treatment group and you can imagine how this might get very tricky but what this does is it eliminates the inequitable demand characteristics of researchers who might believe that their treatment and intervention will cause a dramatic change in the outcome of the study and vice versa and though in among researchers who believe that the control group will not change over time either and so by doing this there may be a top-level researcher who is aware of what group does what but the researchers who are actually implementing the study oftentimes undergraduate research assistants or graduate level research assistants they're unaware of what group they're administering the study to and so by doing this this helps to mitigate an equitable treatment threat what about special treatment threat well this is a very similar concept this is when groups are treated differently as a result of being in the study or one group is treated differently from another group because of their treatment status so this is essentially referring to the same thing again this how this can be mitigated through training of treatment administrators and personnel associated with a study or through double-blind studies where both the treatment administrator or the researcher and the and the participant don't know what group they're in we also have some threats that have to do more with the stew underlying statistics of quantitate and quoi and to a certain extent qualitative research as well we have statistical regression threat and for those who remember your stats classes or who are in them now you might be familiar with the concept of regression towards the mean so what does this mean well this is the tendency to drift towards the mean as one take small test multiple times this is especially problematic when choosing extreme scores which can quote-unquote regress towards the mean due to influences other than treatment so this has to do with a statistical with statistical concepts that are a bit beyond the scope of this course but essentially what this means is is that over time if an individual takes tests again and again and again the overall statistical likelihood is is that their score will move towards the average score so if they have a very high score the first time they take the test they're more likely to have a somewhat lower score the next time they take the test if the average score is lower than their high score similarly if they take a test one time say the SATs and they score in a lower percentile than they would like and the average score on the SATs is kind of is considered to be the midpoint they're more likely to score towards that midpoint as they take the test multiple times so how can we define this well this occurs when individuals are selected for an intervention or treatment on the basis of extreme scores on a pretest so if you're trying to select individuals who are in a treatment condition based on scoring very highly in something say you want to treat depressed patients right and so you choose the individuals who score the highest on a standardized measure of depression one popular one is the Hamilton Depression Inventory another one is the Beck Depression Inventory and you choose individuals who only score the highest right extreme scores are more likely to reflect larger whether it be positive or negative errors in measurement or chance actors write something else that's getting in the way of the study so such extreme measurement errors are not likely to occur in a second testing which means that when testing them again they're going to they're gonna score closer to the average score so as Groff put it this refers to the fact that a group at the extreme ends of a normal curve remember that bell curve earlier on in the semester that we presented has a greater probability of being closer to the mean of that curve the next time the group is measured so how do we deal with this well the problem here is with single assessments so two randomly assigned groups should equally regress towards the mean you get a pretty good idea of what's going on and so by randomly assigning participants to groups again with this random assignment one treatment group and one control group can help to mitigate these issues we also have interaction with selection with other factors and this is also known as additive and interactive effects of threats to internal validity and so when certain participants selection or experience drop out historical effects testing issues also affect the group so as one research scholar Van Beek states quote no threat is completely isolated often one threat can lead to other threats or enhance another threat often in a vicious cycle so what we're essentially saying here is that when one threat is already present in the study it makes it much more likely that another threat is going to occur hence the interaction between threats that's stated here so how we fix this well again we keep going back to this concept of random assignment if we randomly assign participants to two groups one treatment group in one control group will be less likely to have threats of any kind so random assignment really is key however in most in many cases especially in quasi experimental studies random assignment becomes very difficult because you're still trying to assess real-world conditions laboratory conditions again and our final threat for this lecture is ambiguous directionality and this is when the independent variable is not manipulated the direction of the influence is not always clear for example the impact of a therapist empathy on client outcome does the therapist get warmer because the client improves or vice versa so how do we deal with this threat well unless the temporal order is clear when one thing occurs first and then another thing occurs second the directionality is really difficult to determine so as researchers we have to be clear in statements of causality and describe when causality is unclear so we can help to try to mitigate this threat alright guys until next time