okay so this is the chapter 7 audio experiments what causes what so in the chapter we're gonna talk about the logic logic of experimentation variations in experimental design variations in experimental context the process of planning and conducting an experiment and the strengths and weaknesses of experiments so first there's an example that they give in the book of research to kind of explain how a lot of these things work so it focuses on research that theorized that prosecute prosecutorial misconduct might be more likely in serious cases because quote prosecutors succumb to increase pressure to convict so they argue that if a prosecutor thinks the defendant is guilty they might engage in misconduct thinking that the ends justify the means so their hypothesis was the greater the severity of a crime the more likely that misconduct will occur from a prosecutor so to test their hypothesis the researchers carried out a laboratory experiment at the University which called for experiments who would produce our experiments called for participants who would play the role of a prosecutor so the participants were asked to read a police report and prepare their own case these were undergrads who were paid 10 bucks for participating and they were randomly assigned either a murder or an assault case which will explain what that means later so they were told that they would be randomly picked to either be a defense attorney a prosecutor or a judge and so they would grab a slip from a hat but in reality all the slips said prosecutor on them right so they manipulated the independent variable this is kind of like precursor to what you need to do to make an experiment happen so you know there was a victim of a crime in one of these two scenarios either they were pronounced dead at the scene the murder condition obviously or they had fully recovered from their injuries which they were the assault case so participants were asked to read a set of interviews obtained by police and compile questions that they would turn over to the defense because of course part of a prosecute prosecutors job is to get the information and give information to the defense that could be used to defend their client right so misconduct happens when they don't turn over that information that might indicate someone else is at fault where someone else is guilty that's where the misconduct is so anyway they're basically they are supposed to read this police report bla bla interviews from the police officers and compile a list of questions to give to the defense so some questions though during these interviews pointed to the defendants guilt but other questions identified the victim's wife as a potential suspect so the way they measured this dependent variable was they would look at the number of questions that implicated another suspect that were not turned over to the defense so it's important to know that their hypothesis is looking at the relationship between crime severity and prosecute prosecutorial misconduct so between two variables in the experiment the independent variable is manipulated in this case the severity of crime which is followed by a measurable dependent a dependent variable right so in their example this is prosecuting prosecutorial misconduct which they operationalized in which questions you end up turning over to the defense so participants are randomly assigned to the groups of either murder or assault treating the two groups as the same so we'll get into how those are the fundamental features of experimental research designs so the logic of experimentation so how do experiments meet the requirements of causal inference so remember the whole point is to show that there's a relationship between the two variables but that there's a causal relationship meaning you're manipulated independent variable is changing resort is causing the results you see in the dependent variable so Association is just determined by whether an experimental condition it differs significantly on the dependent variable so this would be in the example the group that got the murder case versus the group that got the assault case to see if they differ significantly with the dependent variable meaning the misconduct which they measured as what questions you turn right so they also look at direction of influence this establishes the time order of experimental events so the dependent variable is measured after there's manipulation in the independent variable so it wouldn't make sense to measure misconduct before you given them the cases the murder or the assault cases and seeing what questions they turn over and then of course what's also important in experiments is non spurious Ness so experimental conditions differ only in terms of the manipulation because either you know participants are randomly assigned to the experimental conditions and participants are treated the same except for the manipulation meaning either way they're still reading the police reports right whether it's murder or assault the only thing that's different is the severity of the case so non spurious Ness just meaning there isn't some other intervening variable or some other thing that's causing the change you see in the dependent variable so um it's important to kind of explain at this point the difference between random sampling and random assignment both use the term random and this can be confusing for students but there's box 7.1 in the chapter that attempts to explain you know or at least make this clear so random sampling we talked about in the last chapter right it's part of the design before you sample you assign people randomly to receive study information from a small sample that represents the overall population that the researcher is trying to study right so they're trying to see you know if their sampling method is random then they can apply their findings to a larger population that's kind of the whole idea to extrapolate it so random assignment is a little bit different random assignment is when the study has already started so you've already sampled now you have more than one independent variable you're testing like the prosecute prosecutorial man that's that's a hard word for me today prosecutorial misconduct example right they decided that people would be assigned murder and other people would be assigned assault so they pick randomly who's going to be in each group so that the groups are roughly equivalent right so you wouldn't want to have a sample and decide all the whim are gonna be in one group all the men will be in the other group right that wouldn't make the groups similar so you don't want the groups to be significantly different because then that could affect the results right so the groups are balanced on characteristics and what makes it random is the fact that the participants could be picked to be an either group from the odd outset so it's important to kind of explain the difference because I know a lot of people get confused about that all right when it comes to variations in experimental design there are some basic requirements right when it comes to those experimental research designs so I talked about in the last slide right Association direction of influence non spurious Ness but there are differences when it comes to the design so how many experimental designs differ well they differ in terms of when the dependent variable is measured so there's some different designs that have pre-tests so obviously that means it happens before the experiment there are designs that are post tests there's other designs that you know where basically the independent variable is measured after the manipulation of the or the dependent variables measured after the manipulation of the independent variable so when the dependent variable is measured matters right if it's a pretest post-test also it can differ by you know maybe measure both both before and after so some experiments decide to you know they contain multiple post tests at various times right because how you decide these things again it's really difficult to say there's one way to do this they're very specific to the type of study and really the course of your specific research question is going to you know lead your design so you know at the point of your research method obviously this is gonna Wed to your question to try and answer your question so some studies need more than one test to express either change over time or maybe maybe just to give more validity to their results so you know they could be measured one or more times after and it really just depends on the kind of research you're doing so as a result the testing or measuring may be measured one or more times after the manipulation of the independent variable and of course it can differ also in terms of number of you know independent variables that are actually being manipulated so in the most basic experimental research design you're looking at a factor X effects factor Y or like if you're studying effects of you know let's say retention of course information right then studying would be the independent variable to see if that's going to affect whether you retain things right and the dependent variable would be you know whether you retain stuff and you could operationalize that as how well you do on a midterm hint in this class you will have a midterm so you would measure who studied against who those people who didn't and you would compare their scores on the exam and that would be a relationship between two factors right whether you studied and your retention of course information but we know there's a lot of other relevant factors to OFF often have an impact in any research design so in our example right now this kind of retention of class material maybe some students are better at studying than others right some have better work ethics some people are just bad test takers etc etc right so sometimes researchers will manipulate the independent variable to multiple options or even increase the number of independent variables being measured against the change in the dependent variable so when two or more independent variables are studied in one experiment they're referred to as factorial designs right and these are designs where two or more variables have been manipulated so interestingly factorial designs provide evidence of the impact of each factor as well as the joint effect of the factor so you can see how much studying affects retention of course information and how much is your testing ability or you know any of these other factors that would interplay with that as well all right when it comes to variations in experimental context how can these experimental context vary so it just depends on where you're doing the research right so the majority of these experimental research designs are being done in the laboratory setting which is obviously a controlled environment and the purpose of the control environment is that you're just measuring the variables that you think you're measuring right contrary to this are these kind of field and survey versions so the field is conducted in a natural setting so this has the features of a true experiment meaning you still have manipulation of the independent variable a random assignment all that fun stuff but you would be doing it in a real-world setting and we'll get to in a minute how that actually can help with your validity because experimental research designs tend to be low on external validity because if you're doing all of your research in a laboratory setting that doesn't necessarily mean it's going to translate into the real world right so doing research in the field can actually kind of help deal with those problems and then of course um you can also do you can have this experiment embedded in a survey so the book gives an example of how wording is tested through surveys right in an experimental context so they use the GSS or general social survey as an example so you know typically they would ask the same question with two similar questions that are worded differently and you can kind of experiment in other ways in the survey as well right not just on wording or how people interpret wording but you know you could look at public opinion or you know understandings of social policies or things like that where you can still use an actual experimental method within the context of a survey itself all right so this is the kind of steps that you would have to go through for process and planning of conducting an experiment so of course you design the experiment you kind of figure out what it is you're measuring what the relationships are you develop your hypothesis right and then typically people are going to pretest now the pretest gives you an opportunity to see if these things are working out in the kind of relational ways you think they are right so you can kind of get through some of the hiccups or some of the problems through the pretest and then you can kind of reevaluate it before you actually you know pick your group and go with that so the pretest is usually a much smaller sample group just gives you an idea right so then you're gonna recruit your research participants using your sampling design we talked about in Chapter six and then you're going to introduce the experiment right you're going to explain to them what it is about you have to obviously acquire informed consent with any of this even if you're doing something that is a little bit deceptive you're still gonna have to get informed consent even if you're leaving out the true purpose of the research that way you know basically that the person is informed about what the research entails any potential risk or harm that might come to them as a result of doing the research right and really through informed consent you tell them that at any point if they want to leave the research they are welcome to do so which is a big part of you know if we're using people as data we have to respect them right we have to make sure we don't harm them or anything of that nature and that they're warned about what their participation can do so that's what the informed consent is all about right so then at this point in between you know actually manipulating the independent variable and you know explaining the experiment to your participants you're gonna randomly assign participants to different experimental conditions so this could be putting them in a control group versus an experimental group like let's say you know they do this a lot in medical science where they will like let's say they're testing a new pill right and the effectiveness of a new pill they'll have a control group that just gets a sugar pill right so they think they're taking a pill and they measure that against people that are actually taking the real medication and that's kind of where we get that term the placebo effect right the idea that sometimes people just from taking a pill all the time even though it's just a sugar pill think that that is affecting them and that thought process that like psychosomatic thought process actually changes your body's functions so they want to measure the difference between the experimental in the control group because the control group is basically like well it should be at a baseline right but if you see measurable differences there you can kind of weigh that against the changes you see with the actual independent variable in the experimental group and see what's actually happening there right so basically you're gonna randomly assign people to be in one of those either experimental group control group or if you have different variations of your you know you're going to manipulate your independent variable then you know you're in the example in the book the group that gets put into the murder group or the group that gets put into the assault cases right so then you actually manipulate the independent variable right you set up those different criteria for your independent variable and then of course you want to measure the dependent variable at that point so now that you've changed the independent you want to see does that make any difference to your dependent variable right is it causing the kind of difference that you expect it to is there a certain relationship between it like we'll talk about later how these things can be measured the types of relationships so let's say if you have a correlation there's different directional correlations right meaning like let's say um what we know that there's a correlation between level of education and income right and this is a positive correlation meaning the more the higher your level of education the higher your pea tends to be right salary wise but there are inverse ones as well right that when you increase one factor something else actually goes down that would be a negative so we'll talk about those directional things later so anywho you measure your dependent variable you're gonna perform that manipulation check just to make sure that you're actually seeing what you think you're you're seeing right and then you're gonna debrief and especially in a research design in experimental research design where you're manipulating people or they think something's happening that it really isn't like for example the example in your book the prosecutorial misconduct one they when they debrief they had to explain to people some of the little things that they changed like for example people thought they were gonna be randomly assigned to either a prosecutor a judge or a defense attorney but in reality it was to measure whether or not they would do that right whether it was really just prosecutor was the only option so they wanted to see whether or not people would you know do misconduct so it wouldn't make sense to have them in rolls right they were told that so there is some level of manipulation sometimes that takes place in these things and it's essential for ethically obviously and for your research participants that you debrief right and the reason is because people might be kind of pissed about being lied to or being manipulated but often times if you tell someone hey I'm going to study this and then do it they might actually change their behaviors to fit what you're looking for and that's not gonna give you good information right so sometimes you have to manipulate just so that you can actually measure what you want to measure without that kind of unintended influence of the participant trying to make the researcher happy by acting differently than it normally would right so we'll kind of get into those examples in a minute okay so the example they give or the parallel or metaphor whatever you want to call it that they give in the book is that experiments creating an experiment and you know enacting an experiment is kind of like a play right so they say experiments are like producing a play so when you produce a play you have to make a script right you have to pick a cast of characters that will act out this script right you have to have a dress rehearsal and then you have your actual performance right so in this scenario you know the design is like the script of a play right so in the design you know or in the you know design phase you're going to have specific hypotheses that you derive from the theories that are related to your topic or to the relationship between the variables you're looking at right so just like a script tells the actor what specific words they do how they act where they stand all those kind of cues you know in the same way in an experiment your operationalizing your variables in the hypothesis right you're going from taking a character from the kind of play point of view and developing specific words and specific phrases and actions that they will take place right that's kind of the same operationalizing that goes on in research and then they talk about how pre-testing is like a dress rehearsal so the pretest really just applies the procedure of the experiment but only to a few participants so this sees how well the script is working right in this kind of script metaphor so the research in the book really they figured out that participants were not motivated to win their fake court cases when they did their pretest you know because basically if you're going to commit misconduct it must be because you want to win your case right so if people weren't motivated to win that's a big flaw in their design of their research so they decided the way to make people want to win was to incentivize winning so they incentivize winning by saying you would get 10 bucks for participating in the study but if you won the case you'd get 15 bucks right this kind of thing of like you can incentivize about offering more money so a pretest became this important step in figuring out all those potential issues or hiccups that can be avoided in the actual study because if they had gone through with it and done the study and no one was motivated to win you wouldn't even see any prosecutorial misconduct right so that would kind of diminish the whole point of doing the research so the pretest is important so just think about like if you ever had to be in a play or a school thing or something like that you have to rehearse right you have to actually try ahead of time to just make sure that it all fits together right in the same way when you do research you want to have this kind of time to try first and then have the time to reflect and figure out what went wrong so you can adapt before the actual research takes place okay so going through the kind of steps in the introductory phases you're going to recruit your participants right so almost always this is a non-random sample because you want it to be something that can then be reflected back to the overall population right be extrapolated and if it's not a random sample you're not gonna be able to do that right and usually um when it comes to certain kinds of experimental research sometimes an incentive is offered for participation so like the one I talked about in the chapter they gave people 10 bucks for participating which can kind of help you get recruits when it's difficult to do of course you got to go through that absorbed consent like I said before you know you have the people you need to make them aware of any potential harm that can come from the research and explain to them the scope of the research right make sure they know that their participation is voluntary and can be withdrawn at any point then you introduce the experiment to the participants right so participants are given an explanation of the study and typically there's a cover story given that conceals the deception so in a lot of these cases what is being measured is not what the participants initially think is being measured in order to not influence the outcome so in the prosecutorial misconduct example they're told there's a chance of being those three different things right like I spoke about before that you pull one slip out of a hat but in reality all the slips that prosecutor and pulling them out of a hat was really just a ruse that was part of this cover story right and then you're going to randomly assign participants to different experimental conditions and like I said before in a balanced way to make sure that you're not kind of putting people in different groups that could affect the outcome significantly so then you actually measure the variables right so you want to have this manipulation of your independent variable that occurs first and typically you're gonna do some you know validity checking with a manipulation check so this is to see if the manipulation and the independent variable was experienced or interpreted by the participants in the intended way you wanted so they do this by just asking them right you can do this by like a written piece of paper or actually like interviewing them and just really to help explain what they felt or thought during the you know during the research or immediately following the research because that's going to help you understand if their responses are a reflection of what you're actually measuring or maybe they interpreted it differently maybe their actions were a reflection of something else and that can also give you some interesting information so then you measure your independent variable right you see what changes happen and there's different kinds of measure so the chapter goes into this but I'll just briefly touch on in so verbal measures are pretty easy to make right and put the problem with this is is that it only measures what people think and say when you look at the verbal measures versus behavioral measures are much less obtrusive because you're just kind of like watching how people behave and them tend to be more precise because people sometimes will say things and that's not actually how they act right so there are strengths and weaknesses to kind of both approaches of that and then you get to the debriefing right and like I said before the debriefing is it ethically it's completely necessary okay if you use deception you have to tell people what you were actually studying and that's the only way that it that it's acceptable to use deception right and so um this also though gives an opportunity for researchers to get participants reactions and interpretations so you can ask the person what they thought and felt you know about like what certain things happen in the research and their interpretations are gonna tell you if what you're seeing is accurate about their perceptions and understandings right we have a really lucky thing and studying people as you can literally just be like hey is that what you thought was happening what did you think was happening right to get a better sense of that instead of just saying you know oh I put these variables together I operationalize it this way so therefore it is definitely measuring this but what if that's not what people interpret right we do have the ability to interpret these things differently so it's important to do this and of course it must be carried out with care like they talked about in the chapter because people could be bad right if you manipulate someone or trick them or they feel like they've you know been tricked they can be kind of pissed so you want to make sure that you really make people feel comfortable explain how participants were deceived and why it was necessary encourage them to ask questions about it or express any sort of negative feelings that they feel because you just don't want someone to leave the research feeling angrier or more upset as a result of being a participant right alright so then when it comes to the strengths of participants uh or principle of experiments what is the principle strength of experiments well first they're going to provide sound evidence of causal relationships this is an important goal right so the goal is to prove more than the concepts have a relationship or a correlation right but that one in that it's a causal relationship that one is actually causing change in the other another strength though in this kind of research is that it's high in internal validity so um you know this is really just evidence that only the manipulated variables account for the measurable outcome in the dependent variable right so this really means that there isn't some extraneous variables that's really a play that's warping or really causing the difference you're seeing right and why why that is why it's so high in internal validity is because you're literally controlling for other factors especially in a lab setting you're literally controlling for these other factors so you know those other factors are not affecting it right but there are still threats to internal validity that happen um you know if you have types of extraneous variables that might account for some other experiments outcome but really they talk about these three issues of selection so this just means the differences among the participants if they're not randomly assigned so again if you just kind of like put all the left-handed people in one group or if you pick some characteristic to divide people by that could be a problem because you might not have balanced groupings right when we go back to that example I was talking about before about random assignment you want to make sure that the groups are balanced so that you would find the same information in either group if they're not then that could be a threat to internal validity maturation so the fact that sometimes there's personal changes that happen in participants right so if you're if you're doing a study over a long period of time then like let's say you're studying you know a longitudinal study then you might experience some differences that have just come from the aging or the life course where our opinions or understandings change as we age and grow right so there might be some sort of psychological or even physiological change that takes place within the part over time and that could affect this and also a history right so just meaning that events other than the experiment may influence the results of the study so you know this could be you know if other events that are taking place or I don't know like let's say you're on the way to this thing and you get in a car accident or something something that affects your emotional state so that you acting differently than you would otherwise so the whole idea is that you want to make sure you're measuring how someone would actually act if not being observed right to get the kind of best information possible and there are some weaknesses of experiments right if there's strengths and weaknesses of whatever research design you choose but there are ways to counteract weaknesses as well so the principal weaknesses and of experiments the first is that it tends to be low in external validity or what we call generalizability so this is very important because if you find something in your research like a relationship between two or more variables you want to make sure that it's something that's not specific to the lab right it would also happen in a similar way out in the real world where there's like a lot of other factors at play so often to counter play this researchers will do field research right to see what differences there are in the field than in the lab or sometimes researchers will increase generalizability through replication which just means repeating the study with different respondents and the logic is that if you keep getting the same findings more and more this is going to increase that external validity right and another way to deal with external validity is to change up your sample or maybe even just increase your sample size right we're also researchers can look at the same issues but with the different research design like let's say you do an experimental design but then you look at it through a survey or you then interview people or just something else that kind of measures it from a different angle so you make sure that you're what you're seeing is what would actually be found right by other researchers and this is subject to reactive measurement effect so this means that people in experiments they know they're being measured right so people tend to act differently if they know they're being watched and this can affect experimental research studies because if people want to be helpful right participants often want to be helpful and project a favorable image so you know like kids some people want to please the researcher so they might act differently or going back to the favorable image they might change their behavior from how they would act outside of the lab setting you know to look better to a researcher and this can again affect the external validity right like the kind of Hawthorne effect that idea that and you know this guy basically went into a factory to do a study on worker productivity and everything he did cause the productivity to go up he turned up the lights the productivity went up he turned down the lights of productivity went up literally everything he did he gave them breaks he took away their breaks no matter what the productivity went up and that's when they realized oh there's an effect of being studied so if basically if your boss calls someone in and says hey this guy's gonna study how productive you are you're probably gonna be more productive right so yeah there's there is that reactivity problem but this can be controlled in a limited extent so they talk about how when you're doing the debriefing session you can kind of see what participants perceptions are and also this can be controlled by keeping and experiments or blind to the participants experimental condition so that they're not able to like unintentionally bias or affect the participant so like in medicine they use this thing that's called a double line research study for this purpose so this means that the researcher knows what's happening but the person actually administering the experiment doesn't know what's being studied and so the neither the research are neither the experimenter or the participants of the experiment know what's being done in that same way or what's being tested so that they can restrict all the kind of potential for bias in that way right and the real last weakness of experiments that tends to be the most constricting is that they're restricted in what can be studied and so not every type of topic is gonna work well in an experimental research design so some find it limiting right as a research approach so that's one of those ones that's very difficult to counteract versus some of the other ones that can really be minimized as far as you know external validity or reactive measurements but there are certain things that just don't lend themselves to being studied in a lab setting or in a controlled setting in these kind of experimental research methods ways so that can be kind of another limitation or you know weakness of the experimental research design [Music]