hi class so today we're gonna pick up where we left off previously we had been discussing experimental research and now we're going to be moving on to non experimental research design and this is really the seat of what we see in most social science research is that it follows non experimental research design as we talk about this today we're gonna see why that is now as you can probably tell from the term non experimental research is defined by exclusion right so it's non experimental it's some it this requires us to define an experiment before we go on so the distinction between experimental research and non experimental research is based on the degree of control that the researcher has over the subjects and the conditions of the research now the key words here are manipulation and assignment versus observation an experiment is a kind of investigation in which some variable is manipulated so the researcher has enough control over the situation to decide which participants receive which conditions at which times so suppose you were interested in the difference in learning between students who only studied a book and students who read the book and also attend lectures a researcher who's able to perform an experiment would set up the conditions manipulate a variable and assign subjects to the assigned subjects to them as we're going to see this assignment of subjects to conditions makes it possible to determine with more certainty whether the differences between the conditions actually caused any difference in behavior now if the researcher cannot assign subjects to groups but must only observe how students into already-existing classes at a college learn according to the same two methods we do not have a true experimental study in this case we would find that many things could have caused any differences in learning between the classes for example the students may have signed up for one class or the other based on their preference for an instruction the instructor the teaching method or even just the time of day that the class is offered any of these factors could cause differences and learn a second characteristic of non-experimental research is that the data collection procedure often must forfeit some degree of control in return for obtaining the data for example we might decide to study public records that may be almost but not exactly in the form we desire or we might have to keep a questionnaire short to help gain the cooperation of students or subjects so non experimental research is often called correlational research because it seeks causes of behavior by looking for correlations among variables now the term is somewhat misleading because all research is correlational to the extent that it seeks functional relationships between variables but calculating correlations among variables does not make the research correlational in the strictest sense we often compute correlations among variables in the in the truest of experiments what makes research correlational in the common usage is the ability to manipulate some variable independently in correlational research relationships are studying among variables none of which may be the actual cause of the other so you know this is where we get to the correlation is not causation reminder and one can only speculate the causes of the relationships we demonstrate causation when we can decide which variable caused the other and this is best done in an experiment in a true experiment non experimental research is often a first step in starting to answer theoretical questions by empirical methods experimental research frequently is done as a follow-up to previous non experimental observations for example experimental research on physiological and behavioral factors in alcoholism has found has followed from the non experimental observation that alcoholism tends to run in families now it's convenient to distinguish several varieties of non experimental research the first may be called observational research that is research in which the researcher simply observes the ongoing behavior examples are field observation of Ducks from from a blind or television monitoring people in a store the second category is archival research and in this method existing records are examined to test hypotheses about the causes of behavior for example a researcher might study crime statistics in different countries to see if there's a relation between capital punishment and the murder rate in the third category which isn't on this slide but we're going to talk about it in a little bit yeah we call the case study this category is different from others in that the researcher investigates a particular situation that has come-to to their attention and the situation may be a practical problem that must be solved as soon as possible or it may be an event a person or animal that intrigues a researcher an investigator might study the victims of a natural disaster to determine its effects on their psychological health or the impact of severe social isolation on an individual's ability to learn language as was demonstrated in Jones 1995 article where they look to just that case studies are typify by the varied nature of the methods used to study the problems and intensive descriptions of a single individual or a single group of individuals so Freud's early works were prime examples of case studies they really are a wonderful starting point they're observational in nature they focus on few if not just one subject but can we say that they're generalizable well no we can't say that they're necessarily generalizable however they can give us a really excellent starting point and may describe circumstances which we see in the broader population if we can confirm that using more experimental methods now we also have a fourth category of non experimental research which again is not on this slide what we're going to talk about which is the survey and you know we talked about the design of various metrics to measure behavior well a survey is one of the most common that we use in the social sciences and a survey is probably something you're pretty familiar with you've probably taken a number of them online and in this method participants are requested to cooperate by responding to questions now nearly everyone has taken part in a survey and we're gonna talk about all these different all these different methods but when you think about the about designing a metric that collects self-report data we're really talking about a survey okay so moving forward so we talked about that here surveys assesses public opinion or individual characteristics by the use of questionnaire and sampling methods so what we mean is is that we are sampling data from the larger population right so the population is what we're studying or what we're hoping to to get an estimate of what is actually happening the general population and we are taking a sample a subset of that larger population through a survey now before we really go into more detail on the various non experimental methods we need to take a short detour to discuss what it is that scientists using these non experimental techniques are trying to accomplish in the first place now you might be inclined to wonder why this is necessary to talk about this because all psychologists are trying to understand behavior right well the problem is that is that it's more difficult to determine that in correlational research and it's often difficult to decide which variable caused the other or whether both variables are caused by a third variable you know we talked about this quite a bit already so this problem has led some social scientists to consider their task to be the discovery of meaning of a behavior rather than the cause of the behavior so these scientists consider their task to be the attempts to understand and interpret behavior rather than search for its causes so essentially the question being asked is not why did the behavior happen but rather what did the behavior mean now social scientists have adopted the term hermeneutics to describe methodology that looks more at interpretation than causation so this term is borrowed from the field of biblical interpretation where the task of the scholar is to find out what the original meaning of a text was to the people originally meant to read it so psychologists who use hermeneutics are a hermeneutic approach may try to interpret the meaning of two people who are holding hands for example so if we attempted to explain the behavior in terms of cause and effect we would have to try to determine what caused pull a hand holding on the particular occasion for the couple this attempt to explain the interaction in terms of cause and effect would be much clumsier than to interpret its meaning to the individuals as a public indication of close relationship as Goffman's 1971 work described so as we proceed through the lecture we'll find that in some places the traditional cause-effect approach seems more more appropriate and in others uh the hermeneutic approach is more natural so what we're essentially talking about yeah when we talk about non experimental methods as we're talking about observational research and there's various subtypes of this but observational research involves recording ongoing behavior without attempting to influence it so this method takes two general forms naturalistic observation and participant observer research so naturalistic observation is research conducted in such a way that the subjects behavior is disturbed as little as possible by the observation process and the observation is made in the environment where the behavior naturally occurs and you've probably seen films showing natural ists or observing Birds and/or other animals from within a blind which is a device for screening the observers from the view of the animals being studied you might have seen something that looks like this in the social sciences naturalistic observation is also referred to as unobtrusive research all right so the idea here is that the term unobtrusive really just refers to the effort that researchers make not to influence or obtrude on the behavior being studied still another term for naturalistic observation is nonreactive research and the term nonreactive emphasizes that the subjects are unaware that they are being studied and therefore do not react to the presence of the observer so why would all of these observational methods take all of this effort into account to not let the subjects be aware that they are being observed well for one simple reason that might sound familiar from previous lectures it's that we do not want to confound the report of the data or the data that we're getting from people knowing that they're being observed so think about this in terms of a reality TV show all right let's say for the sake of argument that there's no screenwriters or no individuals who are kind of trying to weave together a specific kind of storyline to influence the participants in this reality TV show now that you might say okay then we're actually seeing reality well you could say that that might be true except for the fact that all of the participants in the reality TV show all the actors as it were have cameras in their faces so if they have cameras in their faces think about it from your perspective if you had a camera on your face would you feel more natural would you feel more able to express yourself as you would if you didn't know that there was a camera there or would you be acting in a certain way to either look your best or respond to what you think that the person who is taking the film or the video of you wants the idea here is is that if you introduce the knowledge that the person is being observed then it is very likely to influence their actions and their behaviors right so the idea behind observational research naturalistic observation is various types of naturalistic observation in in particular is that we don't want the participants to know that they're being observed or we don't want them to be interfered with in their natural behavior because that represents a confound and that's what we've referred to numerous times in the course before this idea of a confounding variable you as the researcher don't want to be the confounding variable yourself so methods of observe observational research are really as varied as the subject matter being studied so researchers have been very ingenious in devising unobtrusive measures of behavior one broad category of unobtrusive measures is known as physical trace measures these measures make use of physical evidence of some behavior so for example researchers have studied graffiti in school restrooms to discover attitudes towards racial integration of schools smudges on pages of library books to see which pages are most red and grease prints on display cases and museums to see which displays are most interesting to children so these examples illustrate the behavior of a group of individuals can sometimes be studied quite effectively by naturalistic observation even though it may be unethical to observe an individual's behavior in certain circumstances so for example it's illegal to obtain a person's borrowing record from the library right so you know you have to do this within ethical boundaries but there are ways to use physical traces of behaviors to get a sense of what people are doing and surmise why they are doing it now although much has been made of the cleverness of social scientists and using physical traces of behavior natural ists have for many years used scratches on trees to study territoriality and bears droppings to study the eating habits of owls and similar similar physical traces of animal behavior and as the field of psychological research was very influenced by behaviorists physical trace research played a very large role in in in psychological research on humans as well however it clearly is not the only way that we can look at psychological behavior another example of naturalistic observation is provided by Don Zimmerman and Candace West in their 1975 study they were interested in weather patterns of speech would reflect power through dominance of conversation in particular they were interested in looking for sex differences in interruptions and other turn-taking behavior in conversations they tape recorded conversations between sets of two persons in public places around university such as a coffee shop they chose conversations that could be over her easily in the normal course of being in a public place they stayed 10 female to female pairs 10 male to male pairs and 11 cross sex pairs whenever possible they obtained the permission of the conversationalist s' to use their recordings sometimes however people felt people left before the researchers could approach them all personal identifications were edited out of the transcripts of the recordings and there was also informed consent privacy and debriefing that took place whenever possible and what Zimmermann and West found was that males made ninety six percent of interruptions in the cross sex pairs these investigators decided to learn whether this finding was limited to the type of setting or to people who might be already acquainted so they brought pairs of people into a laboratory where they were asked to chat and get acquainted before the experiments before the experiment began and in five conversations between males and females all of the males did more interruptive with men doing 73% of the interrupting on average the amount of interrupting was unrelated - which person talk first so the Zimmerman and West research illustrates a typical admire aggression in observational research from idea to naturalistic observation to laboratory this move to laboratory allows researchers to rule out sources of confounding variables that would occur in the naturalistic situation it also illustrates that observational research is not limited to field settings but often occurs in the laboratory where it's called laboratory observation so another kind of observational research that has yielded important results is participant observer research in which investigators participate in groups and record their observations and one of the most famous of these studies was that of leon festinger's HW wreckin and Stanley Schachter in 1956 who joined a group that believed the world would come to an end at a certain time the group members believe that they would be rescued by a flying saucer the psychologist carefully observed interactions among group members and the effect the dis confirmation of their prediction had on their behavior surprisingly when the world did not end the group members began to be more open and less analytical about their beliefs this research was instrumental in the development of festinger's theory of cognitive dissonance which predicts how people deal with conflicting beliefs we should note that careful records and Diaries are crucial in evaluating participant observer studies because of the increased possibility of subjectivity in these situations so when is participant observe observation appropriate well the usefulness of participant observer research is limited to certain types of situations Jorgenson's 1989 study show really talked about this and it's it's most useful in studying a small group that is separated from the population as a whole when little is known about the group or when the group's activities are generally not available to public view so an example would be what the study that we just mentioned where you might be looking at groups that are cults or criminal enterprises or groups that engage in certain behaviors right so in my own research when doing my own literature reviews on heavy metal music there was a lot of participant observer research on the sociological and social psychological size sides among metalheads and people that were within the heavy metal community because it made up a sub cultural group so that would be very appropriate context to engage in participant observer research another important level of this layer of this to keep in mind is to take the insider in Group perspective which is really taking the point of view of those that you're studying so participant observation is characterized by the effort to view some behavioral activity from the viewpoint of an insider to a situation and the methodology needs to be open-ended it needs to be flexible and needs to be opportunistic the approach to theory is often hermeneutic emphasizing interpretation and understanding some participant observers will take the role of a central participant because it would be impossible to learn much without doing so and as in studying the people who hang out in a bar in a really rough section of town that would be an example other times a researcher can stay more on the periphery as in studying and organization such as a religious group by definition participating in a group leads to problems of objectivity the researcher must strike a balance between taking the viewpoint of the group members and maintaining scientific objectivity a researcher who's studying police officers tries to take the point of view of the officers but he she or they may find that their attitudes towards criminals and towards society as a whole start to match those of police who may happen to be more prejudiced or more authoritarian than the population as a whole based on the nature of the culture their own experiences or they may have different views but the idea here is is that it's it's not unheard of for the researcher to really truly become part of the group it's also not unheard of for researchers to be converted by a religious group that they were studying some researchers maintain their objectivity by regular contact with other researchers who debrief them about their experiences so that it's really important to keep in mind and engaging in this kind of almost anthropological research that there are certain risks associated with it and the loss of objectivity certainly is one now sometimes it's possible to be open about the fact that you are there to study a group as when a person openly enters a fundamentalist church for the purpose of research and is welcomed as a potential convert this is known as undisguised participant observation right and this is when we talk about issues of gaining access by contrast certain groups are hostile to the larger society and suspicious of anyone who shows interest in them admitting that you are there to study them might result in your getting kicked out or worse in that situation the researcher would use disguise participant observation to hide their true purpose from the people being observed in addition it's often unrealistic to expect researcher to inform each and every member of a group the first time they meet that he/she are they is there for the purpose of research it would be highly artificial to begin every interaction with a new person by saying that your researcher and there's no hard and fast rule about whether to inform the group we should consider the ethics of that also when we talk about informed consent it's best to be truthful where possible and to adopt a disguised strategy only when necessary so kind of harking back to our discussion of the ethics code using deception only when necessary but still let's think about this in terms of the ethical ethical nature or lack thereof so obviously there's certain problems with observational research two important problems are present in participant observer research first by entering the group the observer is the observed by definition changes it to some extent therefore the act of observing the behavior changes the behavior to be observed which is the big problem a large group may not be influenced much by observers presence whereas a smaller group may be influenced pretty considerably and in general participant observer research is done in unusual groups that can absorb an observer whose presence would have little effect the second problem is the ethical question of invasion of privacy so we talked about the ethical question of deception but this invasion of privacy piece is huge participant observers cannot always obtain informed consent from the people they study some researchers hold that participant observer research is therefore always unethical and that's something that we should think about is it always unethical others point out that professions professionals such as journalists are permitted to engage in this type of practice they argue that if psychologists do not perform participant observer research they're withholding the application of psychological techniques and insights to social problems so these are of key importance and of key concern so what about archival data we mentioned that a little bit earlier in the lecture well the term archival research refers to research conducted using data that the researcher had no part in collecting and in fact a lot of research done in the social sciences uses archival data it's often a lot easier if you have a good data set that was collected for researching something else or answering other research questions to use that data to assess your own research questions in fact it's one way of preventing a very long and drawn-out IRB application process because usually as long as you have a right to that archival data that kind of research is either a exempt or subject to an expedited review so essentially the researcher here simply examines or selects the data for analysis and archival research is appropriate in many instances so data that bear on the hypothesis may already exist in collecting new data may be wasteful or ethics or logistics may make it infeasible to conduct an experiment relating to variables of interest so think about research that's studies suicidal behavior or completed suicides that looks at sex crimes both topics would be inappropriate for experimental research but if there's pre-existing data on that it would be really perfect for our Chi ville research because you don't need to do anything unethical in order to answer a lot of questions that you might have for a new study now obviously archival research has its limitations however first most archival data are collected for non-scientific reasons governments and private agencies collect the data for their own purposes and such data often do not suit the purposes of a scientist for archival data to be scientifically useful the agency collecting the data must ask questions similar to the scientists or must inadvertently collect data that are valuable to the scientist second because archival research is by nature carried out after the fact ruling out alternative hypotheses for particular observed correlations may be difficult so a researcher who relies on archival data is at the mercy of any biases that may have occurred in collecting the data so police records are notoriously subject to bias many categories of crimes are seldom reported to the police and it's estimated for instance that only one in four attempted rapes is reported so if you have data on those that are reported it may be skewed if you're trying to do forensic psychology research and that's according to the Bureau of Justice Statistics in their 1990 publication so this isn't just coming from me and my views on it so there are certain considerations to keep in mind when using archival data primarily what is the source of the archival data and what biases would you need to keep in mind and also talk about if you choose to move forward with a study using archival data from those sources because you would need to include that in your papers that you would be doing or your research projects that you would be doing a successful use of archival research can be seen in David Phillips analysis of motor vehicle fatalities in his 1977 article authorities had long suspected that many automobile fatalities were suicides that were completed rather than accidents and Phillips hypothesized that if suicides are triggered by reports of other suicides then motor vehicle fatalities should increase just after the publicized suicides he studied all motor vehicle fatalities in California during the week that followed the reporting of suicides on the front page of the state's two largest newspapers then he compared the fatalities with those in a control period in another year he found a 9% increase in the number of fatalities in the week following the suicides with a maximum increase of 30% on the third day after the stories appeared in those newspapers by correlating the increase in fatalities following each story with the total circulation of all newspapers that covered the story he showed that the publicity was likely responsible for the increase further analyses bolstered his conclusion that the increase was the result of the publicized suicides details were similar between the suicide stories and the types of accidents that increased following the stories so this is really interesting stuff this is a really good way of using archival data to answer questions about human behavior that had nothing to do with initial psychological variables it was just essentially accident and crime statistics and piecing those together to talk about human behavior became something that was very valuable as well so case studies I promised you that we talk more in detail about this and case studies constitute a category of research that is difficult to characterize with a simple definition because case studies often include the use of observation and archival methodologies the distinction among them are not always clear nevertheless it's possible to say that case studies tend to involve an ongoing situation that presents itself for investigation the principal characteristic of case studies is that they examine individual instances or cases of some phenomena sometimes this is an individual person as in a psychiatric case other times it's an individual institution as in the case of mass hysteria at a school for example or it can be an individual method or treatment such as the effectiveness of a new surgical intervention on patient's health the individuality of the situation or case characterizes a case study so robert yen's 1994 article defines a case study as quote an empirical inquiry that one investigates a contemporary phenomena within its real-life context when to the boundaries between phenomena and context are not clearly evident and in which three multiple sources of evidence are used so according to Ian's definition the multiple approaches distinguish a case study from other non experimental methods many case studies result from problems that present themselves to researchers as opportunities that must be grasped quickly or are lost little time may be available for planning and the study often must be conducted under difficult conditions an example of case study research is the study of unilateral visual spatial neglect reported in a 7 year old boy by Rebecca Billingsley and her colleagues in 2004 in this neuropsychological disorder patients ignore stimuli in one half of space so patients will draw a flower with petals only on one side for example or eat the food on one half of their plates so frequently this disorder is observed after a stroke in elderly persons but in this case it was observed in a young child so it was really interesting and disturbing at the same time when the little boy first visited the medical team he had experienced difficulty in moving his right hand they had no trouble in paying attention to anything in his environment an MRI of his brain revealed the presence of a cancerous tumor near the ventral foul of the brain and after surgery to remove the tumor the child began to exhibit symptoms of unilateral neglect so just leaving off the beginning a few letters when attempting to read long words and drawing a clock with numbers only on the right side and actually one little side note is that a neuropsychiatric screening tool is for individuals that may have visual spatial neglect is just to tell them to draw a clock and so if they only put the numbers on one side that suggests that that suggests that this issue may be present so getting back to the story at hand fortunately six months after his surgery he was able to overcome these difficulties and again attend to space somewhat normally now this case assists in demonstrating specific brain regions that are tied to unilateral neglect and the development of visual spatial attention and this visual this study of visual spatial neglect is typical of case studies the efforts of the physicians and psychologists were directed towards a practical problem the case appeared unpredictably and required prompt attention and a multi multidisciplinary team approach was used to elucidate the problem members of each discipline used several techniques to rule out explanations and narrow the possible causes other case studies are quite varied in fact one of the few generalizations possible about case studies is that it's difficult to generalize about them it's difficult to generalize anything about the idea here really is is that it often focuses on a single case or a single group and it really is difficult to generalize but it serves as an excellent starting point and can reveal very valuable data but one of the the difficulties here is that you can't generalize because there's too many potentially confounding variables and you only have usually one or a few data points there's not a lot of standardization that takes here and one thing you may remember from our discussion about experimental methods is that standardization is key you want the conditions to be as identical as is humanly possible between subjects so you can truly say something about the nature of the behavior that you're trying to measure and the conditions you're trying to measure that behavior in absent any potentially confounding variables so interesting stuff important stuff as well so when we talk about research using observation and case study methodologies we often start with some general questions in mind but without specific categories of behavior defined in fact this characterizes a lot of observational research so what you want to do is you want to have field notes which are really a notebook of all of your observations and interpretations so the the next couple of minutes I'm going to be talking to you about how to put together good field notes for these kinds of observations because in any good research methods class it would be a bit of a how-to guide in documenting your data especially in observational research especially and non experimental designs where you want to get as much of your data down and as much of it accurate as possible to eliminate potential confounds so the first point would be to be systematic so you should keep a notebook of all your observations and their interpretations and make certain that you think about the categories of behavior that you're concerned with and clearly define your behaviors of interest you should also decide the length of your observation and observational intervals so checklists and other recording devices that you could use can help considerably in keeping clear records now this documentation may sound kind of daunting and unfortunately it often is you may be so busy observing that you don't have time to take notes or you may be in a social situation where note-taking is impossible either cuz it'll inhibit the behaviors you're observing or because you're taking part in the situation in such cases you should consider inconspicuous recording devices with obvious due consideration to ethical issues or at least write up your observations as soon as you leave the situation you might be tempted to skimp on writing up your notes later because you're tired or you think you'll remember what's important this is often the case and taking notes after class you may experience this but if you think you won't forget you're probably wrong so just try to remember everything you did a week ago well what did you wear II say what did you do probably not going to come up with a lot of information so your notes should include as much detail as possible including the time and the setting later on the significance of these details may make sense in a way that they don't when you're writing down those notes and you may want to write your notes in stages you can just jot down some brief notes as you work and elaborate them elaborate on them later however remember that a memory is reconstructive so the more you can get down as soon as possible the more accurate you're gonna be the other pieces be selective the problem of having too much to observe can often be solved by taking samples of behavior instead of watching television all day long for a week to observe the content of commercials you might watch only the first batch of commercials after the top of the hour instead of watching all channels you might focus on only one instead of watching everything that all children in a daycare center do for a whole day you might watch four instances of one kind of behavior such as fighting or only those behaviors that you're prepared that you prepared on your checklist so you know keep in mind there are ways to kind of hone in on what you're looking for and record your raw data use audio or visual data if at all possible so that brings us to content analysis and when dealing with textual or photographic materials you don't have to take field notes because the material is already in a permanent form the problem remains however that of deciding how to deal with what may be a large body of material suppose you're investigating the sexually explicit content of magazines for example how do you decide what to record there are two basic approaches to content analysis the first one is based on counting the frequency of some objective measure such as frequency of certain words that may be sexually charged now this would be known as coding the manifest content of the text because it's relatively simple to count the number of times the word kiss or love or whatever appears in a body of text the coding of manifest content is very reliable there may be a problem however if words are used in different ways that could be a big issue the word love appears many times in a certain chapter of the Bible where it has nothing to do with sex you would draw the wrong conclusion about the point of the chapter if you just counted the word love the alternate method of analyzing the content of text or photograph is coding its latent content with this method the researcher reads a passage of text or looks at a photograph and interprets the presence of a particular theme that same passage the Bible would easily be seen to be about what might be called brotherly love rather than a romantic love right and because latent content analysis is inherently subjective though it runs the risk of being less reliable than a manifest content analysis so it's a good idea to do both manifest and latent content analyses when you're doing work like this and compare the results if the results turn out the same by both method pretty strong evidence of the validity of your results if they differ you can look for the reasons why they might differ so what's the reliability of content analysis well whichever method of content analysis that you choose there is a problem of reliability if you use latent content analysis another researcher might interpret the same passage differently even with manifest content analysis there's the possibility of a coders making mistakes when going through large amounts data so to establish the reliability of your coding scheme it's important to use at least two coders so if you're part of a research team this is helpful if you're helping somebody out with their research when coding raw data you can help them and they can help you with your research but having multiple coders helps now there are various techniques based on statistical correlation that are available to quantify the reliability of coders and coding schemes but the the primary method that I think is is used with this from my experience and from the experience of the researchers that I've worked with is a reliability analysis so it's a correlational statistical method of determining a degree of reliability between two coders and the the idea here is that of inter-rater reliability the idea that you want those two coders to be interpreting the same things in the same ways as much as possible if it diverges too much then you it calls into question the reliability of your coding scheme so an example of a Content analysis is that of reject Bledsoe and Rasmussen in their 1991 study and they use content analysis to study personal ads and newspapers so this is kind of old-school so think before online dating apps so think single white female desires to meet professional male who enjoys dancing long walks on the beach etcetera etcetera so the researchers were interested in discovering which ads were most successful in eliciting responses from readers and categories they coded were statements about looks such as attractive petite tall son status such as affluent or employed or professional on intelligence on sincerity and age and these categories were based on previous investigations so the category looks was used to code information that other investigators had labeled attractiveness appearance sexuality or physical status because it was an inclusive category that encompassed all others similarly they chose status instead of financial security which in a previous which a previous study had used because it would incorporate social prestige so a career person or a gentleman or a professional as well as material possessions so two of the author's coded all the ads independently then they got together and discussed all the ads until they agreed on the coding of each one after the coding was complete they sent a questionnaire to the advertisers through the newspaper asking about the results of their ads they found that ads placed by younger women were more successful than those by older women and in contrast ads by older men were more successful successful than those by younger men so we could kind of see some of some cultural biases coming through in the results of that research the authors were surprised to find that advertisers who claimed to have good looks and high status did not receive many more responses than those who did not and the researchers interpreted this finding as an indication that most of the ads contained such claims so little information was to be gained by considering looks of stats the content analysis was an essential component of this study and then define the independent variables against which the outcomes were correlated and inter-rater reliability was huge because everybody had to really be on the same page and how they were coding the information so when we talk about when we talk about your procedure in terms of recording methods for experimental research procedure is a bit different from method a method is a broader term that encompasses all aspects of the study including the logic of the design and the steps for carrying it out but procedure refers only to that latter point what the researcher does in translating the design into action I'm picking this apart because method when you think about procedure you might think oh this is the method section of all these papers that I'm reading and the methods section that I'm going to be writing in my term-paper well it's a component of it but it's not exclusively it so the design for example of your experiment may be an experiment with two conditions that are both experienced by each subject so procedural concerns include whether each condition is tested on the same day or different days procedure also involves instructions and how they're given the debriefing and so forth the actual procedure of your research many details must be worked out in the course of translating a design into practice and the step of going from design to procedure can be a difficult one for a lot of people especially students who are just learning this because it involves going from a logical plan in one's mind to a practical plan of action in let's say a laboratory study after the design is completed you should develop a clear idea of the exact procedure or protocol you will follow so what's a protocol well this was mentioned a little bit earlier in the course but a protocol is a list of the exact steps needed to test a subject from start to finish a written protocol is helpful for beginning researchers especially when the one researcher will help run a given study and if this seems familiar to you from our ethics lecture it should this is required a protocols required to submit to your IRB before your study begins write a detailed list of how this is going to take shape now once a protocol has been developed you probably should do a pilot study to find the bugs in your procedure there are almost always problems to be smoothed out one of the researchers let's say your advisor or a friend should be run through all the steps of the study exactly as it will be carried out and although the temptation to skip the pilot phase may be great you should resist it nearly every experiment that I've performed or others that I know have performed without testing some pilot subjects has been either disappointing or had a lot of bugs in it because this is where a little effort can greatly increase the precision of a study the idea of a pilot study is you're going to list all the steps that a subject goes through in a study you're going to execute that in a smaller scale and then you're gonna work out the bugs right so it's it's the idea here is is that you're working out the bugs before you do the full procedure and the the last piece for you to consider for the purposes of this lecture is guarding the integrity of the data and all the care expended on designing and conducting a study is really a waste if this if the data is compromised by care carelessness and recording or handling and in fact the IRB is going to want to know that you have a plan for guarding the integrity of the data so before the study begins the researcher must have a plan for recording and handling the data a good procedure is to keep all the data and other research details such as let's say you're doing a study on psychotropic medication so the dosages or if it's a behavioral study this stimulus settings and like in a notebook right so you want to keep this together in a notebook all pages should be dated and identified in such a way that they can be replaced if any sheets became lost back up your data I tell students about this with their term papers but it applies especially for your for your research projects as well often it's inconvenient to reproduce a blank sets of data sheets with spaces for all necessary information so that nothing will be overlooked so data should be placed in a file and kept in a secure place they should never be carried around in a briefcase or a backpack so so it could be potentially lost I've heard horror stories of people leaving leaving raw data on buses or Subway's or or in their car and their car's been broken into so it's really important to guard the integrity of the data and the safety of the data because it is considered an ethical violation if something happens to data with identifying information so all important things to keep in mind all right so that concludes our lecture for this week you will have your discussion questions posted and the dates are the same as they always are for each week so if you have any questions or concerns please do not hesitate to message me on canvas I look forward to reading your discussion posts and I look forward to our lecture next week as well all right take care