welcome to the fourth video in this series of critical appraisal modules in this module we will be focusing on the critical appraisal of cohort studies using the critical appraisal skills program our cusp approach in the previous module we spoke to you about our cities and their importance in healthcare particularly in terms have been able to tribute causality you may ask why do we need observational studies if the RCT design is so strong in terms of attributing and causality but sometimes it isn't possible to test some hypotheses and trials it's also sometimes unethical to undertake a trial of an agent we believe could be harmful and it is not feasible to study some very rare outcomes and trials for this reason observational studies do have an important place in healthcare generally in observational studies in health researchers are interested in risk factors our exposures and outcomes often diseases unlike in experimental studies researchers do not manipulate exposures and observational studies they simply observe what is occurring and then they can calculate measures to quantify the extent of the risk for the learning outcomes we will introduce you to cohort studies and describe their purpose and value in the context of healthcare research and show you how we can critically appraise one using the class checklist we will also talk about the risk ratio how to calculate these and how to interpret them in cohort studies finally there will be a link to a short quiz at the end of this video which will give you the opportunity to test your knowledge and concepts we will have discussed using multiple questions and answers a cohort study is the strongest research design of the observational studies it generally involves the researcher identifying research participants who do not have the outcome of interest the researcher then classifies participants according to their exposure status and follows the participants over time to see whether or not they develop the outcome in question to take a simple example if a researcher is interested in whether or not smoking causes lung cancer the research you would identify a group of people you do not have lung cancer and then classify them according to whether or not they are smokers the researcher would then follow up the participants over time to compare rates of lung cancer between the groups in the cohort according to their smoking status so how do you express the different rate in risk between exposed and unexposed members of the cohort the most common method of doing this is the risk ratio the risk ratio is the ratio of the incidence of disease in the exposed group to the incidence of disease in the unexposed group incidence refers to new cases of a disease relative risk can quantify the strength of an association between exposures and outcomes if the relative risk is greater than one it means that the exposure increases the risk of disease the higher the number the greater the risk if the relative risk is less than one it means that the exposure decreases the risk of disease the lower the number the more the factor is protective if a risk ratio is exactly then it means there is no difference in risk between exposed and unexposed individuals so let's take an example in which we'll calculate the relative risk together we'll look at some fictional data imagine that we're still thinking about the relationship between smoking and lung cancer we might have conducted a cohort study and obtained the following results we can calculate the risk ratio by first calculating the risk of developing lung cancer in the exposed group then by calculating the risk of developing lung cancer in the unexposed group and then by dividing the risk in the exposed group to the risk in the unexposed group I've given key sections of the table the letters a b c and d as this is a common convention to calculate the risk of developing lung cancer in the exposed group it's a divided by a plus b which gives not point 8 5 to calculate the risk of developing lung cancer in the unexposed group it's C divided by C plus T which gives not point naught 5 to express the risk of developing lung cancer in the exposed to the unexposed group it's not point 8 5 divided by naught point naught 5 which gives 17 so that means that people who smoke are 17 times more likely to develop lung cancer than non-smokers according to this fictional data cohort studies are an extremely useful study design for quantifying the strength of association between an exposure and an outcome however like on research that and healthcare their quality can vary so it is important that readers of cohort studies critically appraise the policy of the research surely the caste program has produced a checklist for critically appraising cohort studies which you can access by following the link below this video let's have a look at this checklist and then we'll apply its use to a sample cohort study we can see that because cohort studies checklist again separates the three key principles of critical appraisal of validity trustworthiness of results and value of relevance into three sections a B and C respectively let's see how they can be addressed with an example the study will use to work through this checklist is by Gerhardt's at al 2015 on lithium treatment and risk for dementia in adults with bipolar disorder population-based cohort study which you can also access by selecting the link below this video the first question in the checklist is to consider whether the cohort study examined a clearly focused question the answer for this study would appear to be yes as the study considered the association between lithium and dementia risk the research question is contextualized against the biological background of lithium inhibiting glycogen synthase kinase 3 and enzyme implicated in the etiology of dementia the second question looks at whether the cohort was recruited in an acceptable way this question is mainly about selection bias bias is a systematic error in a research study which results in incorrect measurement of the relationship between exposure our intervention and outcome selection bias refers to bias in terms of the way participants were selected as research participants ie when the research participants truly representative of the research population are are the participants in some way and typical this study's participants of people aged over 50 with a diagnosis of bipolar disorder from 8 large US states and Medicaid insured u.s. population Medicaid is a social health care program for Americans with disabilities are lower incomes the third question considers a different type of bias namely measurement bias this refers to bias in terms of the way exposure and our outcome a measured which could lead to an incorrect estimate of the relationship between exposure and outcome it is important to consider factors such as whether or not exposures were assessed via self-report as this may be affected by social desirability bias are the fal ability of memory the exposure in this study was lithium use and it was assessed by a health administrative data so it is more likely that the measurement of the exposure and outcome are more reliable than self-report measurements this question relates to a session for possible measurement bias in measuring outcome which in this case was development of dementia things to consider in this criteria relate to whether or not the outcome was measured subjectively or objectively with objective measures of outcome been generally more reliable and to consider whether or not Assessors were blind to exposure status this means whether or not to assess knew whether or not a person had or have not been exposed as if they know this it might affect their assessment of outcome particularly in more subjective outcome measurements sometimes however the fact of whether or not Assessors are blind is not so essential if the outcome is clear-cut for example this studies outcome is more of a factual objective outcome less likely to be affected by Assessors knowledge of exposure status and the use of health administrative data lessens the likelihood that the results of the study have been compromised by measurement bias in assessing the outcome this question relates confounding a confounder is a factor which is independently associated with an exposure and an outcome and which can either hide a true relationship between an exposure and outcome or make it seem like there is a relationship between exposure and outcome when in facts there is no relationship possible confounders can often relate to age gender or health related behaviors such as dietary factors or whether or not people exercise it is possible to estimate the impact which confounders may have on a research study by statistically adjusting for them when analyzing the relationship between exposure and outcome in this study the Astor's present an adjusted statistical model a model which adjusted for gender age and ethnicity all of which are common confounders and a statistical model which suggested for age gender and ethnicity as well as other factors which are potential confounders the authors do note that Nassim treated patients were found to have a lower risk of developing dementia in their study but also that lycium treated patients a lower rates of baseline cerebrovascular disease and diabetes most of which are actually risk factors for dementia and therefore factors which could be confounders this question relates the time spent and following up for the cohort cohort studies are generally undertaken over several years and it is important that the study follow-up period is long enough for the outcome to manifest itself it's also important that the study tries to collect data from as many people who started in the cohort as possible and not to allow people to drop out as the people who do drop out may not be typical of the ones remain in the study and this again could lead to an incorrect measurement of the relationship between exposure and outcome the maximum follow-up time period in this study was three years and judgment is required as to whether or not this outcome period is long enough to allow development of the outcome versus the outcome will be occurring in this study the authors found that 301 to 365 days of lithium exposure was associated with reduced dementia risk but that no association was found for shorter duration of exposure to lithium the measurement used to express the association between lissome exposure and dementia risk is the hazard ratio which is similar to the risk ratio we looked at earlier but the hazard ratio accounts for the rate or time period at which event to happen and instead of just looking at whether or not an event happened the hazard ratio for 301 to 365 days of lithium exposure was naught point seven seven meaning that lithium exposure is protective because the hazard ratio is less than one like risk ratios if a hazard ratio is under 1 then this means that an exposure is protective whereas if a hazard ratio is higher than one then this means that an exposure raises the of an outcome occurring the precision of results are represented by confidence intervals confidence intervals relate to the range in which the true value lies a research study generally involves a sample rather than everyone in the research population and even in a robustly conducted study there will always be some error in the sample compared to the population a confidence interval quantifies this by acknowledging that while the sample produced a certain value the true value in the population is likely to be within a certain range a narrower confidence interval indicates higher precision of results so a confidence interval of three to far around the risk ratio of three point five would be more precise than a confidence interval of t27 around the same risk ratio three point five the believability of results can be assessed by considering the factors we have previously discussed such as confirmed in our bias it's also important to consider whether the results may have been affected by chance assessing the believability of results also requires judgment and consideration of factors such as the biological plausibility of the relationship between exposure and outcome and the contextualization of the cohort study in the wider body of research in the field questions 10 11 and 12 of the cusp checklist follow on from this theme and require judgment to consider the potential local applicability of the results the fifth module in this series will look at case control studies and we will be following a similar format as we used in this video to appraise an example of a recent study thank you for listening these training videos have been developed by the Cochran common mental Dada's group at the University of York with support from TSS can wear valleys NHS foundation trust Northumberland Tyne and Wear NHS Foundation Trust and the Economic and Social Research Council if you would like to test your knowledge on the topics introduced in this module please follow the link below which will take you to a short online quiz