I'm Kerry price I'm a clinical information estat the Welch Medical Library thank you for making time in your day to come to this webinar I think you'll find koba's to be a really super tool I'm happy to share share it with you and I've put my email and our web address here on my first slide you're always welcome to contact me to follow up today's plan is to give you a brief overview of systematic review methodology and typology since that's the kind of review that confidence was really created for then we'll talk about the key features of confidence I'll have some screenshots but I'll do my best to do a live demo where I can then I'll send you on your way with some helpful resources to get you started in your next review with covenants and I'm going to do my best to monitor chat but my caveat here is that I'm on my laptop with a single screen so while I won't be monitored monitoring it constantly I will try to check in periodically and Sharon and Mike who around this call the other librarians feel free to unmute yourselves if I'm missing an important quick or answer a question in the chat if you see one that you can answer and as always I think it's pretty informal so feel free to unmute yourselves if you'd like to pop in with so what is a systematic review by raising your hand or putting a yes or no in the chat box how many of you have been involved in a systematic review before yes yes see a couple yeses okay so this is a familiar methodology to some of you all right so systematic reviews came about roughly in the 1970s and 1980s after dark dr. Archie Cochrane wrote effectiveness and efficiency random reflections on health services in 1972 he caught on the medical community to organize critical summaries based on evidence provided by randomized controlled trials so the first reviews were reviews of clinical effectiveness really looking at the evidence from randomized controlled trials and then we had dr. professor David Sackett sometimes called the father of evidence-based medicine who tasks clinicians with making conscientious explicit and judicious use of current best evidence in decision-making for patients for clinical expertise and to use the best external clinical evidence from systematic research if you are interested in undertaking a systematic review here are some resources I'd like to point out it's important that before you get started you have an idea of what's involved because a systematic review can be a very onerous and lengthy process you'll definitely need a team a formulated research question a protocol and an idea of what it is that you're looking for I should also note that the average time to complete a systematic review is 67 point 3 weeks so over a year and that's according to an article by Borah at all which I think is in the references of this slideshow and if it's not I'll make sure I'll share it later I'd like to visualize the workflow of the systematic review process with the prism of flow diagram this probably looks familiar to some of you so starting at the top you'll see that the research team identifies records through database searching and this is comprehensive and systematic searching so we're not looking at like five or ten articles here we're looking at hundreds or thousands probably and with the help of a librarian of course you need to do the database searching with the help of a librarian and you'll identify additional Records through other sources so I don't know if you can see my cursor but the top box on the is records identified through database searching and the top box on the right as additional records identified through other sources so you might be aware of unpublished literature or you might have found something on Google that you're not finding in a bibliographic literature database and then you'll move through the part of removing duplicates and screening and that's where Covenants comes in covenants is an excellent screening tool that's going to save you tons of time here are a few different types of systematic reviews this is according to the Munn article that's linked at the bottom of this slide so systematic reviews have generally been clinical in nature but you'll see here you might see a framework that fits your topic a little better than a review of effectiveness so qualitative for example or I think there's one for what's psycho metrics and surveys and more qualitative data so there are different types of systematic reviews and you might find that you're interested in one of these and I encourage you to read this article it's a really a super article to help you develop your question and proceed through the systematic review process and even though I'm not an expert in this area I've put together some resources for translating the systematic review methodology into the education discipline this is not my area of expertise I'm strictly a clinical librarian so you need to check out these resources and decide for yourself if a systematic review approach is right for you that's a really quick overview of systematic reviews and really the motivation for a tool like confidence which is now being used more and more by other teams doing other types of research and Cochran which is a systematic review Cochrane is an evidence-based medicine research collaboration publishing systematic reviews that are very rigorous and so now let's talk about confidence and let me check the chat box to see if anyone has had any questions scoping review scoping review yes I think in my references at the end of this slideshow there's an article that's called when to do a systematic review when to do a scoping review it's by Mun at all and I'll make sure you get a copy that it does talk about house scope interviews can be used to inform policy map the current evidence find out where there are research gaps and inform future practice and Juliana's says happy to see these resources about systematic reviews that might be helpful outside of the health and medical environment thanks you're welcome I think it's really a growing and expanding topic and methodology so let's move on to covenants really the reason why we're all here ovince is a web-based platform means that you can access it from anywhere and you don't have to download it to your computer and it's a systematic review production tool created by a nonprofit team out of Australia and as of 2015 covenant is the standard production platform for all Cochrane systematic reviews I want to note that confidence is not a citation citation management program you can't write your papers with it you can't do in-text citations and bibliographies so you'll still need to use a citation management program and you can check out a few of the different kinds on our Welch Medical Library citation management guide to help you make an informed decision it's really a matter of preference let me open this link for you think I'm sharing this screen can everyone see a guide so the tool that Hopkins supports is called refworks legacy refworks and ProQuest refworks which is the newer version you can also purchase and note and then Mendeley in Zotero are free up to a very generous limit and these are citation management programs so after you get past the point of screaming and doing your textual synthesis or your evidence synthesis you may need to move back to one of these so that you can write your paper and insert your references and create bibliographies alright let's go back to the slides key features of covenants it has a lot of really super features and people just rave about so it's really the first step here that people rave about and that's the title and abstract screening the others are optional so confidence is going to allow you to move through that first step of title and abstract screening with ease and that's really what I want to show you today and I know that some people on this call used it for title and abstract screening so feel free to unmute yourselves chime in if you'd like about how that process worked for you and I'll pause here for just a moment if you'd like to do that hi this is Julia I'll just say that yes I'm one of the ravers we used it for a project and it allowed us to go from start to finish in a modest literature review in less than a month and there's no way we would have been able to do that without tearing our hair out and without using the covenants and you really think Harry for introducing us to it I'm really glad great that is super to hear I haven't heard anything bad about it but that doesn't mean that there aren't other tools out there that do similar things but confidence is a really super tool so the first thing you'll need to do is create an account with covenants following these instructions and I will make sure that these are sent to you and in fact let me do that right now oh if I go to our watch and you'll have different entry ways you'll have the Eisenhower library but the way that I go in is through the Welch medical library under research tools if I click on covenants here are the instructions and the links for signing up so the first thing you need to do is create your own account and once you've created that account then you request an invitation to the JHU Welch medical library which gives you access to our subscription so as your as yourself you get exactly one systematic review and under the JG Welch medical library umbrella you have unlimited reviews so I'm putting this link into the chat box right now and you'll be able to follow these instructions to set up your co-variants account you and then let's proceed through I am strongly encouraging you to work with your librarian and that's because you'll see later that covenants only accepts certain file types they accept PubMed XML and note XML in something called ris format so if you can work with a librarian to get your search results aggregated into one file type which is typically ris format you'll have a lot better luck importing it into covenants without any problems I saw a chat question come up the invite process is very quick and it's also listed on the Sheridan library's website under databases thanks for that clarification so once you've once you've signed up you've gotten your invitation and you're ready to start your review this is what you'll see this is your home page your home dashboard this is my screenshot so you'll see that I have two reviews here called ventilator-associated events events and laryngeal injury post excavation so you can ignore those though they'll only show up in mind when you're ready you click start a new review and then you make sure that you're doing it under the JHU Welch medical library and not under yourself because you only get one review so make sure you click the radio button for JHU Welch medical library and then click create review you before you upload any records to your review you need to set up your review in settings so you click on settings up here at the top where the screenshot cursor is and it's circled with dark blue and you'll be able to give you a review a name and all of these are optional I don't think any of these are required you'll be able to input the date of the last search document your search strategy which to be honest with you I usually document it another way I don't usually document it in code and you'll be able to review citations if you're updating a previous review and set the number of reviewers required to screen to one or two so in a true systematic review you would need to systematic sorry to reviewers to avoid bias but if you are planning on working on this alone and it's not a true systematic review you can set it to one same with full text screen you can set it to one or two reviewers and then you can agrement and remove reviewers and they don't have to be from Hopkins they can be from any institution so that is kind of super I just learned that recently when you do sign up you'll need to sign up with your JH mi or JHU that edu email but you can invite different people to your review who aren't from Hopkins so you can really collaborate with this tool you'll be able to modify your team settings which has user privileges who can do what you can delete you can add and then you'll have to add your inclusion and exclusion criteria so when you are setting up to do a literature review of course not all the literature that you pull will be perfectly relevant you'll have to decide are we looking at articles in English are we looking at randomized controlled trials are we looking at observational trials do we need a article that mentions a specific instrument or measure and so you can put those inclusion and exclusion criteria into covenants and you'll also be able to set up something called highlights which is where you can set up certain words to be highlighted so that it makes it easier for you to screen and you can also last but not least add study tags so if you are working with mixed meth or you just want to be able to tag certain references or records with different categories you can set that up before you get started so here I'm going to exit a slideshow and take a look at covent in real time so I have already logged in I think yes and I'm going to start a new review today I have some references I've set aside for us so I'm just going to do a test review and then the first thing it asks you is are you creating a Cochrane review well unless you've been invited by the Cochrane Collaboration you are not doing a Cochrane review so for most people the answer is going to be no and then you get to name your review just so I remember later what I've done I'm going to call it the education webinar test you'll see that I have zero reviews left but unlimited reviews under the JHU ultra medical library I will click that and then create review let's see I can hear it's already pops me automatically into settings if it hadn't done that or if I needed to come back to settings let's go back let's go so here's my confidence dashboard and where did my review just compared us so they're listed from earliest to latest so my most recent review is at the bottom I'm going to go back into education webinar tests you see the steps that I'll need to take but before I get started here are those settings so I'm gonna click on the settings wheel and here are the things we just talked about you can set up the date of last search it was today and you see it's a European format you can put your search strategy here if you'd like I'm often searching multiple databases so this doesn't really work too well for me I prefer to keep a separate document if you're updating a current or existing review you can put the citation here and then you can set your reviewers required for screening today it's just me so I'm going to set that to 1 and reviewers required for full text review I will again set that to 1 and save so that part is set and if you look up here at settings there's review of settings and the next step is to add and remove reviewers so I'm going to go in here myself I've already there and I will invite another reviewer and I will do share yes that's fine JH u dot edu okay I was gonna look it up okay so I can invite Sharon to this review it's a pending invite I can see when she's accepted it and now I'm working with two people here I can add additional people if I'd like so if you do have two reviewers it doesn't have to be the same two reviewers that look at every article or record it can be just a minimum of two reviewers so you could potentially have figure four people working and two people reviewing each article so I could continue to invite but I'm going to move back up to my team settings and take a look what's next so here this is kind of tricky I can see who's done what so right now Sharon hasn't yet accepted her invite so her listing is not here but once she is I'll be able to see what she's done and what's left to be done so you won't let me to see if I can show you in another review that I have going and go back back back I think you can yeah I'm going to click on my ventilator-associated events I'm going to go to my settings I'm going to go to my team settings and here we go I can see that and this was a review that was never completed I have done nine dan has done eight and VIN SIA has done zero and this is where you can also set the rules where everyone can do everything or you can manage the rules so that certain people have to screen and only certain people can resolve conflicts so if you do have two reviewers and there's a conflict and I'll show you how that happens later you'll have one person who needs to settle the conflict so you can pick that person usually it's your team lead I'm gonna try to go back out to my review here think yep just click on the icon up at the top left and I'm going to go back up to settings so team settings so right now I'm going to leave it on everyone can do everything if I were planning I'm moving through the full text screening and confidence I can do the same thing and the same thing for data extraction looks like Sharon's accepted the invite so she's there so you can see how it's going you won't be able to see how people voted but you'll be able to see how the progress has gone because it's blinded next we'll move the criteria and exclusion reasons now today I have pulled some examples that are really sort of just a sham I don't have a specific topic in mind but if we did let's say has to be an observation has to be 1990 English and I don't actually encourage you to make a language bias I'm just showing you a language limit because it can introduce bias but if you really don't have the capability of having an article translated you might make that as a criteria needs to be talking about adult education and then on the other side you might say not what K through 12 not pre 1990 so there's there's lots of ways you can do inclusion and exclusion criteria it might have to do with geographic location population it might do with language timeframe a measurement tool an instrument or a survey so you can set all that up here and when you're ready click Save and then I mentioned highlights oh yeah let's go to highlights so if you wanted to see words highlighted you can type them here interprofessional I want to see it highlighted for inclusion I think it will be highlighted in green and for children I want to see let's say children I set up some highlights here so if a record has interprofessional it'll show up highlighted in green if it has children or grade school let's say your K through 12 it'll show up in red it's going to make it easier for your eyes [Music] and if you're planning on doing full text review and covenants I'll tell you the reasons why you might not later it just gets a little complicated but if you are planning on doing full text review and covenants you can set up exclusion reasons here so if you do get to read the full text of an article and you say okay well this is excluded you'll need to note for which reason so it's the wrong population the wrong outcomes wrong intervention wrong comparator and you can add your own so let's see I just want to make sure those are saved yep everything is saved it's super and last but not least we'll look at study tags I haven't really used this too much before you might just want to categorize all of your records I'm just going to add a tag for interprofessional all right so we worked through every piece of settings here and now I'm going to go back with the back button up at the top left corner and now we're on the dashboard for this review and we know it because it says up here on the upper left education webinar test review summary import title abstract screening text full text screening and data extraction let me get caught up on my notes let me check the chat before we move on no questions in chat all right let me go back to the slides so now we're ready to import references into the review in order to have a working review we need to have references I guess they call them records I'm always getting tongue-tied on everything so we'll call them records so what kinds of files can be imported as I mentioned earlier you can use EndNote XML so that's a file that would come from the EndNote reference manager PubMed XML which is a format that can be exported from the database PubMed or if you're using an EBSCO database like Eric or cinahl or abi/inform or some others the names are escaping me right now you can actually export straight to our is text format and if you're working with a librarian we'll be able to export them to arias text format for you and remove duplicates so keep that in mind you can import up to 50 megabytes and up to 15,000 references so I don't think anyone is going to be limited by that I think it's a pretty generous amount note that larger files take longer to upload and covetous will automatically deduplicate and keep track of duplicates for you so the the do the deduplication process is always sort of a human error kind of thing so I always look for duplicates when I give files to my users but they're inevitably going to be duplicates remaining and covenants will try to identify those for you and eliminate them for you so then when you login to your dashboard for the review you'll be able to move through your screening process so here we have title and abstract screening I can continue this is just a screenshot and actually let's so let's do this now in person because I did set aside some references for today I'm going to go back to the webinar webinar test review that I set up and I'm going to say import it's up here in the upper right hand corner and it's the FIR step I'm going to import it into and you need to know what your next step is so our next step for any review is going to be screening so we'll say screening and I will browse to the file that I set aside earlier I'm not going to go into database searching today this delivery to is an ris format file that I got from the database Eric on obscure host so I am importing it into screening and I chose the file and now I'm going to import so we're going to see what we get here I have it's only 14 so like I said this is not realistic you're probably going to have hundreds depending on your topic it didn't find any duplicates that's good and so you'll see now that the first step or the second step really title and abstract screening has been populated with 14 studies to screen 0 are done 0 have 1 both 0 have conflicts and 14 are left remaining if I needed to check my team settings I can do that here or I can continue through the screening process so when we are screening we're going to get whatever data the database could give us so it's usually the citation information maybe some metadata and the abstract we're not looking at full text here it is a quick question yes oh so you did an EBSCO search using some parameters yes and you came up with 14 articles yes then within EBSCOhost you um you must have set up that file somehow yeah let me I'll show you yeah I don't understand any of that so okay so I'm on the Welch medical library homepage please note you also Eisenhower which i think is new and improved and looks really great and I'm just going to go to Eric I tried to make a pretty small perimeter search let's see what I can come up with today so here is Eric hosted on the EBSCOhost platform and I'm going to type interprofessional education [Music] and it populates actually it's smart so there are some suggestions here I usually like to go with their suggestions and let's try to make it narrow narrower so anytime we're talking about interprofessional education in Neurology or neurological or neuroscience so we're just casting a net for some articles 218 the way I did it this morning was just to narrow it down by date so let's say the last whole years so your filters and most databases are always going to be over there on the left oops and I'm just gonna limit it then to academic journals so I don't want dissertations I don't want reports so I have 25 articles that seems pretty reasonable look through them here but we really wanted to pull them into covenants we'll say share results 1 through 25 and they appear in my folder on the left-hand sorry the right hand side so I can continue to search your browse or I can go right to folder view so if I'm in folder view I can see I have 1 through 25 of 25 records and I can now export them and I think it's the first option the first option is direct export in ris format and it'll note some citation managers here so we see a note reference manager Zotero there's some other formats here refworks if you're using that one but we're going to stick with our s format which is a universal file type for citations and then I will click Save over here on the Left I'll save the file and then if I look in my downloads you here's the one I did earlier today here's the one I just did it's called delivery - in delivery three but that's what EPS cohost cause they're export files and you'll see the type is Ras formatted file if you're working with Eric or a database hosted by EBSCOhost you'll be able to export that way that helped yes thank you here we are so I'm not going to import those but I could if I wanted to so we'll just stick with these fourteen that I had found earlier today and let's move through the screening process it assigns a number and it shows you the first author as the title the other authors some citation information and then the abstract you can add a note here let's give history it just shows me that I imported it today and I'm the only reviewer on this project so I can just say yes so it fits my criteria it works for me I want to include it in my final review so I can say yes and so I'm just going to do that I want to see if we have any highlights showing up I thought my search was about interprofessional education but I'm not seeing any highlights well if they were there we'd see them so I guess I didn't pick any highlighted terms that are going to show up for us today but you would see them there if you had done it right and since I only have 14 I'm just going to continue to move through this process ideally of course you're reading these you're looking at the title you're looking at the abstract and you're deciding if it meets your inclusion and exclusion criteria so I'm going to vote YES on Nate most of them I'll vote no on a couple don't don't do what I'm doing now if there were another reviewers if I was requiring that Sharon also vote they would need her vote too this is a really nice interface I should note because prior to this I think a lot of people were using Excel and Excel is nice but it's not going to do this for you so I have just screen 14 references in my title and abstract screen and so now you'll see the title and abstract screening 3 I I guess 3 I voted no on and there are 0 left to screen and now there are 11 for full-text screening so let's go back to my slides screening yes no or maybe that's what it looks like if there are conflicts and you have a third reviewer to break the conflicts this is what they'll see so you'll go into your review you'll go into this section that says conflicts and they will be able to make a final decision of whether this article should be included or excluded and then you move to the full-text screening and this is where I'm going to say that this is not confidences strength this is probably where you're going to want to export your references or records and do it in a way that's familiar to you and the reason is because confidence does not automatically import PDFs so you'd have to import them yourself there is a way to bulk import PDFs you can find it on the knowledge base the confidence knowledgebase if you're dealing with a lot of Records like in this screenshot 480 258 studies the screen there is a way to bulk import those PDFs but I think for a lot of you you'll be pulling your references out at this point to continue in a way that you're familiar with let's look at the chat we have a couple questions when you click that yes you include it in the final review isn't this just an I'm sorry I misspoke when you click yes you include it in the full text screening so once you click yes in title abstract screening it moves over to full text screening the second piece of the process thank you for catching that confidence for most people has been a title and abstract screening tool yes and I think Juliana who commented earlier I think that's what she used it for I'm not sure that a lot of people are doing data extraction and full text screening in covent let's talk about it if you did want to do it this is the next step so after you've done full text screening and you've decided to include four reasons or exclude four reasons you'll be doing quality assessment and data extraction and covetous has templates for these let me look at my notes so covenants uses the Cochran risk of bias for quality assessment and the data extraction template is something that you can set up yourself but it's very important that you pilot it first before you get started actually doing the data extraction or you might have to start from scratch and this is not a point at which I'm involved very often so I really just want to show you what it looks like but if you wanted to find out more you'll need to go to confidences knowledgebase which I have provided here so they have a really nice I think about eight minute long video on doing quality assessment once you've done the full text review and done inclusion and exclusion it's going to ask you for things like selection bias attrition bias and determine if the article that you've included has a good methodology and then data extraction so this is where if you're doing a meta analysis or any sort of data synthesis you might be looking at a I think it's called a forest plot or a table that's going to help you make sense of your results so this is also a video embedded in these slides and the slides were shared with you in the calendar invite so feel free to click on these watch them I think they're both about 8 minutes long so more resources for data extraction confidences knowledge base and note that the quality assessment and data extraction form templates and Co buttons are clinically focused and may not be the best for your research it just depends on what you're doing and so you may consider exporting your references at this point and employing a quality assessment that's more in line with your methodology so let's look at how that will work this is a screenshot I'm gonna go back to the I'm gonna check chat I see there's a question yes Juliana said she did not use the extraction mainly because they were very rushed and it wasn't a full systematic review so you could be more idiosyncratic thank you for sharing that know let's say I've done my tile and abstracts training and instead of 14 references let's say it was 400 references right so you can see that it would really make your process a lot easier but now we want to pull these references out so that we can continue to work with them for our literature review so up in the top right hand corner we're going to say export and it's going to ask you at which stage would you like to export the screen that the screening ones which were the 14 references or the full text review which after I voted yes and no there's only 13 references so we're going to say we want the full text review and then you can export it to CSV which would be an excel file you can export it to EndNote if you're using that Mendeley refworks or Zotero which are all reference management programs so we talked about those earlier so these are probably good options for you if that's what you're using I'm going to put it let's say CSV you know export and it might take a moment and it worked so now I can download it we're going to open it with Excel and here are my 13 references like I said you'll probably have hundreds it has the title the authors the abstract the year the journal volume issue pages doesn't have a DOI seems like it should and then if you do need to go back into covent it's going to give you the reference number for covetous so in covenants this was number 10 and they're calling it Terry 2018 so it's just an identifier so you can find it in Covenants and then if you are doing a true systematic review or any type of review where you want to include a Prisma flow diagram it's going to help you with that as well and whenever I show people this they say wow that's amazing let's go back up I'm gonna go back with the back button so I get back to the dashboard page so here's my review we know we're in the education webinar test review we did settings we did screening we did export we can look at Prisma so Prisma is Prisma stands for preferred reporting items for systematic reviews and meta-analyses and you've probably seen it before so it has 14 studies imported for screening you'd fill in here actually I think you exported you're just going to export this and then you'll be able to use it in your own Prisma diagram but it's going to tell you how the process worked and if you google print diagram first result and here's a word template open with so depending on which journal you're submitting to or what methodology you've chosen for your review you may be filling this out let's just take a look at it I think it's pretty interesting if my word program can open we'll come back to it in a moment let's look at the PDF so here here's the Prisma flow diagram just a little bit neater so identification of records screening of records eligibility so full-text inclusion exclusion what's included and then your synthesis if you're making a synthesis this is a video on exporting citations so if you need to refresh your memory about how that works this is a short video on how to do it from confidence and then at the Welch medical library recently we put together a short tutorial on signing up for covenants I'll just start it real quick you so this is a six-minute video that will walk you through signing up for covenants and refreshing your memory about how it works and I want to leave time for questions I'm gonna leave you with some of these slides actually it's in your it's in your slides these are all links to the covenants knowledge base the Welsh library is systematic and expert reviews guide and how to find your librarian whether it's me Sharon Mike or someone else you can find them on the web library homepage or the Sheridan library's home page and with that I think let's see there's references here that you might find interesting and I'd like to open it up for questions if there are any left there's one in the chat okay someone had to leave that's fine you're allowed any questions about confidence or how you might use it for your reviews hey Carrie this is Julianna Hey so I'm in addition to wanting to use this for our own purposes this faculty we're considering supporting students to use it for assignments I just wondered if you know if other schools have have done that sort of used it in this truncated way that you're describing here less about the excel sheet piece more about before that and in addition yes one other thing I know you mentioned it but can you remind us again where in your slides or where we should go to get more information about creating our own a more idiosyncratic template for the cut and paste piece if we do want to you use that but aren't doing a full-blown systematic reviewer or otherwise yeah first question supporting students as far as I know there is not a seat limit or user limit on covent it is a fairly new tool so what other schools are doing it's hard to say as a as a clinical information is working mostly with doctors fellows and I can say that even they are using it for the screening part and then most of them are pulling it out and doing the full text reviews separately because they're using such a variety of tools that the Cova doesn't quite offer yet so they're doing it their own way by exporting it I think it's probably a great tool for students the other option being of course like an Excel spreadsheet or an annotated bibliography or something like that here's the Cova dense knowledge base and I'm just going to look for data extraction so setting up extraction tables for outcomes if you do decide to stay in confidence after the full text review and you'd like to play with the data extraction you can look at more information here depending on what kind of data you're working with but as I said this is not a step that I've been involved in very often it'll probably make more sense to you guys excuse me a second any other questions or concerns about confidence or anything I didn't answer we're at 154 so I'll hang around for a minute or two I thank you for coming this will be recorded here and I hope you all have a great rest of your day thank you Carrie thanks thanks so much sure Carrie thank you so much for joining this far fellows appreciate it