Feedback and Coaching in CBM II

thank you all for coming this afternoon it&#39;s a real honor to be invited here and to to see so many people in the room I&#39;m really thrilled to be talking about feedback and coaching in particular in CBM II as I think that that you our group are sort of embedded in this work in the middle of it and that there&#39;s lots of learning that each of you have to share so that you&#39;ll hopefully what I have to say some of it may be novel I can appreciate that maybe you&#39;re you&#39;re really quite far along in the process that I&#39;m sure that I&#39;ll learn lots from you as well so thank you very much so my my objectives very briefly given half an hour is just to review the role of feedback and coaching in CBM II review research and coaching especially a little bit on the culture and the components what makes up coaching this thing that we&#39;re calling coaching I&#39;m going to describe the r2c to coaching model which is my field of research for the last six or seven years and just very briefly talk about some research findings around that and then talk about challenges moving ahead which then I&#39;ll be really lucky because the panel that&#39;s coming up are going to be addressing a number of those challenges so it will be a really nice segue and they can provide some of the answers so the goals of CBM II I&#39;m sure that you&#39;ve heard these before I understand that Erik combo was your visitor this time last year and I&#39;ve probably taken this from one of his publications he if you know in June of 2017 there was a there were updated series of articles on CBM II and Erik was one of the lead authors again so the goal of CBM II as he points out and others point out it&#39;s really towards the population to improve healthcare outcomes reduce errors increase safety so that&#39;s the goal that we have to keep in mind down here at a lower level feeling like we&#39;re worth working in the in the ranks in the streets and not quite seeing the forest for the trees that remembering that&#39;s what we&#39;re doing it for the other one is for learner development so that they can provide that safe care and so that we can help them to progress and grow as efficiently as effectively as possible and although I think within the CBM --red Uruk we often hear the word assess perhaps more than we hear the word develop and grow but that&#39;s really the goal of CBM II is to help our learners develop and grow as effectively as possible to give that safe care so I want you to think well first there I&#39;m going to comment on this little guy why I chose this little guy climbing up a ladder to be on to be on some of my slides because for me it really represents CBM II in just a really simplistic way it&#39;s the learner climbing up you know one milestone at a time and what isn&#39;t there is the coaching and so for and the supports that faculty provide so just take a minute that your table or with the person next to you and talk what are the differences between feedback and coaching the these are suggestions made by Keane and stone which are actually authors of a book on feedback that sue you have there what is it called it&#39;s the thanks for the feedback it&#39;s I don&#39;t whether some of you know this book but it&#39;s it&#39;s excellent and and now mind you they wrote this in 2014 when I think people were just beginning to talk about coaching from an educational sense and so they&#39;ll a talk together and I guess it was about assessment and feedback that you know make tells you where you stand you know where you measure out how you measure up what&#39;s expected and it can feel a little bit intimidating and emotional and then I think the points that you folks brought out at this end of the room about coaching its longitudinal and the points that really you know that resonate with me this notion that enables you to learn and improve and helps you play at a higher level like to me that really sort of says what we says what we mean when we&#39;re talking about competency education we really want our health professionals to play at a higher level we want them to move up that ladder and to be able to perform more efficiently and effectively and to learn as effectively as possible now it&#39;s learners centered it would would be longitudinal however the point that you make that coaching needs specific descriptive timely feedback data and I&#39;m going to use these words feedback and data a little bit a little bit differently and and you know people have different views of what feedback is but we really need if you think about data that you need data of some kind and when you observe a learner you&#39;re collecting data about that learners performance or if you&#39;re a program director reviewing a learner&#39;s portfolio you&#39;re reviewing the evidence that&#39;s been written about that learner that&#39;s the data so in the next slide I&#39;m going to give you a written feedback exercise so this is an end of the rotation Eider I put which thankfully is disappearing I think isn&#39;t that hopefully there isn&#39;t gosh if if someone here is really went to eiders I apologize we can talk afterwards about why perhaps at least in their current format why maybe it&#39;s helpful that they might be disappearing so I want you to complete pre critique this feedback this comes from a busy program director who assures me that this is often what she reads on eiders and so try and look at this as this residence coach yeah is it specific enough to help the resident improve is there consistency between the scores and the narrative and does the narrative provide enough description ie data to guide the coaching okay so I&#39;ll slide it up and give you a few minutes to take a look at it and then see what you think so just so I&#39;ve given listed each of the can meds roles just told you how many items free then I gave you the average rating and then the overall rating at the bottom and then the comments that were written about dr. R so if anyone has any observations around is it specific enough is there a consistency between scores and narrative no could you coach this learner based on this feedback and you give me some specific examples but what&#39;s the conflicting evidence for timeliness in there in clinic they got a five on professionalism right but and this is you know this is I mean this HAP this happens the other question is is this narrow is the narrative sort of written in some kind of code is it is it skirting the issues and here I&#39;m boring from shift for his wonderful work around around you know what we want we write on our formal evaluations like this code around what needing a written plan for not in enhancing knowledge like you know what what does that mean for example so you know I&#39;m using this just to highlight the fact that in order to coach we need to have good feedback good feedback means good data so requirement number one for coaching specific objective relevant timely narrative description of the observed behavior and that&#39;s the the actual data the performance data in the same way that a basketball coach or a skating coach would have observational data to base their coaching on what you&#39;ve observed if the learner of you&#39;re giving feedback in the moment or what you have in that written narrative if you&#39;re doing it longer to Denis and over time that&#39;s the the data that you have sorry this as we all know numbers don&#39;t tell you how to change that you&#39;ll being a three year being a four doesn&#39;t tell you how to improve and so you know words have power and so that&#39;s part of our part of our message around being able to coach is that we need those narrative descriptions or we need to be able to observe a behavior and describe back to the learner what we observe and confirm with them is that what you thought you were doing or am I on track when I say I saw you do X Y and Zed so remembering that words have powered then as far as the role of you know words and feedback and coaching and competency-based medical education this model depicts this triangle pyramid yet another pyramid and education depicts the progression the developmental levels from novice to advanced beginner to competent to proficient to expert and it&#39;s really the the words the feedback and the coaching that we give provide to our learners along the way and I mean that in a rigorous and structured and way that will help them move more efficiently from from one level to the next and so for me CBM II is more thinking about the the words and the feedback and the coaching that we can share with our learners to help them get better rather than you know the assessments that if the purpose is really for them to be able to develop how can we help them measure develop efficiently and effectively mind you we need to assess that as well however perhaps mean that it&#39;s time for the pendulum to swing back a little bit to the learning side and focus more on how we can help them learn so now I&#39;m going to move on to the role of culture in influencing feedback and coaching and this is another area where quite recently some of our colleagues have been devoting their study first of all does anyone know where this where this quote comes from like Earth&#39;s something like that like culture eats evidence for lunch you know that if you were just you know we are in Drent entrenched and doing something the same way and even if we&#39;re presented with the evidence that we shouldn&#39;t do it any longer we just do it because it&#39;s easier and so there&#39;s lots there&#39;s been evidence 20 or 30 years of evidence that we&#39;re not doing feedback well and you know there&#39;s barely any change so that&#39;s not that&#39;s not the answer so what are we learning we&#39;ve learned lots from Chris Watling and I&#39;m sure I would expect that he&#39;s you know perhaps been here and chatted with you at times or and you&#39;ve had lots of chance to talk with him and his PhD was on feedback and especially studying the culture and you know this really rings true that medical and medicine medical and medical education culture emphasizes autonomy and independent practice and that prevents direct observation of performance by faculty and acts as a barrier to open feedback would you agree am i saying am i saying anything that you know we&#39;re all high achievers we all want to be excellent we all want to be autonomous and so therefore it makes it makes it hard it makes it hard for us to think about being observed and receiving feedback and then I&#39;m not sure if you&#39;re familiar with Suva romani who is actually a PhD candidate at the University of Maastricht and it&#39;s just you may know that that ph.d program has compiled of a number of papers and she has just published a number of her papers as you can see - towards the end of 2017 and and one in 2018 all in medical teacher the last one in 2018 is 12 tips for enhancing the feedback culture so really timely for us and then the academic medicine one was a publication what&#39;s that what&#39;s pap-pap publication published ahead of print thank you very much and that one is on her qualitative research around a culture of politeness and excellence and so I was reading these just over the last couple of weeks and again it helps she builds on Chris&#39;s work which was quite wonderful and really talks about you know not only the excellence that you know believing that we&#39;re all excellent or working as at a high level but also this politeness this notion that we don&#39;t want to say something that&#39;s going to you know upset the applecart or that&#39;s going to ruffle our relationships and so when we look at this we can understand why why it is so challenging challenging for us to really develop this feedback a culture where feedback is accepted and where it&#39;s sought and where it&#39;s given in appropriate ways so those now I there weren&#39;t printouts of my slides but I&#39;ll certainly leave them and they can be distributed and I can certainly share these references and Schiffer would have access to them as well as she is as you know my wonderful colleague at Dalhousie Karen Mann was one of subas superb eyes errs and Karen passed away a year and a few months ago and so Schiffer I understand that you&#39;ve stepped in and and are working with her so you have a wonderful connection to to this work so this is another quote from Chris coaching requires a shift in philosophy so why don&#39;t you just read this and just tell me what you think I see a few nods around the table and I heard some of you and you&#39;re in your when you were describing coaching using word like that words like that and it&#39;s it really sets you can really see the dichotomy can&#39;t you it isn&#39;t about the finite exercise of teaching something to someone but about the infinite exercise of continuous improvement and so thinking that if we as faculty can start to model that continuous improvement if we can start to change the culture in our own departments or divisions by asking for feedback from others and saying in that being open when we don&#39;t know something or when we made a mistake and asking for help that that again can maybe begin to change the culture towards where our learners will see that it&#39;s okay not to it&#39;s okay to ask for help and it&#39;s okay and it&#39;s a path of continuous improvement I put the website that Chris had written that in one of the the the school his schools moves newsletters but I just thought it really described it nicely so coaching is about promoting growth it&#39;s about getting better its longitudinal based on goals and so it&#39;s really about a growth mindset how many of you have are familiar with Carol Dweck&#39;s work on growth mindset you I&#39;m just going to show an excerpt of a video as it describes the difference between a growth mindset and a fixed mindset team and I think thinking of a fixed mindset that if you think you&#39;re already at the top of your game you&#39;re already excellent then the it&#39;s hard to think of room for growth and so it&#39;s it&#39;s helpful for trying to think about this growth mindset and how we might help our learners one of the studies that we did at Dalhousie was around residents and trying to understand if they would ask for feedback and under what circumstances they would ask for feedback and so you think you asking for feedback is a growth model right but you&#39;re asking for asking to learn how to do something better and and what do you think their response was when we asked them yes now this was five or six years ago six or seven years ago so maybe maybe the culture is changing but their response was you don&#39;t want to put your hand up you don&#39;t you don&#39;t feel be there like a red flag saying you know I don&#39;t know something so therefore I just stand at the back and hope I can learn whatever it is that I don&#39;t know by just listening to others so to me it&#39;s a pretty compelling example and this was shared by the group of residents that we were studying it&#39;s a pretty compelling example that it wasn&#39;t there the culture did not promote a growth mindset and so so I&#39;m just I&#39;m just showing it to you as as something to think about and to to think about how can each of us in our own environments begin to support that that growth environment maybe not all the time but but perhaps a little bit more at least in my school a little bit more than I do we do already at Dow so let&#39;s move back to coaching this is a definition of coaching and education so when we started talking about coaching people immediately went to the sports coats coaches definitions in education I think we think about it differently is an involved one that&#39;s a one-to-one conversation focused on the enhancement of learning and development through increasing self-awareness and a sense of personal responsibility so this really brings out the idea that you&#39;re not just giving you&#39;re not just coaching in that moment for them to do that one task better but you&#39;re also coaching because you&#39;re trying to develop those skills of self-directed learning and self critique so the coach facilitates the self-directed learning of the coach Eve through questioning and usually open questioning active listening appropriate challenge and in a supportive and encouraging climate now I can imagine that you would look at that and say oh my goodness we can never do that and you&#39;re a really busy clinical setting but I only have two or three minutes with a learner however there are your aspects of it that you can that you can sort of you know work on even though even the fact of asking open questions shows that you&#39;re interested in the learner and I&#39;ll talk about that a little bit more when I talk about my own research so that&#39;s one one definition to think about so let&#39;s talk a bit now but what&#39;s involved in this some activity that we&#39;re calling coaching and this is work from been levels systematic review published just before Christmas and also from other other work as well so the coach provides individual individualized real-time feedback ie performance data so we we saw an eider that would have been you know at the end of a rotation but if it&#39;s in the moment it would be that individual feedback what I observed when you did when I saw you doing something the coach and the coach says goal for the coach or the excuse me for the coachee for the learner so it&#39;s co-development if it&#39;s the moving away from the more directive model where the faculty says well I think you need to do this this and this it&#39;s trying to build that that self direction by coaching them in what what would be appropriate goals facilitate the coach facilitates the development of new behave years insights and approaches to work so again it&#39;s working to facilitate the learners growth its learner centric trying to foster reflection so what did you think went up went wrong here can you tell me what you were thinking when you did such-and-such so just trying in a very few minutes to get them to think through what they were doing and to self critique and and to think about what they should do next and this one is interesting about the coach has an analytic analytical understanding of the game what the games all about like the game of medical education or to become a surgeon or whatever an ability to motivate the learner towards excellence so and you know in other words they sort of know what good looks like even even if perhaps the you know it&#39;s there if it may not be in their area of specialty that they know what it means to be a good clinician you need to have a set of skills and so that they can they can coach from that holistic perspective I&#39;m going to say just a few words about my the body of research that I&#39;ve been part of for a number of years and where&#39;s Ivan silver he&#39;s over there he&#39;s been part of it for a while as well and this is just a local study that we did it at Dalhousie and then a larger international study at the bottom used about the r2 c2 model and you have a brochure there&#39;s a couple of row shares on the table the thing is just due to the limits of time I&#39;m not going to go through the model in detail because I want to share some of the research with you I will just or I&#39;ll go to the okay first of all I&#39;ll tell you so the theory this you know we a lot of theory and we gave this a lot of thought and had some prior research on self-assessment and receiving an accepting feedback before we came up with the model and there are three bodies of sort of theory that inform this work this notion of informed self assessment that you know we need external feedback to inform and let us know how we&#39;re doing and so in order to get the learner to reflect upon what external feedback they need you need to ask open-ended questions you need to guide reflections and guide them to self critique it uses learner centered approaches so you&#39;re really focusing on them based on humanism you want to post positive their motivation and theirs and support their self direction and then the third area are the cognitive domains influencing behavior change we know that there are many factors that influence behavior change and we started using a learning change plan to address these just to help that scaffolding help that learner move up the ladder so it&#39;s a four stage model and you&#39;ll see in the brochure and and in the brochure we give some of the purpose of each of the phases they&#39;re iterative and go back and forth between them and some phrases that are used the phrases are taken from our research there are ones that facilitators have actually used and residents have said that they&#39;re helpful so the first phase is the rapport building or the relationship building and what we&#39;ve heard from residents and supervisors that even if they know each other fairly well you can still learn more about them asking questions like well what kind of feedback have you had before you know what what is really helpful for you you know what you know what really what did you really gain from this past couple of weeks what you know what eh what are your goals so again thinking of whether it&#39;s in the moment or when someone comes to your clinic just for a day or whether it&#39;s a two week or the end of rotation session that you&#39;re doing then exploring their reactions so that and this really comes from them they work on inform self-assessment that if you don&#39;t agree if the feedback you&#39;re getting doesn&#39;t confirm how you see yourself it&#39;s often a surprise and it&#39;s I can often be an uncomfortable surprise it can make a sad it can make us angry if we get negative feedback and so it&#39;s being upfront and asking are there any were there any surprises and what I told you about my observations or are there any surprises on this end of rotation form form that we&#39;re reviewing and then just to talk about talk about those and bring it out on the table and have a conversation and normalizing it you know we&#39;re all disappointed when we have feedback that doesn&#39;t quite jibe with how we think we&#39;re doing and having that conversation then moving into the first C which then once you get the reactions and the emotional part over and moving into the content the knowledge bits so so did what I just described to you and when I observed you you know taking out those sutures or whatever it was that it makes sense to you can you understand why you know why I made the comments that I did and are there questions that you have and let&#39;s make it something more complex maybe it&#39;s time management around managing a roster of patients this is what I observed what do you think and then where where are the opportunities for improvement based on my observations and how you think you&#39;re doing where are the opportunities for improvement which really gets to the coaching the coaching for change however what we&#39;re finding is that you sort of almost have to go through you know those three levels beforehand to get to the coach and to come to that agreement as to what to work on so that&#39;s what the the brochure is about this is the the coaching then is real what we in the last research that we did we used a learning change plan and this is what it looked like and you can see it&#39;s quite involved it&#39;s got the describe specific observable changes that you want intend to make you know the specific actions or the timeline timeline to when you think you&#39;ll see results identify the resources the challenges and how do you know if you&#39;ve been successful when we handed this out in the study our study we thought that it would be too complex but in fact they said that it was helpful now mind you it wouldn&#39;t be helpful it would be too complex for in the moment feedback at the table at the bedside or in the clinic but thinking about it when you&#39;re doing your monthly or try monthly or every six months progress meeting with residents this might be a tool that would be helpful so the goal really of the are to see to model is to trying through this facilitation to try and facilitate you know the notion that this is X feedback is external you know it&#39;s external stuff coming in it&#39;s not in control it&#39;s a threat and it could even be about something that&#39;s bad something I&#39;m not doing well so trying through this listening hoping questions you&#39;re accepting motivating coaching trying to help the residents realize that this is their data as use this is your data and it&#39;s your opportunity and you can be in control of using it to change I know it&#39;s pretty pollyannaish but it&#39;s um it&#39;s a goal that&#39;s our reaching goal and actually I&#39;ve heard from a unit that&#39;s been using it for a couple of years that it is actually beginning to change the culture slowly so I&#39;m just going to go if I got two more minutes alright so the overarching from our research which was an international study on five sites what was valued by both residents and supervisors and using the model they valid they valued the facilitation of the resident engagement and reflection they valued the focus on getting better they valued the collaborative supervisor resident relationship they created that learning change plan together they found it useful with residents that varied levels especially those that were excelling and then their recommendation was to orient and prepare residents that&#39;s new in the same manner is actually and I think later on this afternoon you&#39;re going to hear more about development of repairing residents and faculty together or a change in the culture I&#39;ll just very quickly go through some of the quotes and what this one was around engagement and reflection because it was so different I guess you&#39;re less likely to be defensive because a lot of it was just identifying my own weaknesses he helped me to come to that conclusion and he prompted me to self critique one about the outcomes orientation I&#39;ll just let you read that one so valued the focus on getting better really the specific strategies to take to get better the collaboration and co-creation of a plan we came up with the goals together what I liked about it was what can the supervisor said what can we do can you achieve your goals and the resident that makes me motivated to know I&#39;ve had that support finally with residents who were excelling supervisors reported that it was benefit because they said it provided more substance and structure for their feedback conversations so it gets you to the next level level it gives you more to say normally I would have just said hey you&#39;re doing well Arion so they said that using that format even if they didn&#39;t write it down and fill it out it gave them the questions to ask ask them what do you aspire to even if you&#39;re you know excelling in everything already what would you aspire to and how might you set goals and when would you like to achieve them and who do you need to help you so asking questions that weren&#39;t available before and I think I&#39;m just going to finish up summary and next steps yes so this is summary overall that was just a very brief overview of the research that we did Joan oh I&#39;ve been thinking about CBM II and again this is from aritaum Bo&#39;s were you know point no it&#39;s an evolution it&#39;s dynamic it&#39;s not the night and I&#39;m it makes me think it&#39;s a disruptive technology it&#39;s shaking this up and it&#39;s made us realize that our education health systems and cultures you know don&#39;t probably value sound education principles as much as they should you know the kinds of things that we&#39;ve just been talking about our really based on sound education principles like things that should be the foundation of our programs assessment for learning observation to collect data longitudinal relationships safe respectful environment coaching our learners become you know the best that they can be and I think as far as changing the culture I think yeah we first you know we first changed the culture by changing ourselves and we know that one of the best ways to change a culture is I role modeling and and so in this case being open to feedback and role modeling various other ways of trying to be more open and open to learner&#39;s giving us feedback so thank you very much and if you can have some time for questions and then I&#39;ll look forward to [Applause]

thank you all for coming this afternoon it&amp;#39;s a real honor to be invited here and to to see so many people in the room I&amp;#39;m really thrilled to be talking about feedback and coaching in particular in CBM II as I think that that you our group are sort of embedded in this work in the middle of it and that there&amp;#39;s lots of learning that each of you have to share so that you&amp;#39;ll hopefully what I have to say some of it may be novel I can appreciate that maybe you&amp;#39;re you&amp;#39;re really quite far along in the process that I&amp;#39;m sure that I&amp;#39;ll learn lots from you as well so thank you very much so my my objectives very briefly given half an hour is just to review the role of feedback and coaching in CBM II review research and coaching especially a little bit on the culture and the components what makes up coaching this thing that we&amp;#39;re calling coaching I&amp;#39;m going to describe the r2c to coaching model which is my field of research for the last six or seven years and just very briefly talk about some research findings around that and then talk about challenges moving ahead which then I&amp;#39;ll be really lucky because the panel that&amp;#39;s coming up are going to be addressing a number of those challenges so it will be a really nice segue and they can provide some of the answers so the goals of CBM II I&amp;#39;m sure that you&amp;#39;ve heard these before I understand that Erik combo was your visitor this time last year and I&amp;#39;ve probably taken this from one of his publications he if you know in June of 2017 there was a there were updated series of articles on CBM II and Erik was one of the lead authors again so the goal of CBM II as he points out and others point out it&amp;#39;s really towards the population to improve healthcare outcomes reduce errors increase safety so that&amp;#39;s the goal that we have to keep in mind down here at a lower level feeling like we&amp;#39;re worth working in the in the ranks in the streets and not quite seeing the forest for the trees that remembering that&amp;#39;s what we&amp;#39;re doing it for the other one is for learner development so that they can provide that safe care and so that we can help them to progress and grow as efficiently as effectively as possible and although I think within the CBM --red Uruk we often hear the word assess perhaps more than we hear the word develop and grow but that&amp;#39;s really the goal of CBM II is to help our learners develop and grow as effectively as possible to give that safe care so I want you to think well first there I&amp;#39;m going to comment on this little guy why I chose this little guy climbing up a ladder to be on to be on some of my slides because for me it really represents CBM II in just a really simplistic way it&amp;#39;s the learner climbing up you know one milestone at a time and what isn&amp;#39;t there is the coaching and so for and the supports that faculty provide so just take a minute that your table or with the person next to you and talk what are the differences between feedback and coaching the these are suggestions made by Keane and stone which are actually authors of a book on feedback that sue you have there what is it called it&amp;#39;s the thanks for the feedback it&amp;#39;s I don&amp;#39;t whether some of you know this book but it&amp;#39;s it&amp;#39;s excellent and and now mind you they wrote this in 2014 when I think people were just beginning to talk about coaching from an educational sense and so they&amp;#39;ll a talk together and I guess it was about assessment and feedback that you know make tells you where you stand you know where you measure out how you measure up what&amp;#39;s expected and it can feel a little bit intimidating and emotional and then I think the points that you folks brought out at this end of the room about coaching its longitudinal and the points that really you know that resonate with me this notion that enables you to learn and improve and helps you play at a higher level like to me that really sort of says what we says what we mean when we&amp;#39;re talking about competency education we really want our health professionals to play at a higher level we want them to move up that ladder and to be able to perform more efficiently and effectively and to learn as effectively as possible now it&amp;#39;s learners centered it would would be longitudinal however the point that you make that coaching needs specific descriptive timely feedback data and I&amp;#39;m going to use these words feedback and data a little bit a little bit differently and and you know people have different views of what feedback is but we really need if you think about data that you need data of some kind and when you observe a learner you&amp;#39;re collecting data about that learners performance or if you&amp;#39;re a program director reviewing a learner&amp;#39;s portfolio you&amp;#39;re reviewing the evidence that&amp;#39;s been written about that learner that&amp;#39;s the data so in the next slide I&amp;#39;m going to give you a written feedback exercise so this is an end of the rotation Eider I put which thankfully is disappearing I think isn&amp;#39;t that hopefully there isn&amp;#39;t gosh if if someone here is really went to eiders I apologize we can talk afterwards about why perhaps at least in their current format why maybe it&amp;#39;s helpful that they might be disappearing so I want you to complete pre critique this feedback this comes from a busy program director who assures me that this is often what she reads on eiders and so try and look at this as this residence coach yeah is it specific enough to help the resident improve is there consistency between the scores and the narrative and does the narrative provide enough description ie data to guide the coaching okay so I&amp;#39;ll slide it up and give you a few minutes to take a look at it and then see what you think so just so I&amp;#39;ve given listed each of the can meds roles just told you how many items free then I gave you the average rating and then the overall rating at the bottom and then the comments that were written about dr. R so if anyone has any observations around is it specific enough is there a consistency between scores and narrative no could you coach this learner based on this feedback and you give me some specific examples but what&amp;#39;s the conflicting evidence for timeliness in there in clinic they got a five on professionalism right but and this is you know this is I mean this HAP this happens the other question is is this narrow is the narrative sort of written in some kind of code is it is it skirting the issues and here I&amp;#39;m boring from shift for his wonderful work around around you know what we want we write on our formal evaluations like this code around what needing a written plan for not in enhancing knowledge like you know what what does that mean for example so you know I&amp;#39;m using this just to highlight the fact that in order to coach we need to have good feedback good feedback means good data so requirement number one for coaching specific objective relevant timely narrative description of the observed behavior and that&amp;#39;s the the actual data the performance data in the same way that a basketball coach or a skating coach would have observational data to base their coaching on what you&amp;#39;ve observed if the learner of you&amp;#39;re giving feedback in the moment or what you have in that written narrative if you&amp;#39;re doing it longer to Denis and over time that&amp;#39;s the the data that you have sorry this as we all know numbers don&amp;#39;t tell you how to change that you&amp;#39;ll being a three year being a four doesn&amp;#39;t tell you how to improve and so you know words have power and so that&amp;#39;s part of our part of our message around being able to coach is that we need those narrative descriptions or we need to be able to observe a behavior and describe back to the learner what we observe and confirm with them is that what you thought you were doing or am I on track when I say I saw you do X Y and Zed so remembering that words have powered then as far as the role of you know words and feedback and coaching and competency-based medical education this model depicts this triangle pyramid yet another pyramid and education depicts the progression the developmental levels from novice to advanced beginner to competent to proficient to expert and it&amp;#39;s really the the words the feedback and the coaching that we give provide to our learners along the way and I mean that in a rigorous and structured and way that will help them move more efficiently from from one level to the next and so for me CBM II is more thinking about the the words and the feedback and the coaching that we can share with our learners to help them get better rather than you know the assessments that if the purpose is really for them to be able to develop how can we help them measure develop efficiently and effectively mind you we need to assess that as well however perhaps mean that it&amp;#39;s time for the pendulum to swing back a little bit to the learning side and focus more on how we can help them learn so now I&amp;#39;m going to move on to the role of culture in influencing feedback and coaching and this is another area where quite recently some of our colleagues have been devoting their study first of all does anyone know where this where this quote comes from like Earth&amp;#39;s something like that like culture eats evidence for lunch you know that if you were just you know we are in Drent entrenched and doing something the same way and even if we&amp;#39;re presented with the evidence that we shouldn&amp;#39;t do it any longer we just do it because it&amp;#39;s easier and so there&amp;#39;s lots there&amp;#39;s been evidence 20 or 30 years of evidence that we&amp;#39;re not doing feedback well and you know there&amp;#39;s barely any change so that&amp;#39;s not that&amp;#39;s not the answer so what are we learning we&amp;#39;ve learned lots from Chris Watling and I&amp;#39;m sure I would expect that he&amp;#39;s you know perhaps been here and chatted with you at times or and you&amp;#39;ve had lots of chance to talk with him and his PhD was on feedback and especially studying the culture and you know this really rings true that medical and medicine medical and medical education culture emphasizes autonomy and independent practice and that prevents direct observation of performance by faculty and acts as a barrier to open feedback would you agree am i saying am i saying anything that you know we&amp;#39;re all high achievers we all want to be excellent we all want to be autonomous and so therefore it makes it makes it hard it makes it hard for us to think about being observed and receiving feedback and then I&amp;#39;m not sure if you&amp;#39;re familiar with Suva romani who is actually a PhD candidate at the University of Maastricht and it&amp;#39;s just you may know that that ph.d program has compiled of a number of papers and she has just published a number of her papers as you can see - towards the end of 2017 and and one in 2018 all in medical teacher the last one in 2018 is 12 tips for enhancing the feedback culture so really timely for us and then the academic medicine one was a publication what&amp;#39;s that what&amp;#39;s pap-pap publication published ahead of print thank you very much and that one is on her qualitative research around a culture of politeness and excellence and so I was reading these just over the last couple of weeks and again it helps she builds on Chris&amp;#39;s work which was quite wonderful and really talks about you know not only the excellence that you know believing that we&amp;#39;re all excellent or working as at a high level but also this politeness this notion that we don&amp;#39;t want to say something that&amp;#39;s going to you know upset the applecart or that&amp;#39;s going to ruffle our relationships and so when we look at this we can understand why why it is so challenging challenging for us to really develop this feedback a culture where feedback is accepted and where it&amp;#39;s sought and where it&amp;#39;s given in appropriate ways so those now I there weren&amp;#39;t printouts of my slides but I&amp;#39;ll certainly leave them and they can be distributed and I can certainly share these references and Schiffer would have access to them as well as she is as you know my wonderful colleague at Dalhousie Karen Mann was one of subas superb eyes errs and Karen passed away a year and a few months ago and so Schiffer I understand that you&amp;#39;ve stepped in and and are working with her so you have a wonderful connection to to this work so this is another quote from Chris coaching requires a shift in philosophy so why don&amp;#39;t you just read this and just tell me what you think I see a few nods around the table and I heard some of you and you&amp;#39;re in your when you were describing coaching using word like that words like that and it&amp;#39;s it really sets you can really see the dichotomy can&amp;#39;t you it isn&amp;#39;t about the finite exercise of teaching something to someone but about the infinite exercise of continuous improvement and so thinking that if we as faculty can start to model that continuous improvement if we can start to change the culture in our own departments or divisions by asking for feedback from others and saying in that being open when we don&amp;#39;t know something or when we made a mistake and asking for help that that again can maybe begin to change the culture towards where our learners will see that it&amp;#39;s okay not to it&amp;#39;s okay to ask for help and it&amp;#39;s okay and it&amp;#39;s a path of continuous improvement I put the website that Chris had written that in one of the the the school his schools moves newsletters but I just thought it really described it nicely so coaching is about promoting growth it&amp;#39;s about getting better its longitudinal based on goals and so it&amp;#39;s really about a growth mindset how many of you have are familiar with Carol Dweck&amp;#39;s work on growth mindset you I&amp;#39;m just going to show an excerpt of a video as it describes the difference between a growth mindset and a fixed mindset team and I think thinking of a fixed mindset that if you think you&amp;#39;re already at the top of your game you&amp;#39;re already excellent then the it&amp;#39;s hard to think of room for growth and so it&amp;#39;s it&amp;#39;s helpful for trying to think about this growth mindset and how we might help our learners one of the studies that we did at Dalhousie was around residents and trying to understand if they would ask for feedback and under what circumstances they would ask for feedback and so you think you asking for feedback is a growth model right but you&amp;#39;re asking for asking to learn how to do something better and and what do you think their response was when we asked them yes now this was five or six years ago six or seven years ago so maybe maybe the culture is changing but their response was you don&amp;#39;t want to put your hand up you don&amp;#39;t you don&amp;#39;t feel be there like a red flag saying you know I don&amp;#39;t know something so therefore I just stand at the back and hope I can learn whatever it is that I don&amp;#39;t know by just listening to others so to me it&amp;#39;s a pretty compelling example and this was shared by the group of residents that we were studying it&amp;#39;s a pretty compelling example that it wasn&amp;#39;t there the culture did not promote a growth mindset and so so I&amp;#39;m just I&amp;#39;m just showing it to you as as something to think about and to to think about how can each of us in our own environments begin to support that that growth environment maybe not all the time but but perhaps a little bit more at least in my school a little bit more than I do we do already at Dow so let&amp;#39;s move back to coaching this is a definition of coaching and education so when we started talking about coaching people immediately went to the sports coats coaches definitions in education I think we think about it differently is an involved one that&amp;#39;s a one-to-one conversation focused on the enhancement of learning and development through increasing self-awareness and a sense of personal responsibility so this really brings out the idea that you&amp;#39;re not just giving you&amp;#39;re not just coaching in that moment for them to do that one task better but you&amp;#39;re also coaching because you&amp;#39;re trying to develop those skills of self-directed learning and self critique so the coach facilitates the self-directed learning of the coach Eve through questioning and usually open questioning active listening appropriate challenge and in a supportive and encouraging climate now I can imagine that you would look at that and say oh my goodness we can never do that and you&amp;#39;re a really busy clinical setting but I only have two or three minutes with a learner however there are your aspects of it that you can that you can sort of you know work on even though even the fact of asking open questions shows that you&amp;#39;re interested in the learner and I&amp;#39;ll talk about that a little bit more when I talk about my own research so that&amp;#39;s one one definition to think about so let&amp;#39;s talk a bit now but what&amp;#39;s involved in this some activity that we&amp;#39;re calling coaching and this is work from been levels systematic review published just before Christmas and also from other other work as well so the coach provides individual individualized real-time feedback ie performance data so we we saw an eider that would have been you know at the end of a rotation but if it&amp;#39;s in the moment it would be that individual feedback what I observed when you did when I saw you doing something the coach and the coach says goal for the coach or the excuse me for the coachee for the learner so it&amp;#39;s co-development if it&amp;#39;s the moving away from the more directive model where the faculty says well I think you need to do this this and this it&amp;#39;s trying to build that that self direction by coaching them in what what would be appropriate goals facilitate the coach facilitates the development of new behave years insights and approaches to work so again it&amp;#39;s working to facilitate the learners growth its learner centric trying to foster reflection so what did you think went up went wrong here can you tell me what you were thinking when you did such-and-such so just trying in a very few minutes to get them to think through what they were doing and to self critique and and to think about what they should do next and this one is interesting about the coach has an analytic analytical understanding of the game what the games all about like the game of medical education or to become a surgeon or whatever an ability to motivate the learner towards excellence so and you know in other words they sort of know what good looks like even even if perhaps the you know it&amp;#39;s there if it may not be in their area of specialty that they know what it means to be a good clinician you need to have a set of skills and so that they can they can coach from that holistic perspective I&amp;#39;m going to say just a few words about my the body of research that I&amp;#39;ve been part of for a number of years and where&amp;#39;s Ivan silver he&amp;#39;s over there he&amp;#39;s been part of it for a while as well and this is just a local study that we did it at Dalhousie and then a larger international study at the bottom used about the r2 c2 model and you have a brochure there&amp;#39;s a couple of row shares on the table the thing is just due to the limits of time I&amp;#39;m not going to go through the model in detail because I want to share some of the research with you I will just or I&amp;#39;ll go to the okay first of all I&amp;#39;ll tell you so the theory this you know we a lot of theory and we gave this a lot of thought and had some prior research on self-assessment and receiving an accepting feedback before we came up with the model and there are three bodies of sort of theory that inform this work this notion of informed self assessment that you know we need external feedback to inform and let us know how we&amp;#39;re doing and so in order to get the learner to reflect upon what external feedback they need you need to ask open-ended questions you need to guide reflections and guide them to self critique it uses learner centered approaches so you&amp;#39;re really focusing on them based on humanism you want to post positive their motivation and theirs and support their self direction and then the third area are the cognitive domains influencing behavior change we know that there are many factors that influence behavior change and we started using a learning change plan to address these just to help that scaffolding help that learner move up the ladder so it&amp;#39;s a four stage model and you&amp;#39;ll see in the brochure and and in the brochure we give some of the purpose of each of the phases they&amp;#39;re iterative and go back and forth between them and some phrases that are used the phrases are taken from our research there are ones that facilitators have actually used and residents have said that they&amp;#39;re helpful so the first phase is the rapport building or the relationship building and what we&amp;#39;ve heard from residents and supervisors that even if they know each other fairly well you can still learn more about them asking questions like well what kind of feedback have you had before you know what what is really helpful for you you know what you know what really what did you really gain from this past couple of weeks what you know what eh what are your goals so again thinking of whether it&amp;#39;s in the moment or when someone comes to your clinic just for a day or whether it&amp;#39;s a two week or the end of rotation session that you&amp;#39;re doing then exploring their reactions so that and this really comes from them they work on inform self-assessment that if you don&amp;#39;t agree if the feedback you&amp;#39;re getting doesn&amp;#39;t confirm how you see yourself it&amp;#39;s often a surprise and it&amp;#39;s I can often be an uncomfortable surprise it can make a sad it can make us angry if we get negative feedback and so it&amp;#39;s being upfront and asking are there any were there any surprises and what I told you about my observations or are there any surprises on this end of rotation form form that we&amp;#39;re reviewing and then just to talk about talk about those and bring it out on the table and have a conversation and normalizing it you know we&amp;#39;re all disappointed when we have feedback that doesn&amp;#39;t quite jibe with how we think we&amp;#39;re doing and having that conversation then moving into the first C which then once you get the reactions and the emotional part over and moving into the content the knowledge bits so so did what I just described to you and when I observed you you know taking out those sutures or whatever it was that it makes sense to you can you understand why you know why I made the comments that I did and are there questions that you have and let&amp;#39;s make it something more complex maybe it&amp;#39;s time management around managing a roster of patients this is what I observed what do you think and then where where are the opportunities for improvement based on my observations and how you think you&amp;#39;re doing where are the opportunities for improvement which really gets to the coaching the coaching for change however what we&amp;#39;re finding is that you sort of almost have to go through you know those three levels beforehand to get to the coach and to come to that agreement as to what to work on so that&amp;#39;s what the the brochure is about this is the the coaching then is real what we in the last research that we did we used a learning change plan and this is what it looked like and you can see it&amp;#39;s quite involved it&amp;#39;s got the describe specific observable changes that you want intend to make you know the specific actions or the timeline timeline to when you think you&amp;#39;ll see results identify the resources the challenges and how do you know if you&amp;#39;ve been successful when we handed this out in the study our study we thought that it would be too complex but in fact they said that it was helpful now mind you it wouldn&amp;#39;t be helpful it would be too complex for in the moment feedback at the table at the bedside or in the clinic but thinking about it when you&amp;#39;re doing your monthly or try monthly or every six months progress meeting with residents this might be a tool that would be helpful so the goal really of the are to see to model is to trying through this facilitation to try and facilitate you know the notion that this is X feedback is external you know it&amp;#39;s external stuff coming in it&amp;#39;s not in control it&amp;#39;s a threat and it could even be about something that&amp;#39;s bad something I&amp;#39;m not doing well so trying through this listening hoping questions you&amp;#39;re accepting motivating coaching trying to help the residents realize that this is their data as use this is your data and it&amp;#39;s your opportunity and you can be in control of using it to change I know it&amp;#39;s pretty pollyannaish but it&amp;#39;s um it&amp;#39;s a goal that&amp;#39;s our reaching goal and actually I&amp;#39;ve heard from a unit that&amp;#39;s been using it for a couple of years that it is actually beginning to change the culture slowly so I&amp;#39;m just going to go if I got two more minutes alright so the overarching from our research which was an international study on five sites what was valued by both residents and supervisors and using the model they valid they valued the facilitation of the resident engagement and reflection they valued the focus on getting better they valued the collaborative supervisor resident relationship they created that learning change plan together they found it useful with residents that varied levels especially those that were excelling and then their recommendation was to orient and prepare residents that&amp;#39;s new in the same manner is actually and I think later on this afternoon you&amp;#39;re going to hear more about development of repairing residents and faculty together or a change in the culture I&amp;#39;ll just very quickly go through some of the quotes and what this one was around engagement and reflection because it was so different I guess you&amp;#39;re less likely to be defensive because a lot of it was just identifying my own weaknesses he helped me to come to that conclusion and he prompted me to self critique one about the outcomes orientation I&amp;#39;ll just let you read that one so valued the focus on getting better really the specific strategies to take to get better the collaboration and co-creation of a plan we came up with the goals together what I liked about it was what can the supervisor said what can we do can you achieve your goals and the resident that makes me motivated to know I&amp;#39;ve had that support finally with residents who were excelling supervisors reported that it was benefit because they said it provided more substance and structure for their feedback conversations so it gets you to the next level level it gives you more to say normally I would have just said hey you&amp;#39;re doing well Arion so they said that using that format even if they didn&amp;#39;t write it down and fill it out it gave them the questions to ask ask them what do you aspire to even if you&amp;#39;re you know excelling in everything already what would you aspire to and how might you set goals and when would you like to achieve them and who do you need to help you so asking questions that weren&amp;#39;t available before and I think I&amp;#39;m just going to finish up summary and next steps yes so this is summary overall that was just a very brief overview of the research that we did Joan oh I&amp;#39;ve been thinking about CBM II and again this is from aritaum Bo&amp;#39;s were you know point no it&amp;#39;s an evolution it&amp;#39;s dynamic it&amp;#39;s not the night and I&amp;#39;m it makes me think it&amp;#39;s a disruptive technology it&amp;#39;s shaking this up and it&amp;#39;s made us realize that our education health systems and cultures you know don&amp;#39;t probably value sound education principles as much as they should you know the kinds of things that we&amp;#39;ve just been talking about our really based on sound education principles like things that should be the foundation of our programs assessment for learning observation to collect data longitudinal relationships safe respectful environment coaching our learners become you know the best that they can be and I think as far as changing the culture I think yeah we first you know we first changed the culture by changing ourselves and we know that one of the best ways to change a culture is I role modeling and and so in this case being open to feedback and role modeling various other ways of trying to be more open and open to learner&amp;#39;s giving us feedback so thank you very much and if you can have some time for questions and then I&amp;#39;ll look forward to [Applause]

Transcript for:Feedback and Coaching in CBM II

Transcript for:
Feedback and Coaching in CBM II