2019 TEACH Education Day

October 23, 2019

Self-Regulation of Professional Practice: Promoting Insight in Students and Clinicians

Speaker

Kevin Eva, PhD
Associate Director and Senior Scientist at the Center for Health Education Scholarship
Professor and Director of Educational Research and Scholarship in the Department of Medicine
University of British Columbia

Objectives

Upon completion of this activity, participants will be able to:

Discuss the evidence of relevance to enabling effective self-regulation in our students and ourselves.
Explore ways in which health professional educators can facilitate the development of insight into one’s knowledge and skill set.
Consider how these practices might shift the culture of medicine in ways that enable improved professional practice, faculty development, and teaching/learning.

PowerPoint

View the TEACH Award Recipients

Video Transcript

Yourself in your role and so I'm so pleased to introduce dr. Kevin Eva and their 2019 has a reveal his 102 not part of that so that dr. IVA is associate interpreted senior science director of educational in an apartment that's under breath modern biology we also energy for the German education that rebus has insulted broccoli around globe including advisory roles for the nvme the national health services education master of depression medication dr. IVA and policies a variety of mutations narrative assessment critical thinking interview she started me is teaching and scholarship awards they aren't me haven't you know I gave myself that excellent I just wanted to echo this empathy deserve it I'm thrilled that you came at the lunch hour I trusted most fear here for lunch rather than from you so please do feel free to keep on yourself as the our work London's quite happy to be interrupted an efficient they provide a little more for them any comments or questions that you want the topic for today is moments near near my heart it's self-regulation and Brian issues that are wrapped up in that and specifically what an expensive time I talked with you about is how we might enable learners to to better understand the strengths and gaps in our developments obviously in the hope of improving performance or respectively as we move along I'm using that word learners very deliberately because I believe everything's going to say is true for everyone from medical students on through the hardy once career the challenges I'm going to perform are relevant to us just as much they're relevant to our learners and so one of the things that I hope they'll think about that for a while is how we might wobble some of the activities we may be resolvable all encouraging to think about at the same time as encouraging our learners to take those so these are the questions that I plan to address again more than happy to go off on the tangent that there's something else that you'd like to discuss one set the stage a little bit by saying why aren't going to go educators interested in this topic at all what does the research literature tell us and all that really building towards trying to give an understanding what we as educators need you to attract promote an encouragement of practice and sake of our students a mistake about her Betts has its guards looking to say to to get concerted if you look internationally every health profession that come across would you look at their regulatory documents in terms of how the profession is expected to develop and maintain his confidence universally across all the health professions across every every paper that I've looked at and that Peck looked at in this 2000 review it's been seen that the ability to self assess is there to be a cornerstone of professional software finish this is the the Shenmue model of self-regulation because where's the shampoo bottle and lather rinse repeat here generally is that we should take soccer Baram abilities too except to try to fill in any gaps we have a practice and just repeat that over the course of our careers in principle time very sensible and it's driven partly by routine events like this one where something is going wrong and and it's often blamed on overconfidence on the individual who's up beyond their capacity and if you've been in a role where you're trying to help students who are struggling oftentimes we complain that they're locking in sytems and they're seeing three unaware of the limitations of their I'm not going to belabor those examples but and this is just a simple model to get our conversation started in terms of where does the concern to generally arise usually we start complaining working the insight or self assessment or variety related concepts because the people having to play the depression abilities why this has been built into our someone's our conversation about how to perform well as a profession is think that if we help people better understands the gaps in their practice and then what that will offer is some guidance regarding what they should work on but some our nation writes better awareness of their needs and gives them some benchmarks for towards all with the goal that is ideal the individuals being competence and confidence able to do the things that we expect of them in their roles and really are comfortable to do those things as well again that's the wall just American folk view of white matters let me tell you a little bit about four years worth of research that has led us to have some concerns about whether or not all was as simple as it might seem Gant surface if you've seen the matrix burn after this whole loop he'll for the next hour and and think about some things that might seem intuitive to us but maybe don't hold up your closer inspection I'm going to start my comments around kind of for illusion so that I'd like you to consider the first of which is simply stated as being an illusion that we actually have unique insights into our own strengths or weaknesses then we can become a good self assess errs in any sort of generic way I'm going to tell you one set of data but it's this really is a 30 to 40 year old literature it's accumulated in a variety of different ways this particular study of choosing primarily because it was our effort to correct a number the methodological flaws that others had suggest the proclaiming this literature don't need to go into the details but have a different really wants to afterwards what we did was we asked students in the context of something we call progress test how many questions they were likely to get correct and this was chosen in part because it's an objective measure waiting for you can just count how many questions we got rekt and the students in this study I had a lot of experience with this tool this is done the McMaster when I was there and students it in this example when they same difficulty every four months of practice it's a way of giving them data so they can track how well their knowledge is going over the course of time that curriculum so they know what they've scored on this in fascination of the very good sense of how well they're likely to do as results when we ask them to estimate how many persons they were right before they actually sent the test despite what it - sent the correlation was very much alive before we had crazy people were surprisingly poor by telling us whether or not they would succeed on this activity when we ask them after the exam how many questions did you get right so they're now seeing all the question is that if they've experienced that struggle in trying to determine the reach ones are they comfortable with in which one said they struggled with before they she got better still not great in fact this is where the higher correlations do see in this literature of what was most striking to us in this particular context is if they had simply told us the score they got on the previous example then therefore this would be level for myself so they had all the data they need to offer to discouragement but for bribery they just work visa again this this is count for 40 years been a bit of a treadmill where we see it was only like this and and but that cost will be corrected brief we saw this aspect my for experience with ourselves than we do with other people how can it be this there we have such little more additions so methodologies that we get over and over again challenges come forward or people say I know I'll never play Quebecer football so therefore I must have something capacity to self-assess we need to fix it need to teach it better we're just not doing that up or my favorite reaction basically to us and they got my favorite reaction because that they think dude the most satisfying research results that come across in my career was the study that showed that if you ask people how well they self assess just a minute pretty sensitive uh beverage there might be the best guess hey this is somebody's developer behind this it's um ready to grasp or feel fairly comfortable with so I'll just pause and ask you for your reactions alike of any of these or in the menu this is comfortable to that we struggle with these sorts of assessments two points I agree with all these the other lawyers do you see these are a dichotomy and then we got the predominance paradigm or Ritalin heat these are great questions and partners they actually speak fundamentally to what my skills are our generic competencies right so so we talk about people as with that person understands their abilities so that person's a good communicator or that person starting with plot of activities each one of those things - a surprise - degree is context specific we don't tend to be good communicators overall we can you get better in areas that we know something that we're more comfortable with and communicate less well in areas where economists as with the material domain or skills that one has capacity you can't your beer be a much better self Assessor than for skills where you don't have the capacity it will all search the name in a little while too during course at which but the general notion is that the appearance of good self-assessment of being able to judge whether or not you perform as well is actually driven by being able to perform tax and it's design the capacity to self-assess it allows you to do that sacrificing to do the task to makes it look like you and you for it ain't one of us will written something for about others so we'll pretty good tell us that something is important others but the question is do we ever know when working on Sabre or the other so you sure that is a full correlation of mystification happening well wrestling that whole correlation that you get design yeah as a general rule for socially desirable activities anyway give it tend to overestimate more than their best names there's some couple in fact some of you might come across this little bit of the camp is all of our kids are above average by definition just every possible but where people show up in the study like that in terms of where the rural residential area estimators is very much a matter fact that the difficulty is I gave them very easy tests they appear to be under respirators because there perceptions of their abilities in the serve the manor are having to change for they did high scores relative and expected to give a very difficult test pretty overestimate or so so the overall difference in average I just hear a lot of information that city is a difficulty of knowing where you stand well to company except history report and welcome back about the issuance of an important issue then we can go on bubbly for an entire term talking about this literature it's so fast but but is extensive the main literature reviews I conducted in different areas not just medical condition generally they've all come to the same conclusion that's opposite is poor I'm not gonna believe it a ship I just wanted your little exercise just this one way of reinforcing and why this is so difficult for us and to do that in ask you to trust me just for 10 seconds and to close your eyes this is a very trusted group this what one day we readily did that I just want you imagine further for ten seconds or so yourself driving okay open yours in was under channel capacity I have not just decided that everybody's hungry or just put themselves up it's a it's a fairly easy thing to do can somebody tell me about the vision that you just had we were thinking bigger self so I'm beautiful so in honor that's it so it's a very detailed didn't break specific situation where you were you're following a cop that you're familiar with even though the names of the streets make more pellets that report on the radio you what happened at work before yes I'm starting the tip for today but when you're imagining national dive yeah yeah but to me it's a general principle if you think that yourself driving you're actually investing the time I thought the question was about and there be the the scenario just enough yeah yesterday in that scenario happens saris a weird question I'd realized but did that that's what she what I'm trying to listen to that all of you had no trouble imagining yourself strike the odds in the room but there are a lot of details that dozen equitably hit right there are things that impact on how safe they reside the dirt came from the mind I think right how how well do our drive for themselves those II thought obviously determine yet right even the worst drivers among us they don't think into accidents even 20% to take your car it's a very rare occurrence and so the things that are very readily available memory are the things that change drive our perceptions of what I use details what they're I don't know what's distracting us we don't know we're hungry and all the larvae sitting on our our passenger seat instead of actually focusing on the road seat up the determine what actually happens but we don't still believe with our assessment of our passengers of Jacob one step further for how many of you was the baby you're imagining [Music] if things like this generally what you think is people think idealized situations traffic typically usually it's a warm sunny day driving thru out to dry we're not generally focused in our imaginations about driving very pedestrian start in the middle of the road the most typical scenario the easy situations are the ones that go to my house readily and we know from basic psychology that things that come to mind more readily to be thought was more probable so we get into this habit of of imagining ourselves driving late getting effectively or working well in clinic and that drives our sense of our building levels of the general so Paul that is just an elaborate weird way of saying this when we ask you to self assess if you can't imagine them as having unique access to the lots or behaviors or experiences but the facts they are growing inferences about their abilities just like a joint this is what we only make if we were still using Kings that can be quite palpable in determining what our actual times that's the glass-half-empty working there's a reason for that rose colored glasses take a lot Fitness in them and that is quite simply this that our perceptions of ourselves aren't simply reflections of our past performances but they also help predict how well in the future that the confidence is important to us because if we don't believe we're capable of succeeding possibly we will succeed our much lesser yes just referencing some work that is early seventies but has been ongoing suggesting that that that confidence that I can't seem like overconfidence when someone doesn't have the ability is actually something will lead them to help develop the ability rather than simply problem economy we seem to have some implicit sense of that you can talk to to learners and their various levels and multiple professionals we hear all the time ways in which people discounts negative it just as an example some days I go home whenever your dog but other days you know if I had a head there's next time circumstance that helps us account for why that that Pacific a pig didn't go as well as some other guess we have never to my recollection for you talk about how just injury difficulties they don't find excuses for what things went well as readily as we find reasons to to this couch that's why he can't sound like people are deluding themselves but I think often we do this for court agrees Caroline Walter from University on O is a hepatobiliary surgeon and she was trying to study those moments in the operating theatre when people queuing to the need to imperative automatic mode basically attention so she got talking people that when things went wrong and they routinely say things like this is a dramatic example but things weren't great if it wasn't had put a stop to the visual hierarchy and parted she service long as things became it's important to have these reflexes of Sydney change to do things better than we work with patient safety but it's important have these reflexes because otherwise those surgeons aren't going back to the operating right if they truly come to fundamental belief that they are not capable the candidate was perfect for many of our patients on the other side of things we also have tendencies to see the positive these are some data collected from United Kingdom where the developing web-based feedback system for students who have gone through the Aussie and he was to come back for the documented so that people could continue to learn from that experience when they tracked how readily people have used the system what they found was those who was built for the ones who were struggling especially and less likely to use things and the people who got actual comments or a ratings that felt very coarse and then kept going back to it over and over again as a way of doing this but lose sight of the fact that when we get a negative feedback it can be quite jolting and creates the state of dissonance for us where we are trying and then believe that where were progressive or sir and then after grapple with these indications that maybe they would not do this without work it's okay this that's just the foundation for what we really want discuss which is what are you sudden we support students when we can't trust that any one of them has reliable trustworthy access to their abilities and here going to offer four bits of advice Margaret of this literature what is it we need to keep in mind always whose opinion matters among us to our students remember they have at least 25 years worth of experience with themselves they know whether it was have perceptions of whether struggle button and it had successes somewhere they would have gotten clinical school then we come in and sometimes or with them for no more than a few minutes over three hours that's the same and start bouncing on the biceps it's easy to discount if it's completely inconsistent what they and what we looked at what was deriving students to generate learning goals after a nasty you know Megan thinks about the details too much but he didn't matter what rating we believe deserves didn't even matter how strong the feedback was in the students only impressions would determine whether mother generated verticals was their own perception of their own performance so if you're here to try to help students will be on this didn't know how well they're performing we need to know what they think of themselves in order to make sure that what we're telling them doesn't just fall on deaf ears because it's contradictory or so inconsistent then they have make sense of us this is the distance issue that I was I was other due to earlier release into the stair dilution which is we tend to romanticize how desperate our learners are for feedback they tell us this all the time so it's a completely our moment but you're here all the time and maybe careful studies have shown students have a desire for more a factory of course parties just look we're getting feedback there's not recognizing it but nonetheless the reason I'm mentioning this is the doesn't take a lot of digging to start to see that when the students say I want more feedback often what they're saying is I want some reassurance then they want to have some indicator that that they're on the right path and in medicine at least to establish the culture we're giving feedback is often seen as a as a sign somebody's not doing what because we were the people who are doing fun it hasn't become part of our core normal processes to make sure that everybody is is giving phew they might have a phone for words that means me to say it again the only comments probably suggest we need to think that skill development rather than trying to turn people into good self accessories all credit conservative even if I go back to that wall rather than as if we're given feedback so people self perception improves what we should be thinking about is what feedback will help the student improve performance and just allow the appearance about your self-assessment to come needed probably shouldn't convince them that they're with the bottom dwellers in fact they are about Plus what we need to do is help them move up the ranks and and and not worried and started I'm trying to illustrate that with just this really simple study and this has become a classic in the letter trigger it's actually one of the people that products for research that first makes people laugh and that makes people think this has been been replicated at medicine but the data I'll show you is another reference psychology students they're given a test on correctly and then they were split into four house so my definition got better about each portal when they asked people to judge how strong their grammar was very different impression very flat line even those the first quartile generally believe their grammar ability was above average why the study was so influential is what this simply repulsed is what I said it's been that bus at the time it's office for the current dining targets that burgers and maybe some self-assessment maybe it is that you don't know how big our peers are and so they show the test responses for half a dozen peers to each participant and then said to them now how do you think it is and so the good parts the fourth quartile should be able to look at their the Pearce responses and say I'm actually kind of in most of these schmoes and raise their self-assessment support it is the interesting part was down the other end it was the first quartile also said I'm better than most dish Mo's and it led Kruger Donny dark to title paper unskilled and unaware of it does it be have a perceptual the truth of a perceptual blindness if you don't have skin in the area you don't know how to jump with my head still have that right because if you did know what would create for a good grammar you'd be using good grammar and you wouldn't be down with that one minutes why presenting aside in this moment is specific feeding their follow-up study they should taught people how to use grammar appropriately so they move them to the right along the skirt and when they did that of course their self assessments appear to be more accurate because their performance improved immediate it wasn't the derivative the get better performance by convince people in their self-assessment which more generally leads me to the us dating that what we need to ensure students have access to is better data we need to to create opportunities for them to discover the backers of them of their knowledge because that's gonna have more than anything there's a larger trucks on the call test and acts learn so there so if you take the past you have to learn the material better than if you study it multiple times these they are a real-world example of that this is masters performance on the national licensing exam reviews in Canada the equivalent to your USMLE master is globally known as being an exceptional educational institution but it created called based learning this bread many convictions around the globe its deep dark secrets was a theory that the National exam notice five times the national average this was an oven issue club based learning versus conventional education because every year this is the progress learning year what happened was that early on in the curriculum for the first 20 years or so the philosophy of 11 works probably learning method the education needed to be student-centered and tests were the opposite there were a teacher center because we're part a teacher decided what students needed to know and so for that first 20 years from the 1969 to 1980 night they got no feedback beyond what their tutor pulled them and beyond what they themselves could consult their from their experiences the disconnectedness line is when they knew that progress Tesla mentioned at the start of this talk when they started actually giving people an examination and even the hard data that were allowed to to reliably the only track their progress then that enabled a massive jump in performance and the assessment system of radical things became more robust over time at least the gentleman here's a lot of I'm curious to see your experiences in terms of where you get to support your practices or your determinations of it what you need to do continue to to retain your competence these are a few examples of things that I know to be used that here in the states where different organizations are finding bodies and others have tried to create systems to get data typical sentence that they're well better perform their performance depending experiences with with one of these or something else it's you felt has worked well in terms of being able to do make a nice gesture is is it distressing they can just pretty good moments and your story doesn't the literally practicing getting some opportunity to just have this conversation whether the simulation itself may be valuable but as much as anything - you might get what other people are doing and oftentimes we don't need to disclose that we would've done something different about being able to hear what other people they're making tracks of X maybe I should put under the pretense probation so their actual equipment their lives is directory in specific situations anyway I get to keep my imagine case it's an interesting example because there are homo and that was good for some date would suggest that when we think that basic profiles as a whole we're actually quite bad and even others water-based pockets and so is a generic that something's in there how long my patients doing overall it's a very cool thing to do but that grew there with individual instances that can be a very important piece of information the question is always how do we aggregate that into something bigger so we know how we're doing it again these models there they're coming because of increased awareness that you don't have access to the information that they need to judge whether not there it's something that were Yorkist all desire of difficulties it's a very compelling model suggesting that sakes don't simply help us learn they're necessary for learners especially trainees they don't change us but here as they put themselves in situations where I make mistakes it's - has reinforced our thing to do right eating this as it was a cdata alluded to we can to seek repulsive ever comes and so our well the legislation part is to make sure that people are getting those opportunities to experience mistakes in the State Department where they can actually engage and learn and force himself beyond whatever threshold energy which will leave me finally just through broader point about how we need to think carefully the our own engagement with our trainees that's our models of feedback often amount more to dumping data on students rather than actually engaging in feedback conversation this literature review that according on this slide what did a variety of different interventions for how teaching before us could improve and the intervention was the most robust in his effectiveness with the presence of a coach because having somebody with whom you could your diet you can share your perceptions of things have been pumped other questions or offer ideas as to ways in which you might continue to improve because they have a definition if you're struggling with a particular task probably not the right person to decide how to get beyond that because you're struggling if you remember to go then we've been doing the hump critically important in those sorts conversations is that the coach or the feedback of our community its credibility students don't listen if they don't create the source is credible and in this study German sergeants or that is the credibility has multiple dimensions it's not simply whether mind you have to know where you're talking about but rather it also demands credibility in terms of what does this personal motivation for getting to sleep feel like they're here to help me get better or just because they're they have a job to do - emotion or what they want to demonstrate that they're so much smarter than I am students respond well when they feel a connection with someone I want to do grow with them as results so the models that sure are men in all of your feedback workshops that Linda number of feedback points makes the positive the negative so they're all grounded in American very principles when as students that practice becomes and it seems like we just went through the motions are you as I've said something positive students have select embrace this must be coming available they don't get set up at that price this is why there's so much emphasis being put on the effect or educational process is known as the educational Lance [Music] but don't mean that you need to become best friends with your learner's that don't mean you didn't presenter but the learners need to feel like like part of what you're doing is trying to help support their well be there into the collaborative alliance to to try to help them be as good as which really all for those things that I've just tried to do a basic talk around this is really my effort to say that we need to change the culture we need to start thinking with self-regulation as being helping people understand that they can rely on their selves we need to make people uncomfortable with with seeking didn't recognize how important it is to do so search for identified with queues or the visiting the help inform as opposed to getting away and at find ways through church official alliances and other ways to try to optimize dividual people are receptive in addition to that little bit some house especially for quick slides truth which I hope reinforce the typical messages that have tried to convey and true which I hope we can do a little bit less comfortable even if you're already not totally comfortable just explain that what do thing their own strengths is that we need to think of education as and can tell students don't assume you need to try to create opportunities for them to discover this room are going to be impactful there are good guys in there for how to deliver feedback how to help students work on their performance valerie has has a very good paper in this regard that comes to the conclusion that there's no one-size-fits-all approach plugs into the connection hands but but pretty lunker conclusion was also the feedback is effective to the extent that learners are receptive to us and so we need to think carefully how we avoid the situation where where we are telling whether something is so inconsistent with what they believe that might as well be tough to the table 7 - individual in terms of the discomfort I think the most important thing I can say in all this is is that it's going to go wrong there there is no magic bullets I can hear some news because explore this literature and experimented with these sorts of issues a lot but there's there's nothing you can do make it work every single time so I started thinking feedback isn't proper what's your principles or rules that that help steer the conversation but but you need to be prepared and be the most important is just staying engaged I cannot disclose a specific situation I have recently with the post doctoral candidate room who was working with me for 18 months and then entire time kept telling me how frustrating she's a Canadian Republic she felt like put a Keith virtue nice creamy very people to read between the line and said that she stay over your next I was felt like I was saying what I meant but should we think which is like language she really felt like I just she wanted me to be absolutely once was telling her that's a one day she had a rematch there I've been asked for feedback and I took her to face value I just / thoughts and it was an absolute disaster despite all this I said we're trying to be student-centered students don't necessarily know what they want and then there's a lot of work to do to prepare from that moment Salani I also love with your help to say during a small doesn't necessary - recognized laughter today I'll show you is taken from surgery context students did pre tests on this task we've got an educational intervention they did better and then they were randomized to either be told what they've done well or differently so I can think at that point they took regard to certain feedback continue to improve reinforcing that the self-assessment isn't sufficient those in the compliment group didn't know for themselves how to keep getting better but the reason for showing this variant is it unfortunately those in the compliment for the entire teacher readiness it felt better if elevated burgers on there because they're told how well they do we do that over there just dress [Applause].