Taking Daily Teaching and Practice and Turning It into Research/Scholarship

Good afternoon everybody yay excellent I see a lot of uh familiar names familiar names which is wonderful this will be great for our conversation today today's going to be a little bit of an active session a little workshoppy so um we're looking forward to some great discussion uh today we have with us uh Dr sirkin toy Dr Troy is a pretty new faculty member with us um he is assistant professor in the department of basic science education through Virginia Tech Carilion school of medicine and today he's going to be talking with us about taking our daily teaching and practice and turning it into researcher scholarship so unlocking the scholars scholarly uh Potentials in throughout our teaching so we are very excited for this very practical and meaningful session and hope that you all will engage in some active discussions so um take it away sirkin all right thank you shared I'll go ahead and share um my presentation slide let me know if it's um if you can see it in the presentation mode or not looks great awesome thank you um yeah before I begin I would like to really um thank the and teach team making this possible and and working with them in preparation of this presentation was great um so thank you and good afternoon everybody um thank you for coming to this session about um scholarly potential Unlocking The Scholar Potentials in teaching um so my objective today is to really lecture less and discuss more um in in the context of what is relevant to you um in these different aspects of turning teaching into scholarship and um again this is the agenda for today brief introduction and then we'll have a small uh group work in breakout rooms and then we'll come back and report on what we discussed and then a few minutes for closing remarks in Q A's so to begin really um what scholarship means in in a traditional sense is is that it meant Discovery um creating new generalizable information and that was the traditional sense but back in 1990 Boyer noticed more and more uh College professors were engaged in different aspects that were not really considered scholarship and they were not getting credit through the teaching application and Innovation so here to define scholarship to include these other aspects and today we are focusing mainly on turning teaching into scholarship and then um Lee Schulman came up and elaborated on what it meant to turn teaching into scholarships so he wrote a discussed three main things one making the teaching materials public and that they are available for peer review and critique and that others can reproduce and or build on these materials or products so when we look at these three components really we um we know that the Crux of this is is publication dissemination so how can we turn it into Publications so before we get into that let's take a look at what medical education scholarship has been over the years in terms of output topics and different characteristics when I put medical education term in PubMed these are the numbers I got so we noticed a pretty steady increase over the years and I I limited it by last 30 years just to make the graph a more reasonable to see one thing that I noticed is that you may have already appreciated as well the last year we have the blip maybe that's just a blip or maybe it's again covet to blame for that but I suspect that that number is gonna remain steady this year or even go up it would be interesting to see how that changes this year and these are some of the topics that people have been publishing one review article back in 2012 looked at all the different Publications and kind of put them in more common themes in terms of what people have been publishing about medical education research so when we look at this list many of these issues are still quite relevant today issues regarding student assessment still relevant in clinical skills training clinical clerkships problem-based learning um we noticed professionalism in medicine then number 22 scholarship and education which is what we're discussing today as well um some of these other topics also quite relevant even today so what really is educational research um according to American educational research Association the main focus is on learning processes and the characteristics of Learners as well as some other factors that are pertinent or influence educational outcomes educational researchers try to describe describe understand and or explain how learning takes place while using range of rigorous methods that are in line with the educational research question that's being posed so that's that's an important aspect that's in line with the research question so I just wanted to put this um graph to to show what clinical research process looks like and highlights some of the main differences between educational research and clinical research um although randomized clinical trials gold standard in clinical research in educational research we don't always have an opportunity to randomize Learners sometimes it's because it's unethical to deny learning some groups or we are limited by the number of Learners that we have in our contacts and we can't really um randomize we don't have too many so then what does educational process look like I just wanted to summarize it in in terms of these different steps and that those are the steps that we'll hope to discuss today in the context of your own interest um so educational research is um quite practical in a sense that it starts with an actual educational Gap you may notice a performance Gap and wonder why this is happening can we address this and oftentimes we do even needs assessment to get the scope of this a gap or problem so that we can study it further Loop review is a very important Next Step that will inform our research question as well as study design and the next step is data analysis interpretation and interest of time we'll just focus on the first three steps today so um just a few words on educational Gap um this is an important First Step because not all um performance related gaps are educational in nature so one may notice that um patient had those handoffs are not um happening optimally and he or she wants to create an education to address this Gap but if the actual problem stemming from time constraints no matter how good education you provide to the providers they may not be able to go through the steps to doing proper handoffs due to time constraints so it's important to identify a gap that can be addressed through education for doing turning teaching into scholarship to be able to see actual differences in the outcome so here are some sample problems or gaps that one may notice in their practice second year medical students may be struggling with interpreting ECGs or maybe you are running into professional simulation sessions and notice that some teams are more efficient than others and you may wonder why that could be a good question to start asking as an as an education project or maybe only a few residents are reporting Adverse Events or near misses and you wonder if there is a way to increase that so these are some um examples um so we we did um circulate a pre-workshop um guide that kind of asked you to identify an educational problem or Gap in your own context and talks through these different aspects of it I wonder if we we can get a volunteer to um talk about their problem just briefly what they identified as an educational Gap or problem in in their environment can we take just a moment and um seek a volunteer to share what they think I can share I don't know if you can hear me yes yes we can hear you so I'm Jessica Fleming I'm one of the chief medical residents in Internal Medicine um so one curriculum that we are working on implementing this year is diversity disparities and inclusion in medicine and we noticed a gap in that in our curriculum and so we're hoping to write that up this year but that was the The Gap that we really noticed was that we weren't spending enough time educating folks on cultural humility and diversity and disparities of History Etc thank you for sharing that that's great um do we have anyone else who wants to share this is Bob evansford from the VA and we identified some projects and uh gaps that we see especially in transition from inpatient to outpatient setting um the last thorough review it looks like for the literature review looks like it was 2013. and even then you know we're seeing things like Med reconciliation education follow-up Labs Diagnostics appointments and then also the transition of care is impacted by proper documentation of the inpatient stay to outpatient providers yeah I think in 2013 they listed that um all transitions have at least a 50 chance of some induced error great seems like you did some um lit review too and somebody already did a systematic review maybe that those are always very great points to start with thank you for sharing this is great this is exactly what we need and the breakout room sessions to begin with somebody's identified education problem or Gap and work through the worksheet um so continuing on literature view um after we identify an educational gap or problem the next level next step is to do a little review um this is important because um I liken this oftentimes as you walk into a room full of people discussing very passionately and you don't know what the issues are so the logical step is to sit down listen quietly and kind of understand what the issues are once you have a chance to understand appreciate the issues then now you have a chance to contribute to the ongoing discussion so without knowing what people have done in the in the literature it's hard to contribute meaningfully to that ongoing discussion so it does help us to really get a better understanding of what the Gap is surrounding our problem and what theoretical Frameworks people are talking about in terms of maybe helping alleviate some of these educational gaps it could be mastered learning if it's skills Hands-On skills it could be rapid cycle type of thing in a simulated environment for skills again or maybe it's problem-based learning what the theoretical Frameworks people are talking about and then relevant variables oftentimes you'd like to draw a straight line from our education intervention to outcome um but it's it's almost never the case they're always intervening variables and what are those relevant variables can also be found in the literature also there are some confounders that kind of lurk behind and we can't even appreciate them can't measure them but every study has a weakness it's important to acknowledge them in the limitations and those can also be found in the literature people maybe discussing those and also how do we measure these variables people what people use to measure them those are also very good things and useful um tidbit knowledge pieces that we can gather from literature review and the next step is to formulate a research question which will then inform our study design there are a few tools that kind of help us guide through formulating a good researchable question one of them is finer criteria which is a helpful and mnemonic that stands for feasible interesting novel ethical and relevant if we're as we formulate our research question if we can check all these different aspects and then that that might mean we're on to something good so feasible for example just one example maybe we have only 10 residents that we're working with and we want to do a quantitative research to see if our intervention will change the way they approach certain things we may not have enough Learners to be able to establish any kind of a reasonable difference between Baseline and post-test so that goes to the question of feasibility if it's not interested singing people are not going to be interested to read it it's going to be hard to publish it and so forth and an ethical issue that we need to obtain IRB approval for sure and then the next tool that's helpful is Pico um and you may already be familiar with this because it's more applicable in clinical research but it's even if it's not applicable in educational research certain aspects of it it's still important to consider each one of these and then determine whether applicable or not population is always applicable we need to identify what the actual Target Learners are intervention you may or may not have an intervention sometimes we do have observational studies and you may not have any intervention in that case a comparison group though having a comparison group definitely increases the likelihood of publication but sometimes we just don't have the opportunity to do to have comparison groups maybe we have the same Learners you measure them Baseline and post tests and maybe even a follow-up after four six weeks or so outcome is always relevant and time is especially relevant if you are interested in knowledge and skill retention for example then you can identify different time points to do assessment so study design um this deserves a session or two of its own but just briefly we'll say there's quantitative qualitative and mixed methods approaches and each come with different flavors and we determine which one to use based on the research question that we have just a few words on educational measurement oftentimes in educational research we we deal with subjective and qualitative constructs we try to quantify them and it's not always easy and there may be several different biases that enter into the equations as we do that so as educational researchers it behooves us to identify and use published measurement instruments so we can withstand peer review process otherwise if you're just um writing questionnaires ourselves or multiple shows questions and things like that it would be hard to defend the validity and reliability of those tools um so it's important to again look through the literature and see if others used measurement instruments that could be helpful for our own project and the next thing is really interesting um as we notice um when we look at some of the projects there is usually a misalignment between outcome and assessment so maybe this is the outcome we want Learners to be able to write a very friendly horse but assessment looks like this what does it tell us if if a learner fails this assessment in terms of whether or not they can actually achieve that learning outcome it doesn't doesn't help or oftentimes what we really have in medical education context is this is the outcome we want the Learners to be able to manage difficult cases but then we go ahead and give them a self-report questionnaire to to kind of gauge their confidence or just give them a multiple choice exams now those data may not be very helpful appreciating what they actually can do so it's important to align our outcomes with the assessment that will also increase our chances to disseminate our work and the next is really to move Beyond self-report data and this is Kirkpatrick's four-step training um assessment um framework that's very helpful as we plan our educational research projects to think through these different steps the the bottom is really what I call learner happiness index where they let us know whether they found the teaching helpful engaging and relevant now that's that's helpful but not enough data and credibility to really publish it disseminated we need to strive towards level two and level three which tells us to what extent Learners actually acquire the intended knowledge skills and attitudes as well as what they can demonstrate on the job now the the fourth level is is really great but we may not always have the chance to be able to get at that level which is not always required for publication either so that one indicates the actual changes that happen in the Target patient outcomes as a result of training so this Kirkpatrick fourth step um assessment um framework is very helpful as as you think through um your own educational research project so now that was my lecture um I think I already talked too much now we're going to do a small group activity um teach team have been great at creating breakout room sessions now they're gonna place you in different breakout rooms so that you can have a hands-on experience through this so um we um we will have two documents which are going to be posted in chat one is a worksheet that will guide your um your process and the second document is a PDF that you may find it a useful resource as as you work through the process so to to make the best use of the time doing the small group activity it would be best to quickly identify one person with an identified educational gap or problem and start working through that example using the worksheet and also it would be helpful to have one person guiding the activity by asking those guiding questions in the worksheet and taking notes so that when we come back to the whole group they can report back what the process was and what the thought and what you have come up with um so that is the goal and I will try to float around between different rooms and we will also have the teach team be available in each breakout rooms to help facilitate the discussion yes and if you will just take one minute now to download the two if you click on your little chat bubble um over in the chat section uh Heather posted two they're a Word document and PDF and you can download those to your computer right now so you have them for reference and then we're going to be popping in and out of rooms too to make sure that you have all that you need and that if you had any questions just feel free to ask but if somebody will have those and then we've got a leader and then you can quickly facilitate uh whose question you're going to use for the discussion see on the other side okay here we go welcome back everybody yeah I hope it was a helpful discussion um I was only in one room and before I realized time was almost up so uh that went by quick and Sherry was not joking when when she called this a workshoppy um so workshops need to be longer so but hopefully that was um that was helpful discussing um through some of the important steps in terms of how to take um and identified educational Gap and turn it into a researchable question and then how to think through some of the different steps turning it into a scholarly product so as as we all come back now um can we ask at least one group to volunteer and report back what they discussed what the what the process was um I will I was a group three so we'll go through it Phyllis Whitehead we had a strong group um so the did you want me to go through the questions and answer those is that what you were thinking that would be helpful yeah the gist of what you discussed and what the process looked like and yeah certain kind of these are the guiding questions as as you report back well we started talking about what um topic would be of important uh and identifying than educational gaps so um I we talked about delivering um teaching medical students residents and and attendings on some level uh how to deliver serious news and using like vital talk and discussion which is a training and then we talked about the gaps in the literature courses that there really isn't a con an accepted structural consistent process of being able to teach um Learners how to do this uh and much of which we learn is by experience good better and different right and sometimes we learn by what not to do um and then when some of the um things is that they're how do you measure success with this training um and so the outcomes is a challenging perception of the actual learner you know do you feel more confident are you more willing the um more comfortable in that sense but also then we talked a little bit which I thought was really great from the literature um and it was offered through complaints we could measure complaint support communication uh those who maybe had received a training or had not um when they're you know when folks are giving uh complaining about bad outcomes um so we were in that process of going through not only extending the number of complaints by patients or surrogates or families but even by nursing staff and other ancillary folks so it was the outcome that we were talking quite a bit since we already have vital talks the training we have it as an evidence-based so guys did I miss anything else that was in group three spots on it was a good conversation yeah I mean communication skills always important and challenging topic um and as as you figure it out it's not easy to measure actual outcomes that you may have to be a little creative and as as how to identify differences in before and after that sort of thing um I think we have time for another group if if there's a group who's willing to share what they discussed I can share mine since I had brought mine up earlier mine was the implementation of a diversity disparities and inclusion in medicine curriculum for Internal Medicine residents it's a gap that we've noticed not only in our curriculum but in you know curriculums throughout the country the University of Texas has been one of the programs that I've noticed that has published on the topic and how they implemented it and then we created our own curriculum and I wrote up a different monthly plan essentially for each month of our residency year that would focus on a different curriculum for it in a different way that it was delivered and based on the documents that you all had us follow um we didn't get to every part of this rubric but of course the topic is incredibly important to study because as Physicians we pride ourselves on being able to care for a diverse array of populations and not only be able to care for them but to support them in their cultural differences and come from a place of awareness of our own implicit and explicit biases and how those affect our care since everyone has them and so helping residents improve not only their awareness of their own implicit and explicit biases but also how the systemic structure of our society has affected our patient populations in ways Beyond which you know most individuals can understand and that will make us better providers in the future so helping fill in that Gap through residency is critically important and is majorly lacking throughout most curriculums and most residencies the problem with you know quantifying how everyone has experienced the array of the curriculum that we've now offered is the the lack of a validated measurement tool we talked a little bit about the Harvard implicit and explicit biases surveys that were previously validated but there's been controversy around them and questions whether or not they actually reinforce by us and there has been overall a lack of ways to quantify how people have changed their practice as a result of a curriculum like this so some ideas that we thought of to be able to measure over time and or retrospectively look back where to see if you know the implementation of this curriculum changed the way that people were seeking volunteer work or if it affected the way that they chose research projects someone also suggested looking into conflict resolution within residency programs or amongst care providers of different cultural backgrounds not only just patient and physician relationships but physician physician relationships or physician and other Health Care staff relationships and also looking at not only patient satisfaction data but also patient outcomes data in the little enclaves where these kinds of curriculums have been implemented and see if it affected outcomes or patient satisfaction oh yeah those are great thoughts um and definitely an important topic but again um it's a challenging one to be able to measure certain aspects of it as as you also noted um so I think gme is is somewhat recent in these efforts but um undergraduate medical education I think have been trying to implement such curriculum um have you seen things um in in terms of undergraduate medical education Publications and see what people used in terms of measure measurement instruments or different um kinds of educational interventions whether they were effective or not so this seminar is extremely helpful for going back and looking at the validated tools that other programs have used um and so far other than the Harvard implicit and explicit bias tests um I don't I think that the majority like the University of Texas Publications that I've seen have come up with their own way to measure how much it has affected practice but yeah I have to go back and dig through again and see if there's anything else that's been validated because last time I checked which was about two months ago there was not yeah yeah and um in terms of doing A needs assessment it sounds like you you look through the curriculum and identified the Gap in the curriculum have you done some sort of A needs assessment asking the trainees themselves um and when you do A needs assessment by the way the burden of proof is much lower um than when you do an actual uh intervention um assessment of intervention whether it is efficacious or not um because the needs assessment is helpful in identifying the Gap and the scope of it so that you can prioritize um and do the intervention targeted in in those aspects oh that's a great question and even just the terminology in the needs assessment as you know one of the preliminary ways to study this is new to me since this is all kind of a novel approach for all of us in our program um so we did actually do needs assessments without realizing you know what we were doing we were just seeing where people are starting and so a lot of it was surveys with you know feel uh with zero to five zero being not confident at all and five being very confident for example and had the residents scale a lot their understanding of various ddim topics or diversity just varies the inclusion of medicine topics based on those scales so we started with those sounds great um I think we have time if anybody else wants to just summarize what they discussed otherwise we could just continue with the closing remarks and if there are any q a we can address those I'll pause for a second and see if anybody wants to chime in Jessica and others I I really uh want to add additional encouragement behind what circuit has said and um Arthur olendorf has said in the um in the chat just focusing on there is so much out there if you're just focused on gme in the literature there's a whole world of other things out there within other aspects of medicine even within the nursing World nursing is way ahead of us in lots of areas of education and education research so um looking into different areas uh Beyond gme could be very helpful in finding some measurement tools definitely that's that's a great thought Sherry thanks our group this is Sherry Hartman I kind of um steered our group down the path of looking at um a research question around the benefit of interprofessional Education um and uh I got helpful input on making it more realistic um so you know to which is is always a challenge at you know how do you actually get to study your question but um the the focus of this interprofessional training would be to educate both the rehabilitation field physical rehabilitation as well as let's say addiction treatment field was what we narrowed it down to um because of an identified uh Gap in providing adequate services for persons with addiction in the field of Rehabilitation persons being excluded from rehab programs um by virtue of having a history of addiction for example and then vice versa treatment programs addiction treatment programs not using best practices to make their services their addiction treatment services accessible to persons with the physical you know disability um so this is a a real problem that's been identified the possible solution I always love to lift up education is a possible solution for that kind of problem and so that the idea is that if there were interprofessional education in these fields that we would be doing better in providing services in both domains then you get to the issue of then you know measurement and you know where you're going I think uh after hearing this discussion you know we're we're likely also to be measuring needs assessment you know and not jumping right to outcome measures but um there was somebody in our group who pointed out you know of course your ultimate outcome measures would be increased accessibility you know uh in those two Fields um those who participated in the education did you in fact have uh increased inclusion of the populations that we're talking about yeah no that's that's great too um thank you for sharing um and now we can continue with um thank you everybody by by the way uh for for your active participation and discussion that seems like um people were talking about um very interesting um gaps or problems and I hope this will help you think to the um steps how to turn it into scholarly activity and in closing I just wanted to uh emphasize the importance of real educational need make sure that what you're starting with is an educational in nature and that's a real need um so again needs assessment as we tossed around that term a few times already is usually very important to be able to identify the scope and understand the scope of the need as well as how to prioritize it and research question as as we learned about several tools that can help us formulate our research question and that's a sound and researchable researchable question then we know that it will drive the rest of the methods so that we have pretty sound study design because we have a good research question to begin with and oftentimes we when we look at the rejected papers people do 45 minutes of lecture and expect the worlds of outcome from it and so the intervention needs to be appropriate in duration and scope make sure that whatever you're trying to change that you provide a strong intervention to actually make a change in that and it's authentic within a realistic setting that the participants are actually going to practice this in their work environments it's not some irrelevant thing that is kind of removed from the real life situation and it's learner focused and also the the last but one of the most important points is to moving beyond self-report as we discussed earlier it's important to look through the children identify published measurement instruments if you can't find anything then um it's upon us to create one but make sure that we do the work to go through that psychometric assessment of the the measure so that the data that we gather is um is reasonable and it's that the conclusions that we draw from those are relevant and and meaningful um for practice um so and again medical education scholarship can come in several different formats it doesn't all have to be assessing the efficacy of an educational intervention you could also publish a novel curriculum if you have a really great idea that addresses a significant relevant interesting topic then you can publish the description of the novel approach um but the the challenge is to find the right journal for that not all journals have the that option but if you find the right Journal you can publish your novel curriculum with with you know accompanying resources and everything else so that others may have similar situation in their own context they may be able to take and replicate and use it in their own context and just by definition of scholarship making it public is one of the important steps of turning teaching into scholarship you can provide a perspective on relevant and timely issue I've seen many many prospective pieces in in journals these days and if it's really relevant and timely you can write about that and publish as a perspective or that this review is very important if the topic is is is is such that there's a lot of literature on that you could do a systematic review or if it's an up and coming kind of a new field then you can do a scoping review to kind of scope the literature the depth and breadth of the the literature around that topic to inform to synthesize what has been done so that it can inform our future work in in practice as well as in in research and um again we discussed several times assessment is a very very important aspect of doing educational research because the constructs we deal with not easily addressed using objective means so we um we end up creating sometimes a very you know novel way to assess certain constructs so one can write about that and publish on that as well so anyways I just wanted to mention that there are many different ways you can publish your teaching Beyond just assessing the efficacy of an intervention so this concludes my presentation and I will be happy to take any questions at this point thank you so much Dr Troy this has been wonderful um and I just want to encourage folks anytime I talk about education um scholarship I always uh want to emphasize that education scholarship is a team sport so while it could seem overwhelming at first we have expertise throughout our organization in all different areas of Education scholarship I know um the lit reviews aren't my favorite I do like reading the literature but it is a little bit of a process we've got some fabulous Librarians to help with that we also have lots of folks who are practicing lit reviews and are really skilled at doing so there are folks who are skilled at assessment in all different areas so don't feel like you have to take this on yourself if you are interested in doing some getting your feet wet and doing some education scholarship so does anybody else have any other questions for Dr Troy and if you come up with any questions after this I'm happy to you know take your questions through email as well I'll be I'll be available if you want to email me and ask any questions or if you want to have a brief meeting to think through your question just brainstorm with me I'll be happy to do that as well and let us know if you're looking for other to work with we saw we sometimes have a broader perspective on who's doing what throughout the organization um or who is super skilled they're looking for a new project so let us know through teach we are happy to guide all right excellent session I hope you all thank you all for having such uh engagement throughout the workshop and I hope you all have a wonderful rest of the day.