UME to GME Transitions: How Can We Improve the Process?

Thank you so much for joining us for today's Health Professions educator session we welcome you to add your comments and your questions in the chat function throughout the presentation you'll also notice that we'll be adding we may have already added Deb usually does this to the CME Link in the chat for you presenting for us today we have Dr Arthur olendorf who is the designated institutional official at curling Clinic he is also the associate Dean for graduate medical education and a professor with both the Department of Health Systems and implementation science as well as the department of Obstetrics and Gynecology with the Virginia Tech carillion school of medicine I'll now turn our time over to you Dr olendorf right thanks Sarah and welcome everybody and welcome from the Westin in Richmond where I currently am at our perinatal quality collaborative meeting I'm supposed to be giving a talk at the same exact time but uh we were able to switch it around for us but um I'm going to go do the most difficult thing I hope of this is switching over to so you can see slides and the slides are really there more to kind of keep us on you know kind of be like a level setting here so let's make sure this works okay um I really what I want to talk today is about the Umi the gme transition and how do we improve the process and you know yes you could argue that it's a very timely talk because match day was Friday um I would argue that while some people would say this is the beginning of the transition I would argue the transition happens the first day they come into the into our schools and hopefully during the course of the time or hopefully it's not going to take the whole time we'll have lots of time for discussion and and learning what people think of what's going on um we'll know a little bit more about that so you already know who I am so I have I don't have any Financial disclosures I wish I did um I am the current president of the association of professors and Gynecology and obstetrics or APCO and a lot of this work is Grant work that's actually Grant funded to APCO that I'm part of so that's a bit of a disclosure I'm also the coach of the vnpc but that has nothing to do with this talk per se although I will talk about maternity deserts because I can't not talk about those in my roles and obstetrician and obviously with any kind of talk I mean I'm biased by my experiences and things that I have seen and I will try to minimize that bias as best I can um you know there's definitely some science here I'll share the science and when it's a personal opinion I will tell you it's a personal opinion um and the truth is probably somewhere in between so if there are three things we can get out of this and I think two you know some of them are priority clear to many of you one is to acknowledge why the you and me to gme transition is important um and it's just good to go over the basics sometimes it's it's obvious on some some levels but a little bit tricky in others um and I'm going to talk a little bit about what OB GYN a lot about OB GYN just because I'm more hip deep in that but what some other Specialists are trying to do to ease that transition then hopefully you know in your role in what you do if you can identify one thing you can do to help that transition for our medical students or other medical students who work for that'd be awesome that might be my aspirational goal of this of this session and there are a lot of opportunities No Doubt so this is a slide that I saw when I gave a similar talk at this at our APCO annual meeting um just remembering the journey to becoming a physician and it's a long journey and you know I've been thinking about this a little bit it's my 30th med school reunion um coming in October it was my anniversary my 30th match uh on Friday I matched on St Patrick's Day in 1993. um and this makes you realize like you know college to med school then you know you go to residency maybe go to fellowship and maybe you go into practice and back into Fellowship um there are a lot of different Pathways that people can take and this road to becoming physician is actually not unlike a road for me as a perinatal substance use person the road is not always straight they're bumps they're blips their changes um I'm still always amazed at 95 96 of residents pick the right specially pick the right place and they finish in the program that they start so we actually in some ways we do a really good job of having people find where they want to be and most people are happy where they are but I think it's always going to reflect that this is you know they're they're definitely break points um and you know transitioning to residency is another break point in this kind of Journey to becoming a practicing physician and the other issue is like the gme the ume transition is actually really important to me because one that's a clearly identifiable Landmark right you are you're entering a specialty you're typically leaving where you train so there's actually a change in place as well as a change of focus um most people choose their Medical Specialties in that time although some don't some people do transitional years and pre-lum years and trying to figure it out but I think for me as someone who does a lot of work in perinatal quality and as I you know look at you know what our role is as Educators you know where you go on the type of residency program you pick really influences where you practice and I think that's why you know I'll show you an article right now um you know Eric humbo and Andy Bazemore people I've worked with before they talk about the social contract of medical education and it's a real thing um why is it a social contract number one is a lot of money spent on it you know so CMS and Medicare spent over 11 billion dollars in g e in 2019 um I also think you know medicine is a vocation I think it's a calling I think it's very difficult and I think we're struggling with that right now where you know I've my conversations as a Dio right now are really involving how do we take graduate medical education from a transactional role during covid you're covering the shift you know we have these needs back to the more of that you know educational and having those interactions with with Learners um I think the other thing is really important is one part about being a profession is being allowed to self-monitor and I always rather have Physicians monitor themselves and have someone else do it for us it's how I kind of felt with duty hours I'd much rather have the ACG me come up with duty hours than have a federal law and having special agent Jones coming to say hey so and so has gone over their duty hours like they did in New York back in the day when they had duty hours it was enforced by State Police and that's probably not the way we want to do things so I think if we can really be on front of it and really live to our Professional Standards of self-monitoring and realizing you know who knows best what to do you know I think Physicians but also informed by the public and informed by other people so there are three questions about the ume gme transition that I have and I don't say I have answers to it although I'm developing answers over time and maybe we can answer them together or have more questions together um but the first question I have is our graduating medical student prepared for their roles of residence and if not why not we've had four years you know they come to medical school 99 of our medical students go into residency um we're not we shouldn't be surprised they're doing residency what can we be doing better are there things that we can be doing curricularly are there other things that we can be doing also what are the effect do our curricula have on choices specialty in practice location um I I mean not so confidentially the app go World we're really talking about where's our Focus as we have the impact on medical students and 95 percent of which don't go into OB GYN so what are we doing earlier in the pre-clerkship years the pre-clinical phase whatever you choose to call it that really will help us understand what people know not just about OB GYN but any specialty if you like you know I was really excited to see someone at magically going into PM R and I kind of wanted to ask them how did you learn about PMR when did you get that opportunity it's a great specialty um but you know where is that exposure and how did you know how did that person get that you know that bug and decide to pursue it and then how we were advising our students are we actually advising the way that meets their long-term goals and the needs of our communities um and that's a big broad lofty discussion but I think it's a really important one as we really look at the really key way we affect Workforce and the way care is delivered um so I'll just talk about location um and we know location is is a big part of it um you know the data has been very clear how location really affects practice location when you're really digging that data it's really really interesting um so Michigan is an example you know if you do your residency in Michigan your likelihood ratio is 5.3 that you will stay there after you done so you know if I'm a legislator I'm like so if I get people to stay in Michigan to train they're like going to be or as a physician well not surprising they chose to be in Michigan probably because they liked it then they made their connections in Michigan or fill in any other state this is pretty reproducible it's true in South Carolina like you can see but it's also true for disciplines so in Family Medicine they know that you know if you trade in Family Medicine in a given State you tend to stay in that state almost three quarters at the time so it is a really big driver so where your train really does matter um it's what you're exposed to it's what you do and that really has pathway effects so um where does your practice setting matter well rural primary care or something that's been looked at for lots of people um if you went to a rural High School not surprisingly you were probably four and a half times more likely um to practice that rural area and if you go and you're eight and a half more times likely if you train in Primary Care um people that have done pathway programs you know be it for you know underrepresented medicine or rural or Urban um we recognize that recruiting and supporting those who are going through that process is critically important and you're going to lose people along the way people like you know I want to be you know my wife had a decision to do primary care or do PMR it's actually what you wanted to do well she was in the military the military says no we don't need PMR you're doing family medicine and and she loved it and it was great not everyone has that sort of decision forced upon them but people make different decisions I was going to be a neurologist when I went to med school I am clearly not a neurologist I'm an obstetrician um but also how you train them where they go where they spend time um and this all comes a lot from Randy Longnecker who did all his work at Ohio University now actually lives in the region and I think we're gonna talk to him about his experience in rural training and how to get people um and what curricular elements can you have that really gets people to go into rural practice or primary care and they can live together um and I think mayhex is a great example that was my previous job I was the Dio at mayhec before coming here and then in the class of 2020 we had 26 graduating residents between Family Medicine OB dental and a couple of fellowships 12 of those 26 went into rural practice and 10 into hipster regions but at the same time we had three people going to Fellowship you know 110 FM and two into addiction medicine so once again it's not either or you can do both you can have a rural Mission as well as having people who want to do fellowship and work in urban areas it's just how you develop your curriculum and Asheville is a bit easy because that was our you know that was our goal at a at a state funded ahec um and we also were able to recruit people who wanted to to do that there are definitely ways of doing it because I've seen it um and it's real it exists so as an OB leader putting on my APCO hat on so if if where you train especially in residency is really important location well here's a problem we have an OB there's a big swath of the country that has no OB GYN training programs there is not our OB GYN residency in Idaho Montana South Dakota North Dakota or Wyoming so they're going to have a different tactic of trying to retain a recruit as compared to other parts but not surprisingly these are places that have very low numbers of OB gyns um and to check on the egg phenomenon there are a lot of things that go into that especially post Dobbs but once again it's where we're training and you know this is a place where the whammy program is working in the medical student side if you don't know the University of Washington's whammy program where they selectively take people from Montana Idaho and other places to help build up that Workforce and get people into medicine but we don't have residency training programs it makes it really hard especially in OB they have residencies in other disciplines but they're much smaller in number than other states so for me as an OB GYN this really relates and I'm sitting at our perinatal quality meeting it's about maternity care deserts and OB if we don't train people in regions of care they're not going to be doctors to take care of them and there are some you know you can look at you know it's hard to see this map because it's all together but there's a lot of red in the state of Virginia especially in our part of the state of Virginia um in other and some of these Maps look not inconsistent the map I showed you before of where they're training so once again in OB it's very clear that you know we have deserts we have pockets and training programs might be able to help that if we have the right training it can either be in medical school too but residency would be another great way of doing it being a rural tracker another way of doing it so a little bit of break before you go to the next session I just found this really funny because I've worn my white coat three times I'm not a I was never a big white coat wearer um because I tend to get dirty especially in OB GYN world but yeah this is the the Journey of your white coat um as you go through training um and I know right now we have a lot of fourth year students who are probably donating their white coats and uh you know good for them because they're going to the next stage they get the longer white coat um which you know definitely an honor but uh part of that ume gme transition No Doubt so here are a couple of sort of provocative questions about what I think are the problems that you immediate gme transition and some of it's like I said data supported sometimes it's actually not particularly data supported um so number one is the application process dominates the bulk of the clinical phase of medical school um and and if you were in any part of the conversation we were having at BTC Som about how can we maybe compress and have a three-year medical school program we realize you know part of it is you know they have to have a guaranteed spot for residency that gives you about three months just off the bat and there's no doubt there's the time and effort the cost and the stress um you know the stress release that we saw on Friday is evident of the stress of the process and you know I worked with one student this year who's a friend of my daughter who didn't mention OB she's at another Medical School the stress of her trying to figure out I still want to be nobody doing how am I going to do that and hopefully we'll find a path for her but the process alone not only is it stressful it'd always be that way it's also just kind of crazy and I'll show you some data in a second the other thing is applicants and programs and leaders I would say place a lot of value higher how high or how low you go on your rank list and you know it's sort of a weird metric because I know when I applied I didn't apply to that many programs if I got any other programs I was going to be pretty happy at them and if I got my top one or two or three yeah you're always happy when you get your top choice but what's your difference between your third program and your seventh program I'd argue very little so we stressed out a lot about you know where the best program is when there's lots of great programs for you are you looking at them are you finding them do you know where they are um and I know programs having been a pro previous program director you know the distinctions we make about students and what we call highly qualified or subtle at best and I challenge any program director and I do that to the program directors at carillion go back to ranked lists of years past and see how they wound up and sometimes your top-ranked people who are not your best people and sometimes the lower ranked people are because once again we're trying to predict quality based on really odd metrics and I think we're also probably as good as NFL quarterback picking you know people that are highly drafted in the NFL don't become the best quarterback and then you get the Tom Brady who is the seventh you know round draft pick and you know he turned out okay um and then the other thing that's really concerning is that with the vast majority of our students going into residency when you ask them when we've asked OB GYN incoming residents do you feel unprepared when they start residency and one-thirds say yes so to one of every three across you know not not it's not a VTC issue this is a every Medical School issue um you know why why do one-third of our residents don't feel prepared as they should when we have the opportunity to do so where can we do better because it's already stressful enough having that greater Independence how do we mitigate that maybe never eliminate it but maybe mitigate it so these are things that were that OB and other disciplines are trying to look at if you're not familiar with Simon sinek's Golden Circle we actually talked about it today at vnpc one of my colleagues brought it up it's actually one of my favorite things to talk about um it really talks about success both as organizations and as leaders and organizations and leaders that can talk about the why are very much more successful those talk about how and what right and if you talk about you know the famous example that Simon uses is you know you work at McDonald's you talk to someone who works in McDonald's like you know what do you do how do you do it like and I take the fries I put them in the oven and you know I put them in the oil why do you do it they can't give you a compelling why you talk to someone an apple they can tell you how and what they do they also tell you why they do it and their why is probably they want to create amazing product that you don't even know you need yet like they have a true Vision you know education is very easy for me you know I know what I do every day I help train future Physicians and future Workforce that's really important to me and I do that because you know who's going to take care of me who's going to take care of my daughter as an OB GYN that's really important to me so knowing the why is really really important as we move forward and I think you know the conversation we have in the OB GYN world is you know what's the why well the Why is the program but I think the Y is actually the students and the applicant the programs will always get good people it's the student that gets through this once maybe twice so as we look at this whole um gme transition I really would you know yes the programs are important I'm a Dio I think about programs all the time but I really think about the individuals um and how they're going through the process and how well we can make something that's very difficult make it a smidge easier so here's some data and I wish I had the most updated data for you but this is from my hamoud one of my colleagues who does a lot of work on this and this was looking at oh no this is OB applications I mean looking at the number of applications per applicant um and they're it's ridiculous the average OB applicant applying to 87 programs kind of crazy um does that help any of us do our work probably not our is this multifactorial absolutely so and you can see the growth in the change by you know if you're osteopathic graduate or if not um so taking two for osteopathic graduates you know 60 you know 67 for USMD graduates and overall it's 72. you can look at Psychiatry that number's 80 for all that for all comers so the average applicant of Psychiatry residents applying to 80 programs which is a lot um unsustainable and there are lots of people looking at different ways of doing it and we'll talk about some of those this is the rate of available pgy1 positions per applicant in the main match and this is looking at U.S seniors versus all active applicants so if you are a U.S Senior you know it runs around one and a half if you're an applicant um the chance of getting a space is a lot lower um so you know where you go to school matters um and right now if you put you know listen to Tom nasca who's the head of the acgme speak the largest single growth in people entering residency programs or osteopathic graduates right so that's going to put pressure on the marketplace as we've increased medical schools both in numbers and in size of classes but have not changed residency programs but for OB um it has an effect and then OB GYN another way of looking at it I don't have the number for 2023 yet is the match rate or the unmatched rate um and the unmatch of people who applied and as a MD USMD applicant 16 of those who applied did not match into OB it's 33 if you're an osteopathic graduate at 64 if you're an international medical graduate I was in 21 in 2022 you can see a similar stage so there are people who want to enter the field of OB GYN for example in a true for orthopedic surgery or neurologic surgery any other competitive field who are unable to imagine their chosen specialty and that's difficult to tell somebody who wants to be a certain specialty and because of the way things work it doesn't once again this drives the culture if I want to be an OB GYN and if I'm a U.S medical graduate I know that 19 of the time I may not match I'm going to apply to a lot of programs because to me that is how I increase my odds and then this is the metric that gets shown all the time and I would I would argue this is not a very meaningful metric but it's out of a recent New England Journal article um you know how people you know how often are they getting their first choice program or their fourth choice or lower programs once again I think again to any program you're happy with that's a good thing but this is something that definitely drives Market you know and I have as a Dio I've been told by a lot of my program directors I got you know I got people in my top ten like okay tell me about number 10. uh tell me about that person all right I went down to 112 of my ranked list okay tell me about the 112th person they may be a great person who cares it's a metric that I don't understand um you know you fill your program with good people that's what you really want to do but we get very caught up in you know to get my top three or you know I get four through seven you know I'm a little questioning if that's a really important metric or not but people use it a lot so I bring it up as an example but not one that I really love so I'm going to talk a lot about what OBGYN is trying to do to make the process better and this is part of something called the RR Grant right residency right program ready day one so it's looking at about the application Phase but also how we are preparing them when their first day as a resident in OB GYN and it is in ama Innovations Grant that's given to APCO and Korea Korea is the residency arm of OB GYN World there are a bunch of teams I'm on the leadership team at the president of APCO we have a learner Advisory Group that's huge everything we do in this grant we run by our learner Advisory Group because they're the ones informing us because we're actually doing this all for them in the end and we have different work group teams I'm going to talk about those work group teams and their names are changing so I'm going to give you their old names they may look different um next month as we're changing some names incredibly data driven group um publishing a ton 33 Publications since 2019 um we're about halfway done with this project and we're and I think the thing I'm most excited about and I am an implementation person by Nature we're already talking about what has to transition from Innovation to practice there are things that we have studied we know work it needs to leave the grant World which is innovation and go into the normal day-to-day activities which is the practice world and we're talking about that even before the Grant's done like we're transitioning things actively to our main organization to say we know this works let's make sure it can sustain because the sustainability of this work is really important and some of those become more clear when I show you the data so this is a more graphical representation you know we're looking at our different groups we have a standards group we have something called the arm or the application review metrics group a group called ACI the alignment check index and then we have this very large group called e-wrap that used to be the early result acceptance program we were kind of hoping to have an early match that sort of fell through with the nrmp so now program signaling falls into that there's another group that's really looking at the Readiness side like so once people are going into OB GYN how do we make sure that they are ready day one um so we're talking about some curriculum and assessments some learning communities and a lot of coaching a lot of it is developing coaching teams both as a national cohort a regional cohort and also at a program level talk a little bit about that because that's really gonna really come to fruition the last couple years of this grant program so one is the standards and I I don't think OB is the first through the standards I think we're OBS probably one of the first of the larger programs with the standards you know ENT and Urology and some of the smaller residencies have always had this but really we wanted to have a common calendar of events we wanted to make sure that if we all talk about we're going to offer application interviews on a certain date yes the residents are still nervous but they know to be nervous they're not nervous over a pain of weeks months or days they're nervous over a few days and it gets better um we were hoping it was going to limit the competition for releasing interview dates earlier because you know a lot of this is how do you review an application in a holistic way but also offer interviews in a way that's fair um and what we found you know this is we've done this for four years now in OB GYN that developing standards at over 90 of programs voluntarily partake in has had a High um satisfaction among residency programs and Incredibly High satisfaction among the applicants um you know yes it's still nerve-wracking it's still imperfect some people still hoard interviews but it happens at a much lower rate than what we can tell from the data sources that we have um and I think everyone just feels better about it so this is something we realize that has to go into you know this is no longer Innovation like other people other programs have copied us um other disciplines have copied OB and done this and this is going to start becoming a regular thing that's probably going to live in the world of Korea which is our program director world they're going to set the standards and they're going to get people to continue um you know using them and we're going to widely advertise them to our applicants into our clerkship directors and our Deans so they know what's going on but we realize this is like a really good thing that was low wish hanging fruit but it takes you know it takes a lot of working together in leadership amongst the discipline or especially to get this done so we were able to do that we're super excited about it and it's going to continue and once again we'll probably leave the grant world and go into the creative world which is our day-to-day you know normal residency application mode we have another group and I'm part of this group called arm the alternative review metrics and we're looking at holistic review and how do we identify what are the things that are necessary to do holistic review once again we were incredibly lucky to have step one scores go away um because that really pushed this a lot like how are we really going to evaluate applicants um so we came up and we have a paper that came out about this is like you know what is what's what are the models what can we be looking at what attributes are we really looking for what experiences are we looking for and then how do they align with what applicants are looking for and what programs desire so a lot of us is naming it um is a work group of about 85 program directors and Learners that really help us with this together um and we came up with some you know not surprising type of things you know location matters what type of training did they get you know especially we talk a lot about abortion training or Global Health are we inas are you in a state where you can actually do an abortion and actually experience that or experience counseling or you're gonna have to go somewhere else practice setting is actually a big deal like we talked about where you wind up doing residency we can dictate what type of practice setting some of these are obvious you want to be in a rural area you may want to go to a place where they can train you for Rural or there's experience in sending people to rural you know what are the top what are the what are the issues around um Equity diversity and inclusion um distance traveled we talk a lot about that you know is that something that program directors really value but how do we get that into an application how do we really have a great way of displaying these characters as we really want and professional goals so this was the beginning of a discussion about these are important things how do we get them into an application how do I do holistic review um and how do we have applicants and programs share that information um in a way that we can really match people up in the application side not so much in the final match side um the other thing that came out of this group is coming up with a standardized level of evaluation or the slow we call the slow not the slow e it's slow like slow gen that's how you can remember that um so you know Copa the Coalition provision accountability talked about having structured evaluations and being really important we thought it could be really helpful in mitigating some of the inequities that we see in the evaluation process it really supports holistic review if you do it correctly and once again OB is not first of this one emergency medicine been doing this since 1997 they've gone through many iterations they now have an electronic slow they call it the e-slow or the E slow e if you want to call it that but they call it the e-slow um and they all their letters are that way you know B we started ours in 2021 we mod we've modified it a couple of times along the way and right now one is suggested for each applicant and I think for next application cycle so the 2024 cycle we're going to say one is required although we have no way to enforce that but we've had such good uptake of our slow um and we keep kind of tweaking it based on feedback it's becoming a really helpful one and realizing this is just one of your three other letters you'll have traditional letters but I would you know if you look at traditional letters you know what we've just you know we've had a lot of great conversations about slow with nrmp people with you know I'm preparing this that live serve lead um later on um next year and what we realize is standard letters of evaluation really talk about the letter writer more than the applicant if you read letters enough um and it's how well they write the letter not actually the true attributes of the applicant so the slow helps with that so we add some information about the writer you know we want to know where they're from and what experience they've had with the applicant and we made a criteria based you know we don't want to say they're they're in the top 10 you know top 10 of what right top to ten percent of legal airmen's experience of teaching for 25 years or top 10 in one of my junior Partners who you know been out of residency for three years so we really look at you know you know what is you know what criteria we are the criteria we looked at we kind of talked about beginning intern are they at the beginning intern level now that's still a little bit vague but I think as we're getting better at it we're actually defining that a little bit better and and we're finding out a way to make sure that has a sort of a standard understanding about what that means across all the people that fill us out we also asked for the applicant's most outstanding feature in less than five words which is actually really fun to read and then the thing that's really been hard to uptake for us is three areas of focus that this individual would need to help develop and nobody wants to fill it out they think it's a you know some sort of like a Scarlet Letter like well we all have things to improve upon we just want to understand as you're a lifelong learner about things I need to work on but it's that's been a harder one to um to get done fully um and a lot of people leave it out we still have written comments but we limited 250 words and there's some people just want to put their letter into that space and you know they can do it but we realize that most programmed records don't look at those written comments we have a writing guide so we had a writing guide that was a made-up Learners program directors um img's osteopathic leadership um a couple of dios I mean we're really trying to understand how are you writing this are you writing this as an individual are you writing it as a group composite evaluation is it the shares letter how you can represent that so then the reader knows where it's coming from and how it's assessed what's it based on like when did you work with the students when they were a third year student when they were an acting intern was in the lab just once again we understand you know the people that are reading it the program directors and the program faculty understand where it's coming from um and the writing guide has been a big Boon for us and people have given us really good feedback about it and we're doing actually some more I would say rigorous but we're doing some more surveying to get a sense of where to to improve upon this I think we have some ideas from a lot of um basically focus groups that we've done across the country foreign we have something called the ACI or the alignment check index and it's really a tool for applicants to seek what they're looking for a program and program to try to articulate what they're looking for in an applicant now this sounds really simple but the difficulty thing is the programs are looking for lots of different things but they're definitely trends of what is important for some programs versus others or things that we really don't find important or don't value quite as highly so we have these six groupings right here and we had each each program director with a 100 point scale you know 100 points weight these different ones to see you know what do you find more important or what do you really value and that really helps applicants do this and they go to the Freda system which is run by the AMA and double AMC you put in what you're looking for and it'll give you a bunch of programs um that might match your desires and needs so we look at academic metrics work experience community service research experience teaching experience um and background lived experience journey travel distance traveled rather and once again it's a tool that applicants is really for applicants to say hey these are never even thought about because I never realized that they have similar you know goals and things that I want in a program we haven't this is just did it for the first time this application Cycles we haven't evaluated it yet so we're very curious to see how it was used in the application process more to come on that but once again it's aligning what those those metrics who are that the programs feel are important and then try to match up with the applicants and what they're trying to figure out where they want to go um so program signaling not new to OB GYN um we're a little bit different um actually sorry for those who don't know program signaling it is the opportunity for the applicant at the time of their application to say if they prefer your program based on what they know of your program they have that interview their program necessarily or they have not interviewed with your program they may not know anything other than what to say on their website but this ability of them to say hey these are you're you're a person I prefer many per many disciplines have done this differently some some disciplines have as few as three programs three signals rather orthopedic surgery is 30. OBGYN has 18 we have three gold and five uh actually three gold in 15 silver sorry I don't know why I reverse that um and we finally got to see how that affected applications and interviews we don't have match data yet we're going to start looking at some match data and I've heard some you know interesting anecdotal the match on Friday of what's happened um but what we've learned is actually really fascinating this is from the double AMC preliminary data they're letting us share it um but if you are an out of state applicant so you know you know this include your own medical students okay so the the the graph on the left is in-state applicants and we know geography matters um but if you're an out-of-state applicant and you do not signal a program the chance of you getting an interview is five percent so that is actually very important to realize because if I have if I'm an applicant I'm going to use my signals wisely because you're never going to get into a program you don't interview at and if you don't signal them you know once again an OB GYN which has a lot more applicants than there are spaces it actually does make a difference so to me as someone who advises medical students you know when you apply do all your homework on the front end know which programs you really like know which programs you may be competitive for really come up with a good list of programs realizing that you may get an app you make an interview without signaling them but you probably won't statistically in-state might be a little bit differently because a lot of that was you know in our first year of rollout we told students to Signal your home programs if you're interested but most programs that don't bother signaling we're going to interview on you interview you anyway so that that's to take out some of the background noise but signaling seems to matter in OB when you have 18 signals more to come what happened on the match side um but once again when this shared mental model comes out next year if I'm an applicant next year I'm on a signal if I'm interested in a program I want to go somewhere I'm going to be really good with my signals it may not be for aspirational programs as much as it is for programs you're truly a good match for but more to come on that and then lastly as we kind of run you know kind of go to home and see what questions and what discussions we can have um you know the next part of this grant for the OB world is really the coaching and Readiness um so we're really looking at you know some of the Knowledge and Skills that are necessarily for day one of residency so we do have some self-directed learning we have this really interested interesting um text-based way of doing questions every day for people who sign up for it um that can lead to good discussion um a lot of group coaching and instruction about growth mindset and resilience and then really looking at you know before you start residency and as you start residency what kind of one-on-one coaching can you start at the beginning um and sort of my aspirational dream not part of the grant project but wouldn't it be great if just like in the acgme world when someone transfers from one residency to another you have to give them the Milestones the Milestones assessment of where they're going would be great at medical schools did that and not the letter of evaluation and not the dean's letter which is the aspirational view of things but a true assessment of the strengths and weaknesses so I know day one hey you have a lot of great strengths you have something you want to work on let's work on those in the beginning let's not wait till they show up clinically if we know they already exist so once again we're trying to be more proactive and you know a lot of us is self-assessment um but if we can figure out areas that we know can improve immediately and do one-on-one coaching our hope is that that'll help improve that transition to residency we don't have beta on this quite yet other than some feasibility issues but I'm excited to hear what's going to come of this over the next couple of years as we kind of wind down this Grant and where it goes from there from a sustainability the problem is the one-on-one coaching is very time intensive that we already know from our studies and getting people who are willing to give their time on the faculty side to do that with all the other pressures they have on them is proving a little daunting but I think it's overcome by the value it might have for the applicant or the then the then resident doing a whole lot better in their program more to come on this but we're excited to see where it goes and I think this really fits in now this is I found this slide that Tom NASKA gave you know back when I was a program director of the mid-2000s and this is actually not surprising right we know that students and I'll say students that happen to this point um in this Continuum of practice well different skills will develop differently and they and you know mask on the ACG May recognize this that you're not going to develop all skills at the same time and I would say as a Residency program director when I was a program director I'm sure as a Dio I struggle more with the professionalism issues of our applicants and our in our residents rather than I do with their medical knowledge right so you know are we really working in the professionalism are we really working on systems-based practice and patient care aspects really fundamental skills and for me this really kind of goes down to something I was thinking about when Jed Gonzalo has us in our work groups and we're talking about you know he asked the members as one work group what other really six important skills and the skills that I came up with were humility reflection and curiosity right those are things that I find are incredibly successful and those are the things that are you know are we are we looking for those things are we developing those things and people who may you know you know have good humility and good curiosity maybe not aren't self-reflected and I started thinking about the skills that we needed and I was just thinking you know wow if I think about things that I learned in medical school many of which are no longer applicable because the science has changed but if we change people in screening brief intervention referral to treatment yes I'm an addiction person that's a skill that's helpful in so many ways that'll cross Specialties if you understand trauma-informed care that's going to be a really important skill that crosses disciplines if you talk about motivational interviewing and how to evaluate the medical literature which is where I started into into education and doing evidence-based practice and making people systems thinkers or even better system citizens those are really things that are going to really no matter what they go into it's going to something I believe will affect their entire career um and when I think about system citizenry and people have heard my talk about this in this group before you know system citizenry is an identity an identity is based on skill and assessment but there's a systems thinking skill that we can give medical students that they can then continue to develop as residents and as practicing Physicians like that'd be amazing right that's really how we change care and how we make things better and that's the opportunity that we have in medical school education and ume that's also important in gme which is also important when you go into practice and that's a whole other level of discussion of how we prepare for practice because I have a lot of people right now people that work in gme right now your graduates are asking you the craziest questions my UB residents ask me simple questions in clinic how do we treat vaginitis again it's like you know this but they get nervous like they realize I'm gonna have greater Independence and that's in a program in OB where we give our Chiefs a lot of Independence we understand that but they still have this doubt but as if you're in a program that doesn't do that you know really interesting to think about so you know for me when I talk about I'm a systems thinker and I'm proud of it you know when I think about systems thinking in workarounds you know if we can redesign ume to better meet the needs of what practicing Physicians need and we can really go to completely based assessments to replace mspes and Lors wouldn't it be great here's your company-based assessment that's continuously getting developed and when you're applying here's your update this is where you're at rather than coming up with this artificial letter of recommendation everyone's amazing everybody is amazing but work we know where are their opportunities if we had more gme positions it'd be really help and I think more in gme positions in the right areas and that's why I like being a Dio because I get to be part of those conversations and figure out what communities need and try to use education as a way to improve the Healthcare of communities which I innately believe is true and then the things that are workarounds are application caps yeah you can have you can cap applications people would argue that 18 signals is effectively going to cap applications but that's really a workaround that's a way to sort of decrease number of applications because we have another problem in the system how do we fix that system and program signaling I think is helpful but one thing I think it's a workaround I think you know if we're doing a better job of people thinking about which programs they really want to be at and we get away from you how to get to your top three choices we change culture I think some of these things that are Innovative will probably seem workarounds maybe not so much systems changes but for now they're actually you know application caps no one has the guts to do no one wants the restrain to trade lawsuit against them but program signaling you know you know if the OB data continues to be true um kind of becomes a de facto cap um because you're not going to interview you're going to stop applying so so what's on the horizon for those that don't live in the um egme world quite as much um Geographic signaling is starting next season the double AMC feels that you know we know geography is the biggest motivator for people and where they go to residency so they want to have both specialty signaling like I talked about node B but also Geographic signaling more to come on that is rolling out um soon most people don't know that it's coming and we actually don't know how well programs are going to look at it so it's a double-edged sword holistic review of applications will continue but how to do that best and how do we have to change the application process to get that right now it's supplementary applications that seems like work around to me to change the application why make a supplementary application so lots of discussions and a lot of fields about that OB uh definitely one of those um I would say reevaluating medical school curricula and I think you know right now Jed is going through that great process not just for the um egme transition but looking at what do we need to do to make you know the best practicing physician knowing that's where most of our graduates wind up going um and someone's going to talk about the application caps but once again they need a really good attorney in employment law and constitutional rights so with that I will show you my dog staring at my backyard and why I love living in Roanoke um and definitely any conversations discussions or questions I'm happy to answer and I will stop sharing as you see Murray so thank you so much Dr olendorf we do have a couple of comments in the chat and I'll relay those Dr Schmidt Dalton mentioned or filled in some information uh regarding the learner who went into PMR um and folks can read that comment there but I am just in summary um she said that we have an intro to PMR session in the final M2 Block in CS as well so just to provide some additional information there Dr mistra mentioned we have to enable a culture of curiosity for our learners and it also looks like Heather was able to pull up documents pertaining to many of the things that you mentioned in your session yeah I don't know how to Foster curiosity other than you know develop the right culture right I think that's a lot of its culture and you know being allowed to ask questions and you know I like being challenged you know I'm okay with that probably because I'm where I am at my career I probably didn't love it quite as much when I was first in practice but I think that's how you you know curiosity is where all questions can be asked because you're doing much best for the patient and what's best for the patient you know it comes down to my favorite definition of evidence-based medicine it is the best clinical evidence through the experience of the provider that meets the uh values of the patient so they're all important not one of them is most important um you know my favorite example you know is is in OB you know you have someone coming into the ER that's an email going to three from bleeding fibroids they're unstable yeah you give them blood but they're a Jehovah's Witness they refuse blood okay what's Plan B you know once again that's the curiosity what else can we be doing what else meets their values what else can we be doing we're not saying oh we're done here we're moving on no we're going to take care of this woman um so yeah that Curiosity I think is a culture um you know and you know when you have it and how do you create it I will I am not an expert in that but I think um that's where we want to get for sure yeah I think we are doing that I think we are enabling our Learners to ask questions and we're enabling our Learners to question our practices including all the systems that are in place I think what's important is sometimes we hear the same questions over and over again and you know you go okay we've dealt with this and that culture that we have to build has to start with us um we have to encourage people to question whether this is the right thing whether the system is acting right and so that's what um I liked about all the research that's going on especially in the OB guiding program I mean it's uh it's actually very not only heartwarming but uh also inspiring and we need to make that culture available to all of medicine not just particular aspects of it no thanks without feedback yeah I think you know we we've reflected like why is OBGYN been able to do it I think we're like the perfect size you know we're you know we have about 270 programs about 1500 residents a year um what you know this lots of Goodwill built up between the Umi the gme side the two new APCO which is the ume arm of our education site in creag the gme you know there's a lot of overlap and we've always recognized that and and I think we're able to impact it differently you know I think that's why smaller programs have been able to do that or smaller disciplines I've been able to do that you know internal medicine it's really difficult right I mean it's such a large number of people large number of programs not that it can't be done but I think it's a little bit different the approach we're going to take on the implementation Army if they want to go this way or general surgery or some other larger disciplines but um yeah I think we we feel Adobe we were fortunate because we kind of had the right environment um um and and there's a lot of trust in the field um no one everyone felt were doing this for a reason that was important I think maybe Dr sweet might have had a comment or a question a moment ago I will come off mute only to say no I was just on video no I don't say Witch Doctor sweet maybe maybe your life is true that was rather presumptuous of me no I don't actually know she's on the call but no worries I just I thought I saw you come off mute so I wanted to make sure you got your opportunity to to make a comment or ask the question if you do have a question please feel free to unmute yourself and ask your question or raise your hand if you're not able to mute yourself at the moment uh oh go ahead sorry Margaret that was so incredible I am so inspired by your field and what a joy you mentioned a system one of the students can go online and do this assessment ACI I didn't catch the name of that you said it was available and actually they someone was really great and put in the chat so it's through Frida which is um so yeah just go to that link so yeah um you may not be able to access it as a nonsense you might be able to I've never tried to but you can kind of see what it's all about and someone else actually put a link to the RR website which is where we kind of has all our information through appco.org and I'm and it says RR Grant on the so it's only for OB GYN residents it's only for OB GYN residents right now cool so I'd like to just Echo Dr excelsis comment that this was a fabulous presentation however I'm the dean and I write Dean's letters so you might not believe me because you know everyone is is better than they appear um but no I'm just seriously Arthur um I just want everyone to know that we're so fortunate to have you know two national leaders at The Cutting Edge of this um gme transition with Dr hillendorf and Gonzalo so we we don't have to just watch it you know we can we can hear about all these Innovations happening as they happen and and become part of this process of improving this transition so I just wanted to just mention how fortunate we are and although I heard pieces of this before I always learned something when I listen to Dr olendorf it was a pleasure to be with you today thanks thanks Dean lehrman I always I always think about things a little bit differently the more I talk about it um or trying to trying to explain this to my father who's an attorney is actually fascinating try to explain the process of the match to a non-physician they think you're crazy and maybe we are a little bit but um you know I would say law school is no better if you my daughter's a is a third year law student it's all about your your uh your LSAT score it's your LSAT score where you went to school there's no holistic review of anything when it comes to law school so um we are ahead of our attorneys and hearing nothing I'm looking at Sarah who can't tell I'm looking at her because it's Zoom yeah so uh just one more thing it looks like Dean lehrman did add a comment that I'll go ahead and let folks know about um he said for those who are curious about how well Virginia retains medical students and residents who train in Virginia your data published in 2020 by the Virginia Healthcare Workforce advisory Council of vhha you would be only in Virginia at 32.5 practice in Virginia gme only in Virginia 39.1 percent practice in Virginia both ume and gme combined in Virginia 64.3 practice in Virginia thank you Dean Lerman for sharing that data with us yeah I mean yeah that's really good data to have the only data that we have is over the last two three years we're attending about 22 to 23 of our resident graduates in carillion practices roughly about 40 in the state of the Commonwealth of Virginia um and kind of like a number in between there in Southwest Virginia not necessarily with a Carilion practice but another practice in the region and I think those are those are data points I'm used to looking at you know especially my other job when I work for an ahec we're like hey how many people you're how are you retaining and historically in in mayhack in Asheville you're retaining 60 percent of our graduates in the region um which was and I think that's going to start changing as Asheville gets bigger and more crowded but you know it can it can work especially in the early setting as you're looking at Healthcare Workforce all right if anyone else has any questions or comments please uh ask those at this time or add them to the chat Dr vid has a question yes hold on to your chair officer provocative question um I don't know the answer to the particular question you know are we training medical students to match are we training medical students to be residents and and I wonder if we shift the needle here if we get to a point where we can divorce medical schools from the outcome of the match we have two entities that are both in this for the same reason I.E the medical student wants the match should be obviously there's some visuals for medical schools here around the match and where do my people go and you know we have we have match day that everybody celebrates and we create slides about look where our students went and the like until we divorce those I wonder if we continue to train medical students to match as opposed to a training medical student to be a resident and I wonder how much we shift the needle when a lot of what we do is predicated on the outcome of this particular event and we rather look for a different surrogate marker of what what equals success for a medical school do you see what I mean yeah yeah I think you know I always say the vast majority of times are probably aligned right where the student wants to match his you know probably fits well I think it becomes more interesting when you know actually I'll use my wife as an example and you know my wife was AOA you know top of the medical school class I I think Super brilliant um and she was going to family medicine and the chair of medicine pulled her aside like you should do medpeeds why because it's medpeeds because it was at that point this is the 90s where Family Medicine wasn't as respected as I think it is now you know to him that was a metric like I can't have a top student in Family Medicine why not like that's a good thing so I think we have to change our expectations but once again I think yeah yes you know our students are very high Achievers they've been very good at achieving things that's how they got here um and I mean and like I think most of the time it's aligned you know I think you know I can talk about the six students going to OB I think they all manage the programs they really wanted to go to I'm really proud of the programs they went to I think it worked out really well um there are times where it does conflict right and there are times where you know especially in your in your ultra competitive you know residencies that happens but I think it's also about time for the residency programs to understand what's going on I mean I have conversations all the time with the unmatched student actually that from another school I should do a research here I'm like is that gonna make you a better OB GYN like you know what do you need to work on it's like probably my clinical side I'm like so do a preliminier that's what she's doing right I think it could be both but I think it you know I think the the holistic review will really help us understand what we really want and it's not going to be I had a 270 on my step two yeah you got 230 that's important you're smart enough do you have these other qualities what really drives it and once again it's always going to be competitive and I think you're right I don't have an answer like but I think it's it's a good question to ask it's really a great question Mike I do appreciate it too and I look at it this way um we bring Val there are a number of different perspectives on different sets of outcomes so you may you may be happy to hear that although we have all of the hoopla that every other school has with respect to match today that it's not on our strategic planning metrics at all the things that are on the metrics are how many state Korean Clinic how many stay in Virginia how we help provide a graduates that reflect the populations that receive care from them those are the things so we don't feel any attention between those they're just it's just for us a question of the different stakeholders um the outcomes that matter to them at different times and but you're raising an important point because if we're teaching to the test let's back it up a bit you know teaching to the step one which has become step two or teaching to the match outcome you know that's not what any medical school should be trying to do even though they may celebrate their successes so thank you for bringing that up yeah and and in a very fortunate position I find myself in a fortunate position because and this is no clearly not pointing fingers in any one medical school um you know we we get about 3 000 applications to residency and so we get a wide look at the way different medical schools you know approach things and application packets and early after you've been through a number of these packets you realize it's rinse and repeat um and if you look at what we're putting on there and it's being important uh that's not always important it doesn't speak that holistic review and obviously every student is the best student you've ever worked with but yet when you look at the objective data I suspect you're not the best you you've ever worked with but but that's a different discussion I it just seems like we're we're fixated on fixated an important component of this is the match which clearly is important and I'm with Arthur on this one whether you match with your first or your tense if you're in a place that you happy that that should be good and that would take away some of this tension that's built up in this match packet that is all about the best and you will end up at this institution which I think puts more pressure on the students there's something I call Michael you'll enjoy this I hope the illusion of the single best choice something I made up and you could fill in the blank it applies to I okay let's back up it applies to health careers it applies to Specialties within medicine it applies to programs for residency training it may even apply to areas of practice where you'd like to practice the illusion that there is a Best Choice instead of being arranged of outstanding choices and we struggle to get past a random decision I think it's really hard that's a good way to end it I think it's really hard so I agree with Dean lehrman yes thank you everyone for your wonderful comments um Dr olendorf for a fantastic presentation it was spectacular and we look forward to seeing you all next time if there's anything we can do to support your learning needs please reach out to our team we'll be more than happy to assist and support you everyone have a great day thanks everybody.