Integrating Health System Science Education into a Medical School Curriculum

Welcome so how many of you were at one session yesterday one second two sessions yesterday for you're going to the trifecta today so this is wonderful awesome so this is their really that's perfect if you're here for one session that's fine too but these sessions have built upon one another and dr. Gonzalo will go over some things that you may have missed in previous sessions through discussions but this is as I mentioned in the previous two sessions this is our first Dean's forum session with our new dean and it's taking a little bit of a different format we are going to focus on both system science for at least the next two years and then these forum sessions so we know that faculty development is a large part of implementing something like health system science that you are medicals pulling into our health system so fear not we will get plenty of faculty develop we're working on so without further ado I'm going to turn it over to Dean Lehrman did you do stuff it's out so good morning everyone thank you for being here on a beautiful fall I mean though it's your day and here we are for the last in our trip check our trifecta of thoughts by dr. Gonzalo you've earned his various introductions and you seek his titles there I can't think of anyone who's more qualified to get us starting to think about systems science how it relates to the other things we've been learning along the way about systems based practice and other parts of our history and medical education and that first topic is wonderful please take the grid if you haven't got one already and the second talk helping us understand with the experience or learning component with media Pro System science but we have a patient navigator role the Sun definitely looking also some others hearing lots of other examples of how to not like with students and to experiences that matter that actually add value to caring for patients and and today I think there's going to be very helpful for us because the more we learn about implementation anything new and any change on the scale of Health list of science and some of the schools that have done it it's so bold and so so new that it's actually not changing one thing is changing various aspects of the which we've already done here but it's important to us to learn about change processes as they have occurred in health system science and to really understand what implementation has looked like what challenges have to face what what the Tiffany's have recurred what how that how allah say instructions how a lung speedbumps have been navigated along the way and so for those of us that are going to be moving forward with starting to plan for implementation this talk would be very very very helpful for us so but a further do dark gonzalo we look forward to hearing your comments so I was skeptical that people would show up at 7:00 a.m. as we had a 7:00 a.m. meeting at Penn State people would have quit football at 7:30 is Beverly so this is a pretty full room I wasn't fighting - you're right you're right I give you credit so I think Dean Luhrmann's comments about the elimination piece these three sessions ideally are building on each other my biggest worry walking in the room was do a scan of the audience who are the familiar faces and I can kind of get a sense on who hasn't been at the other - I'm just trying to make sure I bridge the gap a little bit with with what has been taught the basic pieces from yesterday into today but even if you haven't it's perfectly finite I did a check in with dr. Wofford right off or buffered so he's up to speed all right anyway which is which is really good so this session really is is about the challenges of Health System science curriculum in a medical school I'm not only looking at the Penn State experience but across what many medical schools are going through in their struggles it's not a big narrative what I do what I have here in the slide set is really a set of different issues accreditation of evaluation program evaluation student challenges some of the educator pieces how you integrate it but I really want this to be about you about what your concerns are in some of the challenges and I don't mind fast forwarding to the other slides to go to that section to talk about but yesterday's new talk was a narrative not so much for this one I think this is for us to be together with our sleeves rolled up about what can you anticipate in phase one and two maybe three of the bill that sound okay [Music] so here are some of the objectives identify the hijab challenges involved with integrating HSS because we have a lot of learning over the past seven years actually it's gone past gone back several decades it's just taken on a different name and HSS comes in 2015 but you look at different folk fossae of HSS the challenges have been in the literature for longer than that articulate the concept of the implementation deputy curricular innovations I wish I knew this six years ago so I'm glad I'm able to put it on your radar now many of you probably know this concept describe HSS areas within accreditation standards for graduation questionnaire and board examination blueprints and articulate the issues that need to be considered in a systematic strategy this really merits dozens of hours and we don't have all the answers and it's a continual improvement improvement cycle I think we're trying to hit the the big pieces to be thinking about on your journey so I think one of the biggest challenges which I try to adjust yesterday is the idea of language so I'm going to ask to those is we have 95 percent of people we're at the sessions yesterday so what's your reflections on the HSS grid this grid that some of you have it in front of you but this grid as it compares and contrasts with some of the systems areas that have been in place in the literature and curricula general reflections on the language it's the challenges you might have this energy I believe it would work really well anybody well tells us the science that is is it's big and it's scary it is president the third pillar actually is a remarkable inside story but 18 different names we couldn't agree on 91 and there is still some fragmentation and in the dialogue about it healthcare delivery science health system science but there's some but one of the push backs I get is well what's where's the science in this and then the response we wrote part of a paper just focus on that rebuttal and it's well there's been a few signs around the bird for many years they've been improvement science in bed Public Health Sciences so it's a it's an amalgam of different science or the and you can debate whether national science most people are committed but I agree it is scary it goes away after controversial we wish for a way and figure out how we can all speak to somebody this is not just this project a lot of education it seems like people are talking about very similar things we're using very different words and some it's very confusing just how work in assessment very excite they you know the sesamoid of anguish and all these different firms and people use sometimes interchangeably in according to every big Sahar yeah and there's a lot of history to it as well I guess today I gave a timeline on the Patient Safety Qi movement that came in the 90s and again since it's early and I'm getting on a plane in five hours there's a lot of identity the people who grew up at a patient safety Qi frame was part of their role maybe they were the director of patient safety for 25 years and now Health System science comes in and you look at a grid and patient safe is in the value domain and there's Health System improvement that's two of the twelve domains they say all we've been doing this for 20 years I say yeah you've been doing a great job you helped found a field this is an extension of that where's the policy pieces where's the social determinants of health whereas the other ten domains it's not saying you didn't do a great job but there's tension that exists in that identity which is tricky to navigate sometimes so depends partly on culture I don't know I haven't got a full grasp on the culture here about you know it's like human factors you have a human factors lense and it's pretty clear it's on doors in the sim Center we don't use benefactors at Penn State we use operational excellence so there's a language into which you got to try to figure out how to assimilate with this frame so increase live in your experience when you use the component things in the framework slide there and when you use the big bang consultation because that cool framework and you rarely use it or used all the time as well as you support so if you're a medical student applying to Penn State you'll see on our website we have a four pillar model we use it there basic science clinical science health humanities Health System science give heat that's the name of the courses this is so interesting science of health systems is the name of our person second-year course that even my role is associate dean for health systems education partly because the course name and the and my title came about before we found at the field so there's just there's not any congruence there but I would say when you're talking concept in medical school vision we're using the concept in the classroom we tread lightly you want too much concept for the learner we're trying to really show them how it applies what's the concept for the day cost value healthcare policy other reflections thoughts concerns so key takeaway from this figure twelve domains so there's the girl we talked about what most medical schools are doing on these two dots they look on interprofessional education transitions of care which is in a process piece quality of cost which is really in the patient safety bar patient safety is probably the most prominent that's existed in med schools for the past twenty years that's why this Carell lights up I mean cost is everything that every there's a lot of gaps in cross each pillar and I showed this yesterday these are repeat slides from yesterday but just to hit the point that HSS is not cue IHSS is not value you got to think strategically about the pill or how it fits into the larger curricular framework and it dovetails into what we're going to go into about the evaluation pieces that in my view you can't have popcorn but what I noticed that Penn State when we started the work is that okay there's an IEP II event over here and all this these topics they kind of lived they were orphaned topics they lived in different spots they weren't so he said they didn't have the same course co-director and then you sit down you talk to me what's actually going on in that course and you look at signal 40% of that we're doing in a different course no wonder why the the GQ or the or the curriculum feedback from the students is there's there's repetition there's duplicity in what we're doing because we haven't cohesively brought it all together which is one reason why I use that framework to be strategic when I talked to Eric humbo about this at the HD ACGME we talked about the how fast things are going and what I said yeah you know things are moving pretty fast and health system science he said you know I agree he said but we've been pushing for 25 years for this type of transformation and evolution of med ed and I would argue it's going pretty darn slow and it really stopped me in my tracks because that's right the movements been pushing 25 plus years in human on one and the traction sometimes hard to grasp and you know what he's right if we believe this is about moving patient outcomes we need to move forward quickly so another reason why I like coming on these trips or even trying to articulate the framework because we got to be strategic and be comprehensive bring the assessments and evaluations together and approach this in a systematic structural way to get more quick results and we took the popcorn out of the task force and we have a lot of spare you know some research and and some basic science the water neutral profession you're experienced how we did that popcorn put it back or simply should we not worry about that structurally and just cite and it was in your first block of block one whatever that is here's what the students are doing and it could be in any of those domains but it's part of the overall systems thinking and in the professional and practice is that a property that we that we put it somewhere structurally call it that or collect it exciting because that's going to be important for what we have I don't have the best answer for that I think those the two thought the key to mine is number one which is what I've seen happen over six years is those who have multiple no primary role to they end up struggling because it's so part many people Emmons talking about this conceptual models when you have five different people who are scattered and the mental models are not totally overlapping it's really hard to strategize and put it in place comprehensive way so yeah I'm leaving I would probably identify your primary who is the go-to people get them up to speed as quickly as possible in a model and allow them to pull the bag together I make suggestions to work with the team but who's the hoosier go go to I guess my structural questions we don't have to pull it out of the research domain and college this week we leave it there and included yeah this is my second this is my second clip what we're learning too and there's no evidence behind us but we asked medical schools for example health system improvement or operational excellence very few places are doing that they you do trial and error and you realize okay you might be able to do primers in the second year maybe a little bit in the fourth but these twelve domains it mean you can write a paper about this and try to put evidence behind it but how do you map these twelve domains from the development progression of the of the medical student or the GME trainee by the way because our work is not done when they leave here it can't continue on but what do you start with how do you go into Phase two how do you go into Phase three and they got a build we don't quite know yet but in our mind it's social determinants of health and patients health care system transitioning and the improvement sciences that go along that continuum but there's no evidence for that a much simpler example is and earlier in my career lots of papers about it it was both things part of the air you breathe at the president many play but there was also this progression so it certain ways of training on it throughout the student address curriculum we didn't try to stop that work from and they had part of the culture part the air he breathed and was about it so I I can see how both women [Music] because they're totally of these experiential holes I know it's totally on your radar that's the other piece I think about we talked yesterday about force and the experiential world roles work together so if you're targeting a spot in the year to these patient navigator roles or whatever you build how does that synergize we talked about continuity of the curriculum has that synergize with the curriculum so yeah it might live Rick in that in that research of course I don't know what the competency is but it should be juxta in my mind should be juxtapose as closely as you can developmentally with the experiential role like working in synergy that's it does another challenge any other thoughts on there or I get to the so I never anticipated this going into my job I mean I started this role in 20 start my job in 2012 and took the role 2013 and it was a struggle it was a major struggle early on we are getting pretty low student evaluations remarkably low five-point Likert scale is 2.9 in that first year I mean we started a course in the first year of course in the second year and of course in the fourth year ran all the same year which was probably announce to our education Dinah's that go getter so says go but the score started coming in and I had I'll be totally Frank I had the impostor syndrome not to do really good job and master's in fellowship idea I went through a little bit of a crisis until I heard that Dartmouth was doing this type of course word since 2004 2005 great over is who is the former education D out there and I I just I never had a book writer seventeen I was taken without any phone was looking for a therapy phonecall you know I gotta tell you about this is what's happening and his response was congratulations we've been doing this since 2004 and we have never had scores maybe it is not me I'm sure enough when you start to talk to medical schools about the challenges and you build a little bit of trust it never happens in the discussion 1 it happens on discussion 5 & 6 they say you know what we're struggling to what is it tell you about that and what I have found is that every school is struggling in this if their dose is big enough they're struggling with the challenges and Greg his follow-up comments to me was it's like the broccoli a medical education and we had a couple conversations so we wrote that paper that just got published a couple months ago about the challenges it's not palatable in the beginning but we believe it's part of our mission as part of Karelian x' work to care for the community same thing with Penn State we this is when he came in at 7 a.m. today to work on this but it may not be palatable so how do you navigate that ship miss Dulles we see we're doing yeah I said yesterday you should be proud of what your School of Medicine has done it has paved the way clearly there's more work to be done but you were still up that early cusp don't don't think you're in the 40th percentile of HSS you're not in the first percentile but this is remarkable work that's already been done here that is going to help synergize and catalyze your moving forward so I'm just very briefly one of the small meetings about the pedagogical challenge this is an educational concept that comes from defeat and the example that they given the literature about it is if you are going to design a German history course your pedagogical challenge the one that you need to address early and accommodate the learners in her success in a German history courses most people are part but there's so much more to German history so if you don't address that somewhere strategically throwing the development of your course you may struggle so we did this study before this large launch it was qualitative work talking to students particularly those who are leaving our doors before their curriculum was in place what's our pedagogical challenge for health system science this is 2014 and this is what the learners said on the left side there is this current medical student priorities gene survey the base to be political science courses that gets us our grades and board exams which gets us into the best residency program our basic and clinical science course is one of the students that is our golden ticket into our residence that's our current priorities but what you are giving us is this alternative set of priorities on the right where there's the triple pillar of basic clinical and health system science which is what needs to be the best doctor possible this is why I came into medicine I believe in the right side but you are continually pushing me pushing us to the left ie our system is designed to embrace the left side she reflections on that commit our system is co-opted that outside the National Board Exam is co-opted at home I'm reminded of some innovative models in which pathways to residency can guarantee acceptances and so there's one inaudible school being late Long Island School has the NYU Langone it's free and they're also shortening medical school by the year primary care residency guaranteed mother's hospital in that way that a Lala's left-sided priorities and that significance started and enough stories to tell yet so that was pre curriculum and then this is what's happened over the past five or six years this is the concept of the implementation death in KY the citations at the bottom most of the work comes from Harold and Fedor but Carol bland is has done really great work on the evolution of curricula and what you should anticipate my our education Dean calls of the iron gun I was sitting down with them one of the first meetings we ever met she's I'm judging you and I'm you got my said Terry what is that like what do you really mean by that she goes there's gonna be rough times ahead I mean you just got to have an iron gut you gotta be able to take it and keep going and she still uses it to this day and sure enough she's right that's why there was one or two leaders I was talking about somebody who you were supporting they need an iron Doug too to Shepherd it through so the concept here is that this is any course any curriculum they even part of the framing is in college undergraduate courses but the changes that you could introduce wherever that course is and this performance that you have whatever metric are using on the performance of the course or the curriculum takes a hit you think this is the myth you think we're going to build this great course point of the play that's going to be amazing we're gonna be rock stars in this and you're gonna elevate right from your baseline all the way up and you're gonna get to hear but let's what happens is you take this large dip and it takes time for your growth in whatever your performance is to get into some level and it never reaches the myth but you get to a decent point and there's a gap in there the expected performance gap they use this in college courses this is where primarily where it comes from but you're going to get there generally agree with that good you get that past the LCME yeah which part it's a dip all I have that what's your concern with the LCME well you know it's kind of like board grades and expectations you're judged by your LCME report and when things dip that's a challenge and you've got to be able to address that so how do you address consequence because we had our out seaman site visit in February so we start the curriculum at the end of 2013 the LCME is coming to February 2018 which means all the days of 2017 I eat right in the meat of this dip we are fine because how does it manifest to get to the radar of the ELCA site bankers is it coming through the GQ questionnaire and then forming that data is it coming through the Student Survey and there was one or two items but they they were so focused on the basic and clinical sciences and chewing that part up that we had no issue I mean Terry and I were worried about it but we sailed quite I mean it was a remarkable things I'm not saying it's gonna happen for you in seven years but you're already for the wrong anyway you're good that's human perception like these five did there's a time within medical frame I mean I can see that you have priorities in their mind hasn't worn one Bethan was too and then after that you find that they're more perceptive of education because it's past the high-stakes eyes yeah so this is a I'm gonna go philosophical for a second for you to think about because I think some people would say I've had schools say to me yeah we're gonna do everything strategically at the end we're gonna do it right before they go in there right before they go in their internships and we're gonna load it up and make sure they got all this pocket of skills and knowledge and peace and the other side of the camp what most people I talked to leading this work in medical school say that this is a professional development piece and we need early enculturation I'm in education so that it becomes part of their integrated thinking process from day one so when you wherever you sit on that camp that dries part of the answer to this so the answer clearly is yes learners right now we're in January we've been in the course for five or six months for the first years they're going through this cognitive dissonance I thought the healthcare system was this or teaching me this this is a struggle by the way I have an examination so it is a struggle you don't do four hours of out of class time each week we just can't go that high so there's compromises that exist here but I would say generally yes it's a fine wine fourth year when they're leaving the doors they are thanking us that they had it in place it doesn't feel good when you're living through the first year stuff so I think the answer is yes you gotta have the iron gun that would be but feel free what are your thoughts the other thing that we mentioned earlier is helpful that it can be a differentiator and they applied for residency in addition to the sports getting inserted for interviews once they did they were too close to the deal entirely how they make an impression you mentioned earlier that the means letters in state the paragraph over you mentioned their project the curriculum at context seems to be a very effective tool programs excited in Gotham differentiator apartment students much more so than I have a what about my students come back here like about any of this validation analysis and science that's it and people say I had no idea I'm sorry I'm sorry I was I was negative about John everybody not until that thickness you could bind this man on top of in our research Oh what what poor folk I'd agree the different everyone knows differentiation it's a differentiator until everybody starts to do it and then it kind of drips down I think one of the questions I'm picking up by my visit here since you're such a new school I asked I'm not an expert on this differentiate or the difference between brand and differentiator what is the Virginia Tech School of Medicine brand and differentiator your brand is quite different than your differentiator and I'm not sure where it is yet because I think you're you're so young I still think you're fit takes years and years to build a brand you got a strong crown but what is it and I think you're learning triggering that now but it could be both it could be both in the long run so this is what we've gotta start on go ahead after that if you look at the 12 different domains underneath the how do you it's to address which dummy more but more how you rule it out in which the [Music] yeah that's I was alluding to before we don't know one who really knows me if I have a publicising integration some medical schools take the approach ever we're not good under stand alone course and they not be able to stand in one of course they they identify thread leaders they go to their PBL sections they go to their basic science sessions and them they'll ensure that it's hardwired into those sessions and at the other end of the spectrum is what you're hearing from me at Penn State it's blocks plus the integration even though that's more difficult it's probably easier to do blogs so I think that's the the first frame but then you're talking about developmentally over the time span and I think that's first alarm there's some that that need to wait to the workplace reduce the foundation building the workplace and then it should be buried on a cemetery that's also part depends your thoughts on that no I was just process for application development we get passed from those first troops step one for example so if we're gonna pass some of that and were able [Music] he's gonna go one of the board's the past family here but not out there how do we design curricula so that it might be faster so that's what I would be saying in your work group inning is that yeah we worry about the board do what's right I know you're saying the same thing but in my god but I mean they're top having a healthy dialogue about that so how does then what's the differentiate it has a program director look at a medical student from here or Penn State and say we really want this student without a board score just an interesting concept it to me it elevates a lot of this as a differentiator piece back to your point right given the other reason this is on your elements because I students come in with all different levels preexisting knowledge and sophistication some of them comes in with a strong interest in which certain parts this someone it's hard to figure out without sequencing you know it was a study in the early 1970s about medical students on board scores for basic science certainly we've been pushing all these basic sciences for decades as the prerequisites those who score really well on knowledge tests and basic sciences end up having low leadership attributes in the past five years everyone knows we're even increasingly be on file or diversifying what factors we look at for admissions so if the basic science course they are becoming a little bit less important maybe some of the other attributes are moving up but back to your point I think there's still diversity in basic sciences and biochemistry and all the stuff that we're doing outside assistance that would apply there too which means it makes it tricky if there's something in the back of my knee is we created great question consider the title each of these across the career if you know my understanding of that was bullets a spiral pretty low during learning it liberates us the first year that free step one deliberate foundational pieces that were carefully integrated apply that's that's what we focus on very real fun upset acidity my in my book success or failure of any group reinforcements in a couple years and so anomaly investing heavily and how we the infrastructure and domain J's all those things early experiences awesome because that's really important so I wouldn't see that the very front loaded and I wouldn't underestimate the value of these early experience experiences you're building for the clinical site that's why I was stressing that clinic clinical community of practice yesterday focus on the students educational experience but what keeps me up at night is the faculty development I'm sitting back here and then check your laptop a handful of position probably impatient state could you comment on the fabrication to ensure your students have enough yeah yeah that's why I said we needed 12 hours for them in fact some of the visiting trips I do are just on packaging there's a couple seconds you commented yesterday that they struggle everyone can imagine an implementation I'm sorry the diffusion of animation curve you have your early innovators those that three to five percent at the front who are your people that are totally behind this and they are out there right now you've got other end of the spectrum those extreme laggards who don't want anything to do with this they think this is a insult to the profession there's a chunk in the middle that are just kind of neutral I'm willing to listen hear you out so I think it needs a multi-pronged approach it's not a simple issue it's not a complicated shot in fact I think it's very complex it is deeply entrenched in what one's personal identity and journey has been for a year for four or five years as a junior faculty member so how do you tackle that I don't think there is an evidence-based way and we're doing we're trying to do professional development in year-long courses for the early adopters those who want to go into the site teach them per year let them go shine their light in their local environments and try to choose people from different clinical departments to go do it but then we try to get that middle the road to the larger masses which is I don't know what the advocacy is but we try to hit that part of the curve as well and we try to do the laggards which is more for one-on-one I know it's a hardly module but to have the sit-down conversations because I care less about the laggards actually being exemplary role models for the principles in as much as I care more about them not be railing the discussion which is a big difference like what Coalition is we don't have to agree on this but let's at least agree that there's a that there's a learning going on developmentally don't be rail with student because the minute you say X it could totally disenfranchised their learning and that's a tricky thing complicated complicated pop-tarts question is there but we look at all 12 domains it's there a little something from each domain somewhere in the four years like but like it's economics somewhere long wait is someone for finance come and talk to you soon at Penn State you know I'll punch it to the about ten rows from the bottom in the Mayo Clinic there's my clothes quality separately so I think what Penn State on here because I was the lead author and f1m muddy the water is here where I didn't put our professional development program on here either but if you look at their curriculum it's very may impending a similar curricular factor students got a master's degree in the science of health care delivery from Arizona State as part of American which is a remarkable thing I actually think that's the trend in that ad or Pokemon factor thing I'm working on our housing master's degree in Health System science that that grid showed you that there was some there's representation across that across that board to dot versus 1 dot isn't is it a different discussion this is sort of a comment about the faculty development for the brief question as well as the suit levels of achievement because in each case you can imagine sort of a pyramid let's start with doctor development in which there's a lot of offerings for all faculty as we like to learn about teaching more effectively than in and that perhaps teach was developed a specific program like it has for that presentation is now that would take the next people next level of deep learning and then there will be national courses that we can sit highest level of that pyramid perhaps there may be other levels to our masters by the program so that there we tiers of approaches based on Faculty's role and interests and then for students also I think we're we got research you're going students get graduation by fulfilling requirements people not developing the same thing for the health system science cop we can some masters programs under our own relative control in the TV agent program which does have a domain that focuses on population health and sanitation they're masters of Public Health degrees also so together through various levels of achievement based upon interest both my faculty and our students as we take a look at content here at framework at Penn State are we have a Woodward Center which is your teacher program that's partly where our professional development program grew from so I'm just highlighting that I think that's a really interesting thought that program was the prototype for the National Health System Science Academy which day that came to a December which is our second year so there are these tiers and I'll just I don't know this but being here for two days and knowing a little bit about yours School of Medicine there's something the systems dynamics at Virginia Tech I just I just wonder how can that apply could that be a differentiator could be could that be the base of some tiered program that is is elevating the learning for the learners around that that base of knowledge that you already have at your fingertips and I say that because at Penn State Hershey is an hour and a half away from University Park so I don't really have the I don't have a lottery sounds like you do but you don't I mean it's a big geographic to buy but you you can have some strengths here that are I think unique to you the chakram are we are absolutely focusing on the faculty development that's not something Malin are we recognized that that will which part of that can't convey to our learners a whole new program without knowing more about the programmers that's tighten it with the Paulette departments part of what we're trying to do is to get back up to be aware of quality care about it and you know once we identify those folks we will give this faculty development resources and help them to help our learners you know and eventually our murders will be the ones who are interested they're going to be ones with with the background in in health system sciences pervy to help the charge on you know I like the way these all fit in front of you I was sure that I don't know if you're at the talk yesterday at noon but you just that was a great summary of help system sciences on an education framework alone it is a framework that is unifying clinical operations education and potentially research and you just hit it on the head because it's not about med students alone it's about our workforce is about the health care delivery system that's where the idea of co-creation needs to come in and there's your surgery example yeah you're a surgeon is our great great point we were saying yesterday some students say oh this isn't for me I'm going to be a surgeon yeah our biggest representation of educators are the surgeons so the a lot of mental models you're competing with you can only have one person in time you do sir you know a lot of people Society come to Penn State I have like ten things together this is number two so we're gonna have to triage a little bit um this is what I would prepare for I know you're used to an IP but as you expand and ground if your dose grows that there's this HSS performance cap this line I think never really still the second lowest rate of course we're doing pretty darn well I mean you go through the sessions the learning that's happening it's it's going pretty darn good and there's a difference here between end of session and I'll just skip this point I mean there's a unique thing on how you do the evaluation that gets it's for brog for the sessions themselves and the session and of course there's a phenomenon that when you do it at the end of the session you get a higher score as compared at the end of the course if it's around soft skills if it's around the interim or these soft skills system skills and you're looking and of course two months later it's going to be much lower than right after the session itself it's hard to interpret that and I'm sorry does that make it kind of until you heard me say this [Music] reduce this whole concept in reporting to our speaker all four years they were so fascinated by the and so excited about it they were network was implemented and when we talked about how a professional that'd be our native space because they were like well yeah I can easily see keeping this part of it but not that other part and maybe really like like where this was daunted and thirty years on those to create electives for next year when they're for so everyone was sort of looking at ways to get some of this because they thought it was so relevant so yes there there certainly may be does this dip it'll be there are legs for us and all of that but I think it are these our orbital evidence uh players to be interested in these but as you said and as you said yes right okay tip is going to come from students not appreciating the relevance the death will come in what we do it's not the right it's not the relevant so I shared yesterday a five-point Likert scale it's not just Penn State multiple schools four point three to four point five four point two to five point five if you ask them about the relevance they see the importance and it's reflected in this is one of our most recent studies I turned all those negative comments evals like we evaluated all the evaluations over six years did a thematic analysis look at the results unpacked why students are not liking the course a lot of people do but we're trying to unpack the negatives and this is what we found and I'm just highlighting these big arrowheads this is where their preference was in the comments so this there's a two color approach on the left basic clinical sciences and then this three pillar HSS on the priority so the top one is they see this large arrow and HSS education is critical learning for all phases which is what Dean Linderman just said they will say this they will say I'm not surprised our leave endorse him we need this the whole way through the bottom three shift to the left in this to pillar moment this idea that there's a a traditional professional identity of a diagnostician therapeutic physician who does these biomechanical things they are many of them are shifted that way they do not like repetition if you do social determinants of health 101 and you spiral it later to take it to a more advanced level ah many of them are like we already did there it's like studying for the boards I already read that chapter don't try to spiral it some students struggle with that I'm a legacy from their college courses and how they're used to learning but but a challenge that we have particular time dedicated it's always competing and this is the last one the most fascinating two being probably one of the most important and also part about this race is this fact based learning and ability to grapple with ambiguity uncertainty where we put the learners in the system they are they got to your sense today in your school of medicine because of the left being able to master facts do really well on their board exams etc but healthcare in many ways Health System science is dealing with complexity and that challenging for them what did I just learn from this session about health care policy are the nuances for it or why the health system improvement initiative did you go and change the management it's not a crisp knowledge bit and sometimes they struggle with that okay so you want to talk about the GQ a program evaluation accreditation the board exams what would you like to talk about can I ask a question so I I was out of town but I missed you to lectures yesterday so consider me part of the uninitiated so I'm looking at the twelfth domains here and there are expansive there each end and then the subdomains each and every one not to itself is a huge area we're not new to this in medical education it's same thing happens with clinical and basic science we taste basically give people a taste of these things and then they decide what they like and they do a deep dive into one of the areas so we have people who move into quality or population health or informatics or work or what-have-you so I'm just wondering however you integrate this into the curriculum whether it's separate courses are taught amongst everything they how much content can you give up you know past this is important pay attention to this aspect when you're doing things I guess I'm asking what do you actually teach in those courses how do you segment and how do you get anything that's more than superficial in each of these areas boy that's there's four or five discussion points ss-sorry it may help to go to the if you go becomes an outline for the seminary exam that may be felony an answer to that in terms of because I I agree with you thinking just basically pro seminar as you know music exposure to all aspects of it it is a great question and so what the experts tell us Emily or the book nbme borders every okay ESP of the so we'll start here so the answer the first part though about they're going to get a taste and go on in those who want to do these different areas that's a conceptual point that I think people debate about because I think what the some of you could probably predict what I'm going to say we're developing system citizens this is core for in whatever we decide are the core competencies in EPA's its core for everybody it's not a taste of do you like this if you want to go learn more this is what we wrote in academic events and Brown put out an article and so they should be electives give them a taste they go and they they can be rock stars in this area and say if we wrote the counter saying no if you believe this is core for everybody then it should be quarter for everybody so I think even there is a conceptual dividing and it needs to be grappled with because people are going to be all over the spectrum on that and the content piece it's happening in the workplace one of the key pieces about I said yesterday assistance based practice and health system science is the most socially constructed competency domain and there exists because it's not really conducive to to brick and mortar rooms like this say but give just like I'm doing now ironically but let's give you information to learn there's some of that but it's really out in the workplace that might not be totally clear go ahead I agree with you being at national and state meetings with legislators or insurance companies and a lot of the positions are in the group for me around the country are lost in the process of system makes plumbing yeah as part of it because they never grew up with it a lot of times I've fun myself dominating the discussions with sponge somebody from CMS or legislative group with that because they don't understand how all the pieces are connected healthy economic impact the plate lengths back that occurs and a lot of times I'm the lone wolf and everything put on policy yeah every state I think that connecting your two points there's angst at some times should we chat all medical students leaving medical be able to contribute to that I think that caused me angst - yeah I mean it's that gets to my point yeah I think the reality of it is at least at first blush and again I want to know more but not everyone has skills Simone is I just look at those romantics if you're not attune to that you can understand yes everything I do is based on informatics but you're not the one who's gonna be writing the code totally agree you know so how much time and what content you devote in medical school just something that's going to be a lifelong important it really becomes a question no I totally agree what's missing right now in u.s. medical education around this is the articulated set of core competencies or EPA's development we just don't have it it doesn't exist we've worked on it for two and a half years the project is 60% done and it's not going to finish it is so uniquely complex but that you're highlighting a big gap and that's an opportunity to the part of that is also teaching students or residents think is an actual real-life examples because if you're talking about theory even lose yeah and like if you show examples of what's going on how you did I saw this problem and there's a lot of different ways of doing that at various levels I love that that may come up that time I love the energy and they say daunted but history tells us it's not impossible to do because we've done that replace excited over the last 15 years where you said we switch courses love interfacing science content $25 teaching renal physiology in the first year when we do probably says the recursive clinicians in the room and so if there's what ninety people what does emergency fun I would extend that back to the point of that's the learner how much time they spending in the or school of medicine material it's ten percent ninety percent it's the material that's happening in this ecosystem of medical education that is not even within our purview 10 to 90 I mean your students told us that yesterday in the small group meetings same numbers I'm getting at Penn State saying that was my asking in different schools so I mean when you really let that sink and what is the role of a medical school if we are holding true to this content knowledge transfer model of lectures this is a real challenge for - show the University of Vermont is ship they don't have lectures they this is their big tout it's the act of learning curriculum no more lectures there's something interesting with that so I know running short on time but the boards when we mentioned this this is I went through and I took the USMLE board examination all the content and just highlighted every systems more and then did a percentage so this is me doing it on the content outline and this I will be very lenient so these are the system's principles that exist thank you I patient safety policy and economics pop health but always in epidemiology plan and then patient endurance so 93% of their boards right now are for biomechanical 7% is HSS this is two years old the good news is that two months ago we're in November there was some announcements by us I mean that they're shifting this there's going to be an increase in proportion on the exams we push for them to have more Health System science or not quite using all the line groups yet which is perfectly fine but agency insistence based practice there's gonna be more for the students this coming year so I don't know what percentage you never know what percentage that are going getting in the questions but there's a shift occurring in the board exams themselves so I do use that in the classroom I do say one weeks when I start the second-year course I don't do it in the first year I show them this content grid and I show them the shifts and say look I know you're focused on the boards I don't know exactly what's in your first a book but this is a good chunk of your boards I bring students back and say yeah he's right because we had fish bones and and the reason the Swiss cheese model was in there you might want to pay attention to this because the QI patient safety is you're in rows in the short term all the other areas not quite yet but it's coming that's on there they thought how do you assess that in your it's courtesy self the knowledge piece or their performance so let me I'm just kind of highlight quickly the nvme board exam this has been out there for three years the ma partnered with the nbme to to bill the hilt system science exam there's fifteen or twenty schools using this examination started with four this is the content outline I think Dean Lerman you're referring to this back to your question this is the proportion of questions on this hundred item exam and at least it's showing some validity evidence that pre this is pre and post instruction so pre instruction post instruction across these fifteen med schools the scores are going up the take-home is that there's an nvme h/ss exam that has its first iteration won't point out what's going into the 2.0 version that we've been working on for a number of years that hopefully will begin to inform more of the board examinations and it's not quite there yet but there's a big movement into that direction [Music] you know the impossible different invention involved into the customized exam service we do paper questions for the markings in the Clinton it's headed toward the customers if there's been discussions and I'm not in all those discussions and we just don't were you book just came out a couple months ago - I wrote the review book for the for the knowledge multiple choice question format but I think back to your assessment question or I think this is the key piece what we've got and I we put so much weight in the HSS exam because it's the only thing that we have outside of student satisfaction scores so when the curriculum committee says check your scores are the lowest again now at least think of all the broccoli paper and say look there's a national phenomenon first of all second of all the numbers are going up third of all here's the HSS exam where they're scoring two standard deviations above the national means so how do you reckon let's have a discussion about that and boy that shifts the discussion pretty dark flavor but it's stillness noes appear if SBP IHSS is the most socially constructed confidence domain how do we need which gets back to the two individuals in the back developing faculty in our workplace 100% agree because we are living here right now currently workplace based assessments reassurances these principles god bless you some of its team-based teamwork stuff but not a lot of the other areas right bearing for scholarship right barrier for work and it's gonna be a long term I'm cognizant of time and I see Sheree standing yes thank you all this has been a great developmental progression across the three sessions if you happen to be one of the ones who have missed one or two of the previous sessions they will be reported they will be posted on the teach web site so feel free to view them there if you have any further questions doctor Gonzalo Wolfe hang out for a minute do you want to say we will send out a notice about it and how to access the web enjoy the rest of your day thank you so much for coming you.