MedEdMentor: A Groundbreaking Tool for Health Professions Education Scholarship

Please join me in welcoming Dr Stetson. thank you so much for for having me and um yeah this is hopefully to be um a fun casual conversation and I would love for you just stop me at any point love tangents uh I just want to you know address whatever is most relevant to you uh today so this is a um engaging and interesting talk for for all of us um without further Ado let's jump in and that is not what I wanted to share let's see share this cool all right look good let me know if it doesn't look good thanks JT all right uh you can see here the name of my partner Greg uh who I'll introduce now so this is me and Greg Greg is is he just finished Internal Medicine Residency at UCSF and we'll start on faculty there um in September he's taken a couple months off welld deserved um he and I met on his first day of medical school I was his assigned uh faculty Mentor uh so I got to watch him grow and Blossom during his four years as a medical student um and we discovered a shared love of Education as well as just building and creating things um which led to us uh creating this this tool I do need to mention that um medad Mentor is technically a company uh Greg and I are the co-owners we created a company mostly to put all of our intellectual property in one place things were getting a little hairy with our our two universities involved um so this just made everything more streamlined we've generated a grand total of zero dollar uh but I do re receive some grant funding to study the efficacy of the website through the the Macy foundation so we are gonna do a quick intro uh take a tour of the website and then we're going to try out some of these tools with a case from one of your colleagues Dr Lori wolf um and then free open discussion for any questions you all have and again stop me at any point if anything uh comes to mind so this all started uh because I was struggling um I was a biology major in undergrad and then became a clinician you know worked as a Internal Medicine resident and then and then uh attending so the way I approached problems and thought about problems was this this was science to me it was um you have a question and with the right tools um you can get to the right answer um but medical education is different and it took me a long time to realize that medical education most closely aligns with social sciences um and there isn't always a right answer in the social sciences um uh and what helps us to to create partial answers for Things Are theor iies theoretical Frameworks um it's a way to take these complex problems that we are dealing with when we're thinking about humans um and how we interact with one another you're bringing in emotions you're bringing in cognition you're bringing in History Society systems engineering all kinds of different things um that could be influencing what we do in our Health Care Systems um from education to to Patient Care to billing to all different things so it's just way way more complex than the things we're trying to answer in um the biomedical sciences and to answer part of it you need to uh look at it through a specific lens um and those are what theories and these Frameworks do for us is to allow us to take a complex situation and just narrow our scope to make it um to make it manageable for all of us um and I mentioned some Fields another thing that makes medical education incredibly complex is just how many fields are in this uh or how many disciplines are within this field we have education psychology um Neuroscience history economics um we could go on and on uh Lori is going to teach us about her field um human factors that's that's all a part of what we're what we're doing here in this field um so this all started because I struggled and when I finally got my head wrapped around this whole thing I wanted to make it easier for other people and I was relieved to find out that I was not alone um so this is a problem for many of us so this is a paper from 2010 and it states that changes in our field have been made in the absence of theory or at least by a poor understanding of theory so that was uh 14 years ago this is from 2011 and it says the explicit use of theory within articles published in the journal medical education has remained scant over these last 10 years that takes us back to 2000ish and this is from 2017 it says many academic faculty members are Educators yet few are familiar with key education theories that inform their practice so this is just a common ubiquitous problem that many of us face and that's okay um that is uh it's kind of a funny thing we do in our field where we um uh we train people to be clinicians and all the biomedical sciences and then we say hey go be a teacher um without giving them any tools um and that's what I'm trying to fix and when you extrapolate this problem that many of us face on an individual level to our field it becomes a problem for all of us so over the past uh three four years there's been a lot of papers looking at who is publishing in our field um and specifically uh the dispar AR ities that exist in who is publishing um here are some articles you can youall will get the slides um so you can look back at these and and read them if you like but just to illustrate this point I created this graph based on the data in the paper you can see at the bottom right and what we're looking at is the manuscript acceptance rate for one journal the journal medical education and on the left you can see which journals publish uh the most in terms of quantity in this journal so the United States publishes the most Follow by Canada the UK Australia and the Netherlands and the rest of the world is six on that list um and all of those countries listed above are either in the global North they're all uh highly resourced wealthy countries and most of them are English-speaking and then uh the graph part of this is the acceptance rate for papers so the United States has the worst acceptance rate at 12% um the Netherlands is almost 25% and then the rest of the world they're getting their papers accepted at a 5% rate so what uh Greg and I believe the Gap is is an unequal distribution of resources expertise and mentorship is the primary AR driver of the lack of diverse voices in our field and we created metad mentor to try to address that so um before we take a quick tour I want to um Orient you to how we are thinking about this tool so we're trying to think about what is it a mentor does for their mentee and trying to replicate that on the website so the first thing thing that a mentor does is just get their mentee oriented to the field what is medical education how does it work um these are all the things I struggled with was just getting oriented so we have um the lesson these lessons in a glossery to help with that and then once you know what theories are and what they're good for you need help finding them and we have um a theory database as well as an AI B powered tool that will suggest theories for you um we have the first and only medical education specific literature search which is really good right now and we're actually making it even better um and then we have an A Health Professions education specific chatbot medad Mentor AI That's built on top of chat GPT which we'll play a lot with today so with all that let's take a quick tour of the website and then using um Lor's case we can uh put it all into action so this is the site uh this is the site itself here um and right here where it says launch meded Mentor um it's a uh the URL is medad mentor.org um and if you have never been here right here you'll find a box for your email address um and that's all we need to to get you logged in um when you put in your email address it'll send you a magic link that magic link is active for 90 days and then at the end of 90 days you just put your email address in again and you get a new magic link for another 90 days and and so on um the website is free and we collect no other data and we don't use your email address for anything we don't sell it um we don't send you anything um we may start to do like a monthly newsletter but um that's a long way off so no spam no nothing um and then we see that we have 1500 members right now using it um but let's jump into the site itself and so when you land on the homepage you'll see some stuff here um that is uh selected for you but the all the tools are up here on the left and across the top so I said the first thing that a mentor does for you is they Orient you to the field so here are the lessons that can help you do that um I I've written many of them they're written in um uh non-academic language they're uh hopefully approach able um there's lots of examples and analogies to help that helped me to kind of wrap my head around all of this um and these are a lot of the things I struggled with what is theory what's it good for um what are the differences between these these categories um what's a paradigm um methodology and methods and then how to select and apply theories to your problems um so if you go into these they um kind of read like a blog post um and so I wrote this Greg edited it um and there's lots of links that'll take you to different places um and uh it reads a little more fun than your average paper so this is about the size of theories Grand theories macro theories and micro theories what can Theory do for you how does it help you um um and then lots of different uh bullet points and how to find theories a table on how to use them so hopefully these kind of things can get you oriented and then another thing I really struggled with was just the terminology in the field so here are some of the things that are most uh Troublesome for me um and if we click on them uh you know epistemology here are three different definition um from different people and uh their references down here um and uh related uh terms that can help you to understand these different things um so once you're oriented to the field we said the next thing many of us need is uh a theoretical framework something we can use to frame our question and our problem so that takes us to the theory database here you can see all of the theories that we have indexed um there's over 230 theories on this list um and that list is going to grow substantially in the near future um as we continue our work on the literature search um but when you click on these they give you a a nice summary of this Theory so cognitive load Theory um explains how working memory deals with different types of information while completing tasks um and then there's a little bit longer Theory breaking down the different types of load and then here are references where it is better described and then here are references where that theory has been applied to an actual problem so you can see how it might work in action and then here are um resources that are not Journal articles they are other kind of resources um what's also important is over here the similar theories so if cognitive load sounds close to what how you're thinking about your problem but not quite here are some suggestions of theories that might better suit your uh your problem and you can uh go back to the database and find these and read about them if we go back to the original Theory database we also have an ability I I said there's 230 theories um that's a lot of theories and how are you supposed to pick so there are you can filter um these theories uh by the scales I said there's um it goes from intrapersonal so a lot of these are cognitive theories things within an individual all the way to um societal theories um you can organize by Educators portfolio so things that um you could use for teaching and learning curriculum development leadership assessment and mentorship and Advising so different ways to to to sort through all of the theories that are available but the best way to sort through everything is this tool under our AI section called the theory suggestor and the theory suggestor uh we have created our own data base of over 50,000 medical education abstracts and when you type in the phenomenon you're trying to research here um it will uh reference that question against the those actual papers and suggest theories that are in the papers so this helps to um counteract that problem many of us have with AI of hallucination or making answers uh this tool can't make up answers because we've told it the only thing you're allowed to draw your answers from is this database of papers that we've created for it so we'll see more of this when we work on Lor's project but uh for fun we can say um let's see teaching medical students to perform oral presentation ations um and you just hit submit and here are you know 11 12 uh papers or theories that we could think about using to to build a curriculum around teaching students to do oral presentations um rhetorical Theory genre knowledge experiential learning deliberate practice and these are all theories that you can go back and read more about in the database um and this tells you how many papers within that database of 50,000 that reference these theories um and you can eventually you'll be able to click on this and it'll take you to those actual papers um we haven't built that part out yet but this is a huge way to narrow that 230 uh Theory database down into 11 12 um things that are much more manageable um and then the last thing is uh up here is the um is the literature search so this is a a custom database that we have created um that P pulls papers from journals that are uh dense in medical education and basically you're only searching those those journals um and it also uses uh what's called semantic search so we don't know exactly how a lot of big database searches work um in terms of their algorithm um for instance uh Google Scholar their algorithm is proprietary and they don't share it people have tried to um uh reverse engineer it and one thing we know for sure that Google Scholar values highly is Citation count so if a paper is highly cited it's popular it's going to rise to the top of your your search which which can be good um but it also if something is new and has never been cited it will be buried in a Google Scholar search whereas ours if your question matches up um really highly in terms of the words that you use and the words that end up in that paper it'll end up really high on on this search um and we'll see a lot more of how this works with with lorry's project um and then the last thing is this uh custom GPT as you can see this looks a lot like chat GPT 4 um and that's because it is chat GPT 4 we've just trained it to be a Health Professions education Mentor we've given it a lot of context it's given a lot of instructions so that when you show up here you can just start chatting with it um and it will give you guidance and you don't have to um prompted in any way so a lot of people talk about prompt engineering and you have to um tell the chatbot how to behave and who to be what kind of expertise they have we've done all of that for you and given it a lot of uh context um to pull on so this you should be able to just show up here here um and start uh chatting and you can get there through the website um by clicking here um but let's let's pause here see if there are any questions before we dive into Lor's case I don't think there's any uh questions in the chat yet but if anyone wants to mute or type something now feel free and while you take your time feel free to use the chat um I'll stop at any point but we'll dive into Lor's case um and if uh uh if something comes up we'll just stop and address it at the time so um here is the case and Lori if you uh are are able to show us your face that would be excellent um and Lori was generous enough to to share this project with me um and let me play with it to see how I how I would address this problem with with her um but in essence she told me that um healthc care is complex and when Adverse Events occur uh human actors engineering can be really helpful in trying to figure out the root cause of those things and her goal was to um Teach an interdisciplinary course about how to conduct a root cause analysis um was there anything else Lori you wanted to add about this uh this um project you're thinking about can you hear me yeah and see me yeah our our goal primarily was to make sure we had strong interventions um so as we go through this I don't know how much you want me to come in along the way but I noticed this is a big topic to teach and even though it we actually did this in a 4-Hour Workshop but we had to limit the scope because teaching all of it in that amount of time was was hard so we concentrated primarily on strengths of interventions so after you understand what your contributing factors are from an adverse event how do you make sure that you're coming up with solutions that will engineer out the problem and try to reduce the risk as much as possible of it ever happening again so that was primarily the focus so that's how we were able to get ours done in four hours I noticed your your chat GPT agenda and I compared your agenda to what my actual agenda was cool as we go through that okay cool um yeah I mean Lori is an expert in this has spent you know years uh diving into human factors engineering teaching this in a 4-Hour Workshop is going to be highly challenging but a worthy challenge so um the first thing I did was pull up medad Mentor Ai and what I wanted to do was um was try to summarize what Lori had given me she gave me a really detailed description of the background of this problem and what she was trying to achieve and this was the um kind of one sentence summary that that I got back um and this was helpful to put into different parts of the the website so the project aims to apply human factors engineering principles to healthc care safety teaching an interdisciplinary approach to root cause analysis and developing customized Sustainable Solutions for real world patient safety issues identified by the audience so I thought this did a great job identifying human factors engineering looking at safety it's got interdisciplinary in there root cause analysis and then real world problems that are coming from the audience um and that those I thought were the key pieces of of the description that that Lori gave me um and typically when I'm starting a project I always start with a literature search and I took that exact thing that I that summary project summary I gave you um which is not a typical search term uh people rarely will put in a whole sentence with commas and things into their uh searches but um what's cool about the meded mentor one of the cool things about the search function is it uses what's called natural language processing um which is something that a lot of these AI tools do uh but it can take the natural language we use um sentences paragraphs and break it apart into pieces um and it basically takes your sentence and makes its own Search terms from that so you don't have to be super super thoughtful about how you uh write things or phrase things or use those awesome tools and uh tips and tricks that librarians use you can just say whatever you want to say um and these were the results and I thought they were pretty spot on there were only 36 which is great I think if you're starting a project um and you just want to get your feet wet 36 is a really manageable number to work with um and right off the bat we got 12 tips for embedding human factors in healthc care education uh teaching root cause analysis using simulation um human factors in patient safety and Pharmacy uh creating safety through simulation interprofessional 60-minute root cause analysis a workshop to engage inter disciplinary clinicians and quality improvement and that's a metad portal thing so that will have um the whole curriculum available um so that is uh particularly useful um Google Scholar does not pull from meta portal I think um so that is one thing that our our um resource definitely includes um and then these last uh two of these last four were really interesting um just uh these two in the middle requiring Wellness implementation of a comprehensive Wellness curriculum and then enhancing uh evidence-based medicine skills um so these but then if you look at the summary it's got root cause analysis and root cause analysis and when I went and looked at these papers um uh the first page of these articles was the end of an article about root cause analysis it's one of those pages where um there's multiple articles on one page so the article right before these two was the one we actually care about um and have different titles so the the title got goofed up but um the these are actually really good root cause analysis articles um you just have to go in and fill one article back um and then I just W to oops I just want to show you I mentioned there 36 resources um or results these are the last three um so the least relevant are still nursing students learning about safety um undergraduate nursing students clinical reasoning and reflection for safety storytelling is a way to approach patient safety um so all 36 are relevant um any thoughts on these resources Lori as you as you see these have you seen any of them or know any of these people two out of your first three were authored by my uh major Professor for my PhD work so I thought that was interesting of the 12 tips of how to teach human factors nice yeah so sue hignett Dr Sue hignett was my adviser awesome so so thought okay I'm taught the right way yeah she's really good she's a great great human factors engineer do you know this uh vosper character I do not okay cool um so I also ran the same thing through Google Scholar uh we got 16,000 results um a little bit harder to parse through um but some good stuff um from Theory to real world integration Implement implementation science and Beyond human error and patient safety a book so our results will not bring you books this brought a book called patient safety um and uh some other there's a jmi human factors um Journal so it's pulling from that journal ours does not include uh the human factors Journal um and yeah so um relevant uh relevant stuff from from different journals um not education focused necessarily um so it's a great Google Scholar is an excellent excellent resource it's going to bring you different stuff um and you definitely want to use something like Google Scholar to round out your your search um but I almost always start with our resource because it's going to bring me you know the first five I'm going to find two or three papers that are highly highly relevant to what I care about um and that'll just get my feet wet get me started on the project um and then I'll come to something like Google Scholar to make sure I'm not missing anything some of these books are actually text books that you use along the way in getting human factors is either a master's or PhD uh prepared science and so some of these that you're listed off are Healthcare oriented factors textbooks right that's a lot if you're just trying to learn how to teach something exactly you probably wouldn't teach human factors if you weren't a human factors person and so you'd be familiar with all these textbooks as part of your education yeah perfect but in this case we're teaching how to use human factors as part of root cause analysis so that's a little bit different twist yeah and if so um exactly so you may be a an expert in human factors but you want to see you want to build a curriculum now you want to teach it you want to see what other people have done um I think the results from metad adventor are more relevant to that as oppos some of these are getting people up to speed or not up to speed but teaching people human factors um teaching the reader human factors and we're trying to build a curriculum for others um so different different results um so we came back uh I came back to metad metor after doing the literature search and wanted to um create some learning objectives so I asked it can you please create uh 10 learning objectives uh for this project I'm working on and I put in that that summary that I had to generate before um and here they are I sent this to to Lori ahead of time so she could get a get a head start on looking at these but I'm wondering if any of these stood out to you as particularly relevant um anything that seemed off base what did you think of these 10 learning objectives uh I think it's a lot to fit in a 4-Hour workshop and which is what we found we couldn't fit it all in so we were mostly focused on 678 our audience already knew knew how to do root cause analysis and knew five Y and some of those basic um methods to try to drive into a root cause or contributing factors and so we were mostly focused on okay now that you know that assuming they did a good job at a systems thinking how do you now get an error prooof uh solution that will reduce risk and sustain over time so that was the way we focused um scoped down our our uh education in this particular case nice very smart so it looks like this a lot 10 things to put in one session is a lot yeah absolutely and you um it looks like six seven and eight are really highly focused on application exercises and being very active about putting the information into into um yes we even we did this in a safe EnV environment a PSO protected space and people brought their own case studies with them and that was part of the prep work to come to this Workshop I actually did this with uh Brandon Robertson from our team uh he's in our um patient safety quality group cool um in this uh so yeah and Lori makes a great Point usually the fewer learning objectives the more um effective you can be um you can really dive into them and um so asking for 10 was just to see what it would come up with and then we could pick the ones that are most um most relevant or more um Lor's Learners uh were more advanced um they had some baseline knowledge and were ready to do application stuff um so six seven and are great for her Learners may not be great for for newbies um but uh a good she was very thoughtful about who she was working with um I I really liked these in the way that they for those who have written learning objectives this list of 10 really advances in terms of um difficulty so you know one two three are all um kind of lowlevel when you're looking at taxonomy the tool we use to pick verbs to to write learning objectives Define identify and analyze our verbs that are towards the bottom um and then evaluate design articulate create are are much higher and are harder to do um but are thought to have better learning outcomes um so this is a nice progression of of difficulty um when thinking about how to teach um human factors engineering um so uh I do want to one tip when you working with a chatbot of any sort a GPT they actually are better when you're nice to them um so I like to compliment them and thank them for when they work uh when they do stuff for me so this is an excellent list of learning objectives I particularly like that they get more complex as we go down the list and then I said can you outline a 4-Hour comprehensive Hands-On learning session including interdisciplinary Learners that covers all of these objectives um Lori our expert is telling me this is a bad idea to try to do all 10 in um in four hours I agree with her um but this was uh what I tried to to play with um and we'll we'll get our experts opinion on on how this turns out um so it is responding kindly to me as well certainly here's an outline for a 4-Hour comprehensive Hands-On learning section session um so an introduction and overiew a brief overview of human factors engineering and healthc Care explaining the session structure and objectives and then the basics of human factors and root cause analysis um Lori any thoughts so far on these first two pieces those first two I did include in my my uh agenda great I part of it was to teach human factors they might know the systems approach but they may not necessarily know how to integrate or even know they might have their definition of human factors but I thought it was important to level set the team to what I call the science of human factors and how to great that yeah get everyone on the same page make sure you're all using the same terminology to describe things yeah totally um the next three parts were and all these times uh this the the GPT gave me the times that it wanted for these different sections um so breaking into teams assigning roles um within the teams um in introducing the case study um so this seems like a a case that the facilitator Lori might bring not the personal case from a person but giving them a uh specific case that they can all work through and then an exercise looking at the root cause of of this adverse event um using different principles and the fishbone diagrams and five y's um Lori expert opinions um yes I agree with these but again what we cut out an hour there yes I did have a case study that I brought to them um that I went over very quickly because most of the focus was for them to talk about their cases so I just gave them my quick one as an example nice um and the other thing we did introduce was a tool on how a tool that will help uh that Brandon actually um developed her tweak that's based on the national uh patient safety Foundation a tool to how to um rate your interventions for strength and that was a good tool that we use all the time here at kilan so yeah that was a um Lori sent me this tool it it is very cool um and uh I think it would be a good thing to share with this group um afterwards but oh be glad to sh that yeah um the GPT was very thoughtful of our human needs and uh added in a 15minute break which I appreciated um and then uh systems thinking Workshop uh solution development implementation planning I got a little lost in this um Lori does this make sense to you again because we weren't teaching the nitty-gritty on how to do these uh tools we assume they already had systems thinking I did give them some examples of that when I just did the quick overview of human factors because human factors is based on system thinking but we didn't go really deeply into the system thinking we did go through the solution development process though but we didn't spend 45 minutes it was tailored according to their specific question so might have brought a case that they thought was really tough and then we brainstormed with the whole group right there on their specific example cool but all of these words and ideas are reasonable I just don't understand yes they are okay great um and then presentations and pure feedback um the so what Lori spent a lot of time on was number 10 the real world application um and this is kind of a afterthought in this in this one um and then a whole reflection and closing section oh by the way that we we were told we must have at least a 30 minute lunch whereas so chat GPT gave you 15 minute break we were told must have a 30 minute lunch break that's good and that actually did provide good networking so yeah great idea um and then it gave me some extra stuff on how I might make this more effective um using multimedia presentations um handouts or digital resources for references during um exercises so Lori mentioned this really cool tool she and Brandon developed um that would be a great thing to have make sure everyone had when they're doing the the project um real time collaborative tool for team exercises so um maybe a whiteboard a flip chart um there's online tools for this kind of thing um yeah so some some basic pointers but if you're developing a workshop for the first time these are these are great um ideas um so I would love to any thoughts on this um Lori or anyone in the group I'd love to hear what people thought you know all I asked this this program to do this GPT to do was give me a 4our workshop and gave it the summary of the problem and this is what I got back what do people think and uh feel free to unmute if you um would like to or you can put stuff in the chat chat and we can moderate that as well any thoughts about how this could be applied to a project that you guys or anyone is working on here today anyone willing to share um hey this is Harsh from the human factors team hi harsh yeah thank you thank you for your presentations insightful I think uh so me and my colleagues we're working on building some guidelines for um EKG Telemetry systems within our within Hospital systems and how we can Implement them and we may do a webinar soonish so I can see us benefiting from having a tool like this to help us plan and facilitate this webinar and then maybe even a workshop after that nice yeah I think that's a great idea and I would um I think this kind of stepwise approach that I modeled here is a good way to do it so starting with a summary of what you're trying to achieve asking for learning objectives I think the part I didn't do but Lori would have done and you should probably do is take that list of 10 or so learning objectives what whatever you ask for and really pick out the ones you care most about um or tweak them and narrow it down and then take that list of learning objectives and ask for a plan um for the for the session based on that and and it's not going to be perfect um it never is but it's going to give you um ideas and and things you can tweak and and modify um to to make your own um other thoughts Sarah said looks fantastic I could Envision using this to give some additional ideas for sessions I've developed in the past to refresh with new information perfect so Sarah probably has learning objectives for these things um and she could just throw those into the uh system and see what she gets back um and see if there are new ideas that she could do to try to refresh um old teaching stuff great idea other there was one problem with this Workshop well there you know it wasn't perfect as we mentioned but one thing um you all probably didn't notice but I took the time to figure out was um uh this was 5 hours and 40 minutes when you add all the time up so uh I noticed that too I did that I was like how are you fitting all that in four hours so I did I did look up your times yeah so um I was like oh good try chaty PT not quite um so I said let's try it again uh take two so I said again I was very nice very appreciative this is excellent thank you well done my only issue is is not 4 hours it is 5 hours and 40 minutes meaning it is much too long can you make a new curriculum similar to this but meets the time requirement of 4 hours you will need to make judicious decisions to cut less relevant material or condense material or sections of the curriculum so um ah it was very uh contrite uh I apologize for the oversight and timing you are absolutely right and I appreciate your attention to detail let's create a revised for our curriculum and it's similar uh but yeah the whole thing we have a shorter break 10 minutes um and I'm just trying to see we won't look too closely at this but it does hit four hours on the dot and what I liked it gave me this whole thing and then what it did for me is it summarized the changes it made um so I could and I should take the old curriculum and the new curriculum and compare them side by side but it it also summarized the key changes so it shortened the EXT introduction and the lecture portion um it combined the team formation and case study introduction into one uh section integrated systems thinking into a mini Workshop it merged solution development with implementation planning um condensed presentation time um combined the real world application with reflection and removed the separate closing session so things we all probably could have done on our own but all I did was say Hey you messed up make it four hours and it did a really good job I thought of doing that again probably way definitely way too much to cover in four hours um but it was a fun exercise um to to see how how powerful this tool can be um when you go at it with a step stepwise approach um and then it just covers everything um so I wanted to show you the theory suggestor tool in action um and uh we described it before I tried it with Lori's project um but her project already has so many theories baked into it or theories or Frameworks or approaches or um so human factors engineering is its own thing root cause analysis is its own thing so what I was pleased to see it came back with basically all the things that Lori told us about um so human factors Theory interprofessional education human factors ergonomics Theory um Institute for healthcare Improvement modules root cause analysis quality Improvement nothing we didn't know already so this was um I'm glad it came back with all these things um but less uh less useful application of this tool so I came um oops this was supposed to say something else um so I had a question there when it tells you that and I tried to click on what are those two Theory papers on human factor right is there a quick way I can find that when I clicked on it I couldn't go to it not yet not yet we still need to build that part okay it will be clickable eventually um so I'm sorry this uh I must have copied um this was supposed so one thing I'm curious about is um is a lot of people a lot of attendings and residents will tell me that they want their medical students or residents to be more confident to be um be more confident when they're you know presenting or suggesting management plans um and I think that's the wrong framing um I think confidence comes with experience um and what we actually want our Learners to be is brave we want them to be more Brave in in you know putting themselves out there for for critique and criticism um hopefully constructive criticism but it's really we want them to put themselves out there so what I actually wrote here instead of what um instead of Lor's project was um uh instilling bravery in medical students in uh clinical learning environments so we're um trying to look at bravery medical students clinical learning environments is the research phenomenon we care about um and here were some really good theories that I didn't you know hadn't heard of uh some of them a resilience Theory um is sounds really relevant to to this application uh personal identity formation is a big thing we think about in in medical education um uh self-determined mination Theory critical pedagogy stereotype threat microaggressions empowerment Theory stress and coping Theory communities of practice so a lot of good stuff here um it's a lot it's a lot of theories maybe 2025 but way less than 230 um so it definitely points you in a good direction and shows you all the different ways you might come at this question or problem of Bravery in um in these clinical learning environments um so this is the power of this tool the theory suggestor in um taking a kind of an esoteric um uh enigmatic uh question bravery uh in students and clinical learning environments on giving you some some ways you might make that more tangible more real um so just wanted to show you that in action um and I want to show you what's coming down the pipeline so everything in Black exists everything in red is coming um so these lessons are going to continue to grow we'll write more and more and help uh create more Orient um getting people oriented to the field uh one of the things that we want to help people do is to write research questions so we'll have this research question companion something you can describe your project and it'll help give you a list of of research questions that you might um use uh the theory database and suggestor are going to get much better uh we have a brand new literature search that is um about to come online um you're the first people uh I'm telling about um but one of the problems we had with our first one is um there's journals like New England Journal medic uh of medicine which are mostly clinical but have some high yield education papers every once in a while and we've developed a tool that can just pull out that one paper one or two papers from these journals um so it'll be a much more robust uh tool um and then we have we'll have something to help find methodologies uh building an online community and um help with writing um quickly we we have over 1500 users in 80 countries um and people are spending a lot of time on our site and um what you could do is sign up again all we need is a email address um tell anyone you think might use it um that it's this schhol free resource uh use it and then give us feedback um we're always trying to make it better for people who who like it and want it and need it so you are the people who we want to serve and help um so anything we can do to make it better for you is is good info for us and then let us know if we can help or if you want to help uh build this tool and happy to take any questions comments concerns thank you so much Dr Stetson and thanks for telling us first about the things that are um coming down the pipeline we I feel honored that we're the first we're the first ones to hear it so um we are out of time however uh so if folks need to jump off we totally understand but if there are any other questions Dr ston I don't know how much time you have but um folks again can feel free to unmute or type something in the chat um we can try to get to those or um if you want to just reach out to myself or someone from our teach team we'd also be happy to pass those along to Dr Stetson as well totally and I've just put my email address in the chat.