Virtual Simulation in Medical Education

In this episode our guest, Rachel Umoren, MBBCh, MS,  joins Kate Mulligan, PhD to discuss Virtual Simulation in Medical Education.   

General Resources:

• Virtual Simulation in Medical Education Transcript
• Simulation and Game-Based Learning for the Health Professions by Rachel Umoren
• Healthcare Simulation Dictionary
• Society for Simulation in Healthcare

Virtual prenatal counseling simulator

• JMIR Medical Education - Increasing Realism and Variety of Virtual Patient Dialogues for Prenatal Counseling Education Through a Novel Application of ChatGPT: Exploratory Observational Study
• JMIR Serious Games - Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA): Pilot Study

Virtual simulations on newborn resuscitation and care

• Virtual Essential Newborn Care (2023, updated version) 
  • https://sites.uw.edu/virtualenc/
• Virtual simulations for neonatal education - PubMed (nih.gov)

Electronic Helping Babies Breathe (2019)

• Doctor Creates Virtual Reality Simulation to Save Babies’ Lives - On the Pulse (seattlechildrens.org)
• eHBB: a randomised controlled trial of virtual reality or video for neonatal resuscitation refresher training in healthcare workers in resource-scarce settings | BMJ Open
• Using Mobile Virtual Reality Simulation to Prepare for In-Person Helping Babies Breathe Training: Secondary Analysis of a Randomized Controlled Trial (the eHBB/mHBS Trial) - PMC (nih.gov)

Virtual disaster evacuation simulator

• Evacuation of Vulnerable and Critical Patients: Multimodal Simulation for Nurse-Led Patient Evacuation. - Abstract - Europe PMC
• EVAC: Evacuation of Vulnerable and Critical Pediatric Patients for Nurses - PubMed (nih.gov)
• EVAC Washington.edu

Virtual team training simulations

Transcript

Copy Transcript Copied

Virtual Simulation in Medical Education  

Kate Mulligan, PhD, Teaching Professor, Neuroscience and Human Anatomy, Associate Director, Center for Learning and Innovation in Medical Education (CLIME)   

Rachel A Umoren, MBBCh, MS, Associate Professor, Pediatrics, Associate Division Head, Research, Medical Director, Inpatient Telehealth, Seattle Childrens Hospital, Research Director NEST Program, Adjunct Associate Professor, Global Health & Healthcare Simulation Science 

[Intro Music] 

Amanda Garza: Welcome to CLIMEcast, I'm Amanda Garza the CLIME Program Manager. Our guest today is Dr. Rachel Umoren, who is an Associate Professor of Pediatrics in the Division of Neonatology at the University of Washington. She is the Associate Division Head for Research, Director of Research for the Neonatal Education and Stimulation Based Training, NEST, program. 

She's the Inpatient Medical Director for Telehealth at Seattle Children's Hospital. She also holds adjunct appointments in the Department of Global Health, in the Department of Surgery, Division of Healthcare, Simulation Science. Dr. Umoren is active in the Society for Simulation and Healthcare as the Chair of the Virtual Simulation Section. 

She is a fellow of the American Academy of Pediatrics, serving as a member of the section on Global Health Executive Committee and as Program Director for the International Community Access to Child Health Program. Dr. Umoren and CLIME Associate Director Kate Mulligan will discuss virtual simulation and medical education in this episode. 

Enjoy! 

Kate Mulligan: So welcome everybody to today's episode. I'm delighted to have as a guest, Dr. Rachel Umoren, and I'm really excited to be catching up with you Rachel, because it's been years and years since I first met you and you were doing such amazingly cutting edge work with virtual reality in those days that I wanted to catch up and share the evolution of that, um, which I'm sure you can help us with. I realized from a quick chat with you that I shouldn't be calling it virtual reality so much. So maybe the first thing to do would be to talk about some definitions like what virtual reality is or what the preferred terminology is. Can you help us with that? 

Rachel Umoren: I sure can so and I have had help with this as well.Definitions are really tricky, and they evolve over time as you rightly pointed out. And so what we used to call one thing back, you know, 5, 10 years ago, we call it something else now, but it's still the same thing. And so when I use as my point of reference, the society for simulation in healthcare, healthcare simulation dictionary, and it defines virtual simulations as simulations in which a real person operates simulated systems. 

So that's in contrast to what we typically think of as a simulation. So a live simulation, which involves real people operating rail systems, like mannequins or other objects that they can actually feel and touch as we think about virtual reality, which a lot of folks are getting more and more conversant with this because it's entered the mainstream to a greater degree than it had previously that blend of what is physically present to what is 100 percent computer generated is really a continuum. So, there is reality, which I think we all are familiar with, and then there's augmented reality, which is the physical environment that you can see with some computer-generated graphics overlaid on it. So, if you have one of the newer cars, for example, you've experienced that when you're backing into a parking spot, there are these lines that appear as guides on your rear-view video, and they show you how far you can go and how to stay between the lines so that's augmented reality. So, a lot of us are using this every day, and it's helping us. Helping us not to run into anything when we back up our cars. And then there's mixed, so mixed reality simulation is a hybrid. So, you have the virtual reality environments, you have actual real environments so let's say you have a mannequin and then you have these graphics that overlay on the mannequin, but you can still interact with them, that's called mixed reality. And then finally, virtual reality is where you have 100 percent computer generated graphics. So, you're fully immersed in that virtual environment. You don't see anything that's actually real, it's all virtual. So that is the full end of the spectrum from reality to virtual reality. 

And there's a lot in between. And that's why I like to use the term virtual simulation, because that can encompass all these things that we've just talked about.  

Kate Mulligan: That was really interesting and helpful. It's encouraging for me to know that I'm using augmented reality already because some of it feels really intimidating from a, you know, educators’ perspective, the technology, and things like that involved, but I'm excited to hear about what kinds of learning you think virtual simulation can be most meaningfully applied. In a bit of brief research before we started talking, I read some encouraging stuff about forklift driver training and diversity training. And I was wondering if you could comment on these or any other instances that might help make more concrete how virtual simulation is being used in learning in general.  

Rachel Umoren: Yeah, so those are excellent examples of kind of skills based training and more cognitive communication type use cases. But really, when you're considering an educational tool, you have to take a step back and ask a few questions. So, you have to think about the audience, 

you have to think about what you need to teach or assess, and then you need to think about what's the best tool. As educators we think about learning theory as we try to decide what kinds of learning a certain tool might be able to augments or support and virtual simulations, like other kinds of simulations, utilize the same principles of adult learning, like experiential learning and social cognitive theory and to a degree, but game based theory that other simulations do. 

And so when I think of a virtual simulation, I think of it as a immersive experience that learners can repeat over and over and get feedback, even if there's no instructor present. So that allows them to discover things for themselves. Again, but still with that additional guidance and help so that if there are mistakes made, they can get feedback and it's in a safe space where they can practice without necessarily feeling judged either by the instructor or potentially even by peers. 

Now, there are multi user virtual simulations, and that's where the social cognitive theory comes in, because you can have health care team members who are in different roles, like the team leader, like a bedside nurse or respiratory therapist, and they can observe how their actions are interacting with the behaviors of others and the actions of others. 

And they can use these experiences to learn from each other. So that's another way that I've seen virtual simulations used. The other thing to be aware of is that the concepts of game theory can apply to virtual simulations as well. So, like games, these are simulations that introduce goals that create purpose, they're linked to measurable outcomes. And so once a learner is immersed in this environment and they have a goal to achieve whether it's a goal that's in competition with others or in cooperation with others. Um, then they are motivated so this is a really strong motivator for learners to get engaged in this environment and to learn and to come back for more learning. 

So that's really exciting. So, as we think about those use cases, as you asked about that seem to be really compelling for virtual simulations and in particular virtual reality, which, as I mentioned, is the 100 percent computer generated, fully immersed environment. I have come across some really great examples of tools that can be used to teach learners. 

For example, what does it Uh, feel like to be a patient with dementia. So, you are in this environment and you find yourself in a hospital room, you're in a hospital gown, you're on a bed, there's nobody else in the room, but you need something. Maybe you need your glasses and you're trying to reach your glasses and they fall on the floor. 

And, and, you know, you just feel that sense of other helplessness that a patient in that situation would feel and that helps learners to really understand and connect with certain situations or a certain persona much more so than if it was just on a computer screen, or you're reading about what it feels like to be a patient with dementia. 

And then another skills type of learning experience that can be really powerful is point of care ultrasound. So, if we go into the healthcare space now, point of care ultrasound is a growing area that a lot of people are becoming more and more familiar with it's I think one day going to completely replace our stethoscope and so using that handheld probe with an augmented reality headset allows you to both see what you're scanning. So, this is physically with the mannequin, but also see inside the mannequin to see the organs. These are now computer-generated graphics that show the organs that are below the location of the probe so that when you then add another layer, because the instructor can add the organ layer, and then can add the ultrasound image. You can see exactly what you would be scanning. And so this in depth layering of knowledge helps the learners get beyond the I just need to put the probe here and I should see this to this is exactly what I am expecting to see when I put the probe in this location. 

So that's another really nice combination of technology with the physical environments and skills that learners need to have. So. It's generally that combination of learning strategies. So, the experiential learning, the opportunity for reflection and application of the material, combining those together with opportunities for introducing learners to this space. 

So the pre briefing, the debriefing, all these things together, much more so than just a platform or an experience is what brings that learning together for learners.  

Kate Mulligan: Thank you, always thought that it's a lot, it looks like a lot of fun to do virtual simulation or virtual reality things, but I hadn't appreciated the depth of the theoretical foundation for it. 

So thank you for sharing all of that with us. You did some virtual simulation work early on using 360 degree images in constructing a learning experience on patient evacuation. And I have a hazy memory of a NICU rendering, so could you fill in what that tool was and how successful it was and where you've evolved in virtual simulation education? 

Rachel Umoren: Yes, I believe we talked about that some years ago. And so we call that the evac disaster evacuation simulator. And it was one of the 1st projects that I did when I came to UW, in fact, I worked with some colleagues, Dr. Megan Gray and Dr. Anita Thomas, and we captured some 360 degree images of a NICU room and of an ED room, and we use this in collaboration with a developer to create this interactive simulation to teach emergency department and NICU nurses how to evacuate a patient safely in the event of a disaster, and I believe our disaster is earthquakes. And so we had the picture captured in the room, but we had the before picture and then we had the after picture with all these earthquakes, zigzag cracks in the wall, which, which were paper, of course, cracks, but they looked pretty good in the images. And so we actually rolled this out, um, to 60 nurses, 30 NICU and 30 AD, and we compared the performance of these groups of learners, um, that were assigned to the virtual simulation training and to standard didactic lecture on what to do in the event of an earthquake. And the learners with the virtual simulation training were actually more efficient in their evacuation process when we did a standard simulation to see how they would do with the mannequin. And so that was really encouraging. We went on to develop a mobile version of the virtual feedback software. And that actually was expanded to have not just the earthquake, but the fire disaster simulation, as well as the active shooter simulation. 

Kate Mulligan: Thank goodness, there's opportunities to practice for those things.  

Rachel Umoren: I think one of the real, real strengths of virtual simulations are that ability to create that immersive learning experience that would be otherwise really difficult. Or, you know, this is super expensive to do in the real world. So these high acuity, low occurrence events like neonatal resuscitation or pediatric resuscitation, they just don't happen as often as they do in the adult space. 

And so that's really been one of the areas that the various organizations and groups are centered around, and we've also worked with health care providers in low resource settings to train them on using neonatal resuscitation skills in their practice. And we've used mobile virtual reality for that, so I want to say my favorite at the moment is a program that I'm working on called “Virtual Essential Newborn Care.” We've rolled this out in pilot to healthcare workers in Nigeria, and we're following them to see how they are able to retain the knowledge on how to care for newborns at different stages. So, premature newborns, term newborns, in the delivery room,  after the delivery room. 

When to refer, you know, there's so many different elements to this training, which ordinarily would take several days and with lots of refresher training to really consolidate the knowledge that we're hopeful that using virtual simulations will help with this training.  

Kate Mulligan: Rachel, I'm guessing that there are going to be members of our audience that have never put on a virtual reality headset or really thought about any of the applications that you're talking about. Could you paint a little bit more of a word picture about what one aspect of the training that you're talking about in Nigeria looks like? Like, if you're a trainee, what are you encountering or what are you wearing and what one little task that you have to do, how does that get implemented in virtual simulation? 

Rachel Umoren: Yeah, so what you need if you're a trainee is a simple mobile phone. So any, any smartphone would work and you need to have the application on your device. You also need to have a virtual reality headset. They are some really low-cost virtual reality headsets that work with mobile phones. 

So, you just take your phone and you slip it into the back of the headset and then you put it to your face and you look around. And do you find yourself in a room with the mother and her baby and the equipment that you need the simulation walks you through the process by basically introducing the scenario and then you start out by preparing for the birth of the baby. And once the baby is born, you have to respond based on what the baby is doing. So, the interaction itself is directly with the objects in the room. So, you use your eyes to control where your little marker, which we call a reticle is pointed at, and that is where you actually can select it and pick it up and put it down somewhere else and you can pick up the baby and move it over to the warmer and move it over to the scale and move it back to the mother. 

And so, even without having to have fancy controllers, which are great. And, you know, with the more advanced virtual reality headsets, you can even have haptic gloves that you can use. And I was at a recent conference where there was a demonstration of using haptic gloves to help facilitate care for postpartum hemorrhage. So, doing the fundal massage, and you can actually feel that fundus as you're doing the massage, so there's a whole spectrum of available resources now to do lots of different things, but you can be very very simple and basic with the mobile VR, or you can be really sophisticated all the way up to haptic gloves as far as being interactive. 

Kate Mulligan: Thanks for that description of the Virtual Essential Newborn Project. Uh, Rachel, can you talk a little bit more about related research projects? 

Rachel Umoren: Well, these days I'm really interested in learning more about how it works, both from the standpoint of the learner experience, but also from the standpoint of implementation and deployment in both high and low resource settings. 

So we're doing several studies in, as I mentioned, in Nigeria, very resource constrained setting, learning about what works well for deployment. How do you push out updates? What are the ways that you can really constrain the size of each modules so that it's light on the bandwidth and the cost as far as being able to deploy these out more broadly and we've made a lot of good progress. 

And so I'm excited about that. There's a lot of opportunities still for evaluation of impact, and so we recently conducted a randomized control trial that was with the previous iteration of our essential newborn care software that we call the HBB electronic helping babies breathe. Um, and that's available on the Google play store for free. So anyone can try it out. We had an evaluation with health care workers in Nigeria and in Kenya. So the multi country study, a randomized control trial and compared using a virtual reality mobile version of the HBB with using a video to watch a new natal resuscitation. Carried out from start to finish and then we had a control group, which control group had access to digital manual and looking at these 3 groups We saw that the virtual reality group actually maintain their skills over the period of the 6 months time that we were following up. Um, and they were actually several points higher than their immediate post course performance at the end of 6 months as compared to the other groups, which were up to 20 points lower as far as their performance went. 

So, we've really felt that although the groups all did reasonably well and maintain their skills at the virtual reality group being better than when they initially started was really encouraging. 

Kate Mulligan: So what other innovative ways have you seen virtual simulation employed in undergraduate medical education or maybe continuing medical education locally and nationally? 

Rachel Umoren: Yes, so there there's some really interesting ways that I've come across. So, some instructors have used some video conferencing platforms like Zoom and WebEx and combine them with some short snippets of video that portrayed a patient and the patient monitor. And so all the learners get together on Zoom, like, we are doing right now and the instructor screen shares and shows a patient, which is a short video, which is a virtual patient, not a real patient and a virtual monitor and uses that as the starting point for this scenario and makes changes as needed. It's a really accessible way to do a virtual simulation and there are scenarios that are available and for free on the website. I'll tell us some box. So that is a real opportunity. 

If you're just getting started and, you know. So, you have a headset, but you do use zoom or some other video conferencing platform to just get started with a virtual type of simulation. Another example uses Google doc. So, we are all pretty familiar with the way that we can interactively. Move things around or type in Google Doc and other people can be on the same Google Doc at the same time and we can all see what each other's doing. 

So, a group put together as a simulation called the virtual recess room on Google Doc. And again, this is free and it's available, but basically, you have a image of a patient and you have all the equipment that you can move around in real time. And others who are with you in the simulation can move these things around in real time as well, and can type into the Google Doc if the medication is given. 

And so it's all set up in a platform that we're all pretty familiar with, and that we can all interact in together. So that is a very simple multiuser virtual space using a tool that you're familiar with. And then there have been standardized virtual patients that have been used for history taking and difficult conversations from for many years. 

Actually, medical students have been practicing with virtual patients for quite some time. The technology has evolved and improved over time. And so it's moved beyond the screen based, choose what you want to say kind of phrase to, you can have a real time conversation with the patient and your words are being detected and being looped back to the patient using speech detection software. 

Patient can talk back to you, you can do this on a screen or in a VR headset and so just increases the immersiveness of the experience and hopefully the learning and then with funding from CLIME, we're actually piloting a virtual prenatal counseling simulator here at UW as part of a curriculum for pediatric OB and family medicine residents and teaching them how to talk with families when they're experiencing potential delivery of a late preterm infant.So this is a tool that we're using right now.  

[00:23:10] Kate Mulligan: You're inspiring me because I'm thinking about back in the foundation's curriculum, you know, before people get into clinical environments, we often have the paper case study and suddenly it's like Oh, you know, maybe there's some applications for virtual simulation that make that more interactive and more engaging than Okay, What do you think, Jane? What do you think, Thomas? What do you think, Jose? You know, it's just one of those things that seems so deadening sometimes and maybe if the barrier is not too high, we can try incorporating some of these tools. That's really exciting.  

I'm wondering if we could step back now and consider a few more basic questions or a few more basic perspectives about virtual simulation experiences. 

Do you feel like some trainees or students take to virtual simulation learning experiences better than others? 

Rachel Umoren: I do think so. I think everyone has kind of a threshold of tolerance for certain new things. So there's the technology tolerance and trying new things as far as the technology goes. There's also, you probably are aware of the condition called cyber sickness that some people experience if they're in a virtual reality headset that it can feel dizzy or nauseous. And so that can be a limitation for certain people. Um, we can actually uncover sometimes some vision abnormalities, like, if people actually shouldn't be wearing glasses, but aren't wearing glasses, they could have trouble seeing in a virtual environment. 

And so some folks may struggle, and that's why when I think of virtual simulations, I really feel like there there's opportunity and we should push more for experiences that are available on multiple platforms. So I should be able to pick up my mobile phone and have an experience. Experience that's a screen based experience that does not require the headset. 

Um, or I could go to my laptop and experience it as a PC based experience. Um, or as a web-based experience if I have internet access or as a VR experience. So there should be that opportunity for folks that have different preferences or limitations to be able to use these experiences in any way that they are able to. 

And this is really important for equity as well, because we know that not every learner will have access to a headset, and so there's accessibility across different platforms and devices and also considering those options that don't require internet access. Maybe you could download it, but then it doesn't require the internet to run it for example. Or you can have access to it on a flash drive and use it that way and don't need the internet at all. So thinking about those options, particularly for folks that are in more rural areas and who would really benefit from some of these tools because they have limited access to simulation in general. 

Kate Mulligan: Wow. That's great. There's so much to think about. It does sound though, that a basic issue is going to be people who have visual impairments might not benefit as much as people who have regular visual capacity. I'm struggling to think about virtual simulation that wouldn't be visually dependent. 

Rachel Umoren: It's an interesting thought because you have the haptic option as well. Now, I won't say that I've come across anything specific in this, but I think there's opportunity that maybe hasn't been tapped yet as far as the use of haptics for individuals that may have challenges with their vision. I know there's definitely opportunity to make sure that individuals who have colorblindness not disadvantaged because we are intentional about the use of colors. 

So, for example in working with our development group, I think the initial reticle was supposed to the reticle is the dot that you look at to move around and make sure that. You know, what you've selected is selected. I believe my recollection was that initially it was either red or green. It was something that like we decided, no, we're going to make it orange so that it's clearly visible. 

And so these are design considerations as these things are developed. I remember also, we did have closed captioning, of course, very important. And my recollection is that we once had a learner who was hard of hearing, and so there were verbal instructions that were being read this was in a different simulation that I purchased off the shelf to use for communication training. 

And so we needed to make sure that the closed captioning was enabled. And then the other opportunity is in using mixed reality. So, if you recall, the mixed reality is the kind that you can see your actual environments as well as see the computer-generated graphics. So, if there's something that's omitted or missing cues that need to be communicated then using mixed reality to help with being able to see the real environment while you're interacting with the virtual environment can be a bridge to help ensure that nothing is being messed and added information can be communicated as needed. 

Kate Mulligan: All right, thank you. Rachel, can you tell us what the pitfalls are that we should be mindful of when we're thinking about stepping into the virtual simulation environment for our students? 

Rachel Umoren: This is a really important question. It's so important to be able to choose a tech modality platform that we're thinking about that fits the learning objectives and we need to consider the strengths and the limitations of the platform. 

So, uh, for example early on, and I still see this happening today some folks are trying to teach a lecture in a virtual environment. So, taking slides and putting them into a 3D virtual environment, it's just not the best modality because I can read my slides on my screen no problem I don't need to go into VR to read lectures. 

And so that's just a case that's really clearly mismatched with the power and the opportunity of virtual simulations. So you want to hone in on those experiences that are best delivered and mercifully you want to hone in on those experiences that can really not be delivered in a good way in other types of didactic spaces, and so being really thoughtful about what you're going to use virtual simulation for, and how you're going to really leverage the strengths and also the limitations, because there are some limitations that can be leveraged. So I'll give an example so we have our virtual prenatal counseling patient and she talks back to you, but sometimes she doesn't quite get it. Right. So we say, well, she is on magnesium. She's in labor. She may not always make sense. And that's true. That can happen actual patients, because they may be somewhat affected by those things that are happening to them. So that's how we're leaning into some of the limitations by creating the story that explains that  

Kate Mulligan: Sometimes it feels like a new tool comes along and everyone wants us to go ahead and start incorporating that and so I'm very happy that you're emphasizing that you need to match the tool to your learning objectives, not find. I'll learn the objective for the tool. Anatomy is an obvious use application, but there's a part of me that, that thinks, okay, I could maybe create say a virtual reality experience where you're climbing up the aorta and you're seeing the renal arteries going out and then you're seeing the phrenic arteries, et cetera to learn the distribution of those branches off the aorta, but is that can be better than just giving them a little stick diagram to learn and what's appropriate at this point in their learning? You know what I mean? And it's kind of like the amount of learning it would take for me to do something like that for what might end up being the fun aspect of learning, but no more useful really, or effective than just a stick diagram. 

Rachel Umoren: You have to weigh the amount of effort that it takes to create the simulation with the potential impact or effect. And so that, again, is a good example that just brings us back to it has to be intentional and it has to make sense to, to do it this way.   

Kate Mulligan: I was curious if you could talk a little bit about how virtual simulation might be useful in assessment of learners. 

Rachel Umoren: Yeah, so as you think about assessment, you know there's formative assessments and summative assessments. Both are possible in virtual simulations. So we talked about the game-based learning theory as being applicable here, the you know, formative feedback, it can be as simple as a ding for that's positive. 

You did great. And a buzzer for Nope, that was not the right choice. And so, again, as we think about how to meet the needs of all types of learners on the spectrum, simple feedback built into the simulation can be one way to do that. And some of the feedback at the end is actually being pretty extensively used in most virtual simulations at the end, or even during, you can see the check boxes for things that you've accomplished as you've gone through the simulation. 

And at the end, there's additional feedback and the score in some cases to rate your performance and so learners should really be given feedback is maybe that goes without saying and having a way to deliver feedback that's subjective and free of by a consistent across all learners plus the benefit of being able to do that without necessarily having to put extra burdens on instructors to give personalized feedback are all huge advantages of virtual simulation. 

Kate Mulligan: Yeah, from a faculty point of view, it's really impressive that everything's being captured digitally, right? So it's not just you worrying about whether you're focusing on the learner at the right time. All of that information is there. Oh, that's a lot to think about. You said something about bias. I was wondering what kind of safeguards are in place to Make sure that the algorithms aren't incorporating some bias into them, because yes, obviously, we want to, we want to have as little bias in our tools as we can, but I'm wondering if that's something that's in the consciousness of the developers at the moment. 

Rachel Umoren: Well, that that is a hugely important question, and I think 1, that's the whole health care system is thinking about now. So, as far as the virtual simulation goes the thoughts that I had were, it's really easy unintentionally for instructors to provide more feedback to one person, a little less feedback to the other, or maybe more specific or guided feedback in one case or the other. 

And sometimes it's that we're exhausted or tired, or it's been a long day and a lot of learners and so that was the perspective from which I approach that thought is, how can we be more consistent and eliminate variability as we're instructing and as we're providing feedback and, you know, since this is computer generated, feedback is likely to be more consistent. 

As far as bias goes, there's a whole range of concerns around the use of AI and how AI is trained and AI is definitely integrated into a lot of the standardized virtual patients. There's, of course the whole discussion around chatgpt is being another form of the language processing system that can spit out all this information and where does the information come from? 

Well, it comes from the Internet, and so it's really hard to get around kind of the garbage in garbage out situation. But as as we develop new simulations, it's really important to get perspectives from different stakeholders, many end users and ensuring that the simulations are portraying the environments and patients appropriately. 

And that's the cases that we write, whether they're for a virtual or actual mannequin-based scenarios that they're appropriate and inclusive of lots of diverse backgrounds and so that I think will help a lot.  

Kate Mulligan: Thank you. I'm glad people as careful and intentional as you are in that space right now. I was wondering, do you feel like there's becoming a significant body of research on the use of Virtual simulation in medical education? It feels like the nursing field has been doing this for a while and might have a bigger body of research data. Help me understand what the status of the research is at the moment on virtual simulation in medical education or health sciences education. 

Rachel Umoren: Yeah, so that's a really good question. And I think as we think about research on any specific tool or intervention, there's this model called the Kirkpatrick model. And so the lowest level is just looking at participants reactions that they like the training. So that's level 1. And, and that's the easiest to measure. You can just ask people, did you like it? And they will tell you yes or no. And then you go to level two, which is the degree to which the participants acquired the knowledge or the skills or attitude or confidence or whatever it is, but it's just at the level of the learner. And so did they learn what you wanted them to learn? And then level three is, did they apply what they learned? 

And level four is to what degree what they did in the clinical environment actually lately to some outcomes specifically for healthcare. We want patient level outcomes. So most of the research to date has really been conducted at those levels 1 and 2. so reaction and learning and there's a lot more effort currently being put into looking at behavior as a group of interested folks in the virtual simulation section, looking at how can we start to really hone in on these patient level outcomes and ensure that what we're teaching is matching up with what we intend to see as an outcome. Including randomized control trial, so, as I mentioned, we designed a randomized control trial bar group. 

There have been other groups that have done the same and they've also been some cost comparison studies looking at the cost of using virtual simulation versus mannequin based simulation and these generally point to virtual. Simulations being more affordable over the long run, because you can use them with many more learners and learner to facilitator ratio is a lot better for virtual simulations. 

You could roll out one simulations at 1000 learners without even having to have an instructor. There are some recent systematic reviews of the literature to your question that recommend 1 of my favorites is actually from almost 10 years ago now, but they looked at 65 studies on virtual simulations and they found that confidence was 20 percent higher and declarative knowledge was 11 percent higher procedural knowledge is 14 percent higher and retention was 9 percent higher with virtual simulations as compared to just using regular teaching traditional teaching approaches and so that's one that I quote often, but there have been more recent reviews, looking specifically at the medical literature, looking at the nursing literature, which you are right there's definitely more literature in the nursing field than there is in the medical field. 

And then area that has had a significant gap is space of low resource environments and the use of virtual simulations in those spaces. So that's, that's the gap that we are trying to fill. 

[00:40:04] Kate Mulligan: One of the barriers for me in terms of thinking about virtual simulation in the classroom is I thought you'd have to have a huge computer as well as the headset and hand controllers and things like that. 

But you're saying there are definitely simpler way, simpler applications that can get some parts of the job done and some, some of your learning goals achieved. Um, but then you can obviously upgrade, um, if you've got the resources. Um, where would I go if I wanted to experience some of these, uh, already developed virtual simulation? 

Rachel Umoren: Yeah, so it's as simple as a trip over to your mobile play store. So, if you use Android or if you use Apple iOS and just type in virtual reality into your play store application, you will probably come up with any number of Mobile VR apps that you can try if you add health care to your search, you'll likely hit upon some things that are more health care related. 

But there are resources from various groups do the society for simulation in health care organization has a reference list of virtual reality tools and applications and. More recently, we're actually launching a virtual simulation tester database, and this will allow folks like you who are interested in trying out new virtual simulations, perhaps even before they're widely available to get direct access to them from developers and creators. 

And these, this is oriented towards the folks that are in academia, who are working in these spaces as well as others who may be in industry, and they're just interested in end user feedback. So that's an opportunity that's will be available this year.  

Kate Mulligan: I think you've done a nice job of helping us understand a little bit of the clinical environment training. Maybe if we move back a little bit in time, the foundational science training, do you have any tips for how we'd find out about what's going on? 

Rachel Umoren: Yeah, so actually foundational science. Thank you for taking me to that space because there there's a lot of opportunity in foundational science to apply visualizations of all kinds, right? 

So whether you're teaching anatomy or biology or any basic physiology science there these visualizations exist of internal processes of biochemical reactions of anatomical structures. I think that's it's actually where a lot of people have gone when they think about virtual simulations and some of these are have greater fidelity than others and some are more useful for certain groups than others. 

So, let's say, for example, you're training a paramedic on anatomy, maybe they don't need to have the real detailed, you know, layer by layer information that a medical student might need for example. Or a student in the school of dentistry, they just need to have the head up versus the whole rest of the body. 

And so you can tailor the offering, depending on the learner and what their needs are and what level of fidelity is important for their learning, but these are definitely tools that exist. And I think if 1 goes to any, like I mentioned there, there are, you can search online for specific tools. That is an excellent place to start. 

If you're a member of a group that has a simulation interest component to it, you can. You know, check there, because there are generally people who are using virtual simulations. You can talk to folks at your simulation center because they typically will have people who are knowledgeable or interested in virtual simulations and or can connect you with people who are so that's that's a really great local source. Even the health sciences library have some tools that they provided on how to set up a virtual space for a virtual simulation and we can add a link to that as well. All right.  

Kate Mulligan: Thank you. Thank you very much. So, in terms of these tools, Rachel, is it a mixture of people doing this as a labor of love on top of their day jobs and companies and collaborations between them? 

Or is it moving more now towards companies using consultants in health care to help them understand what they need to develop.  

Rachel Umoren: Well, so it's a little bit of a mixture, but it's definitely over the last 10 years move towards industry with those of us that are more on the academic side, being content experts and consultants. 

So, a great example is the National League for Nursing has a content expertise, which they had worked with the Walter Coolers publishing company and Laird all as a simulation company to introduce a simulation. And so virtual reality simulation for nursing for clinical decision making as a nurse , so that's kind of clinical assessment. Components and clinical judgment was a space that they felt could benefit from using virtual simulation as a tool. And so you can see, it's a partnership between organizations, publishers, and simulation companies to create a really robust simulation. There are other examples, Anesthesia Association worked with to create a series of assessments for anesthesiologists and that's been used for several years as part of their CME and evaluation. There are several really robust scenarios that have been developed and targeting different user groups in these partnerships. Um, and then there's still folks that are grant funded and working on individual applications at the academic level and doing evaluation and contributing to the science that way. 

And so there's, I think it's still a bit of a mix.  

Kate Mulligan: How close do you think we are to expecting widespread virtual simulation infiltration into the medical or health sciences, educational system at the moment. 

Rachel Umoren: Well, you know, I think we are closer than we have ever been. Once I see major simulation companies adding virtual simulations to their offerings, alongside mannequin based simulation, integrating our virtual simulation options so that learners even without a mannequin can use the platforms when I see publishing companies collaborating with industry, collaborating with academia to create more curricula and to actually test evaluate these curricula to ensure that they're standardized and that, you know, they can be broadly used across different learner professions and groups. 

Um, I think we are close, uh, we, we've come a long way and there are a lot more opportunities than there have been in the past. I think people are more aware, headsets are more available for those that need to use headsets. More development has been put into using different platforms agnostically so that you can get on the web or get on your phone or get on a headset. 

And then you would have a similar experience. And so these are all ways in which those barriers are gradually coming down, there are still barriers though, and I think that was probably going to be your next question and those 2 barriers I see primarily revolve around faculty development. So, awareness, which I'm really glad that we're having this conversation today to start to raise that awareness and start to introduce these concepts and get people thinking about using virtual simulations more and then IT support. 

Because we need to bring the IT teams alongside with us so that they understand more about the platforms more about the equipment more about the devices and so that these can be more available. But I've seen especially from the standpoint of clinical simulation carts that have been put together that have monitor and then just have a headset hanging on the side and you can just wheel this card into a clinical space and people can use it just for just in time training and they don't need to have a mannequin and just train there or you could take a mannequin and you know, overlay images on mannequin and have a better experience. And of course, learners can do this at home. They don't have to come to a simulation lab to be able to learn.  

[00:49:04] Kate Mulligan: Thank you so much, Rachel. I really enjoyed talking to you. You've blown my mind in so many ways, but what I really appreciate is how intentional you are and I love the emphasis on matching the tool to your learning objectives, keeping those front and center. And I'm so heartened by your efforts to take these kinds of tools to under resourced places. I'm so excited to hear about your studies in other countries. So thank you very much for joining us. 

And I believe that we're going to try and link some videos of some of these Scenarios and virtual simulations that you've been talking about in the show notes. So thank you very much. 

Rachel Umoren: Thank you so much for having me. And it's great to talk with you about these tools. I am continually inspired and excited by the possibilities and look forward to connecting with anyone who's interested in talking about More.  

Kate Mulligan: Thanks, Rachel and we look forward to the results of your CLIME funded research,  

Rachel Umoren: I look forward to sharing. Thanks. Bye bye. Bye bye.  

Amanda Garza: Thank you again to Dr. Umoren for taking the time to talk with us about her work in virtual simulation in medical education. Don't forget to subscribe to CLIMEcast on the streaming app of your choice to get updated when new episodes are released. 

Thanks!