Michigan State University researchers have developed a virtual reality curriculum to prepare health care professionals and students for the complexities of caring for patients with tracheostomies and laryngectomies.
A tracheostomy is a surgical procedure that offers a solution for individuals with breathing issues. The procedure creates an opening in the neck into the windpipe to provide an airway. Then, a tube is inserted through the opening to aid breathing. A laryngectomy removes all or part of the voice box and is often used in patients with laryngeal cancer or severe larynx damage. Speech therapy is a vital part of the recovery for both procedures, which can impact a patient’s ability to breathe, swallow and communicate.
Jeff Searl, professor in the Department of Communicative Sciences and Disorders, or CSD, within the MSU College of Communication Arts and Sciences, leads a project funded by the National Cancer Institute that bridges the gap between the entertainment world’s cutting-edge technology and the medical field’s practical needs to engage practitioners in immersive, low-risk training that enhances both learning outcomes and patient care.
Virtual meets practical
For years, health care students have faced limited opportunities to work directly with tracheostomy or laryngectomy patients, Searl said. Despite the critical importance of managing these cases correctly, the existing training options — a combination of textbooks, videos and hands-on learning during clinical rotations — do not always provide the depth of knowledge students need. Many health care providers, including experienced physicians, also lack confidence in working with these patients.
“The likelihood that medical students would actually get very much patient exposure to these kinds of unique groups of patients, such as those with a trach or a laryngectomy, is relatively small,” said Searl, who directs MSU’s Lip-Tongue-Larynx, or LiTL, Lab. “The other issue is, we basically have patients that become the learning ground for our students. And that’s not always great if your very first learning experience is on a live person — especially when it’s a delicate procedure that you’re doing that has the potential to create at least psychological discomfort and maybe actual discomfort and pain.”
The VR curriculum aims to address this gap by providing learners with a virtual simulation where they can interact with patients in a realistic, controlled environment without the risk of harming real people.
To do that successfully, Searl needed to call in another kind of expert.
Enter the GEL, or Games for Entertainment and Learning, Lab, led by experienced game designers Professor Brian Winn and Professor of Practice Andrew Dennis. After all, the same game engines used to create immersive experiences in entertainment, like Unreal Engine or Unity, are also key to creating realistic VR simulations for health care.
To replicate the immersive feeling of a real-life medical scenario, VR requires high fidelity in both visuals and interactivity, all while maintaining a smooth 90 frames per second to prevent motion sickness.
“We’re trying to make it as real and as good-looking as possible,” said Dennis, who is a co-investigator on the project. “There’s all the modeling, all the things that go into it . . . but with the stuff that I teach, there’s the added challenge: it has to work in real-time.”
Kathryn Genoa-Obradovich, a communicative sciences and disorders doctoral candidate and clinician, said working on this project has fostered connections across departments, bringing together disciplines that traditionally have little overlap.
“It’s so cool to be able to work with the game design students and to get to know them and their realm of study and expertise. I would have never been able to collaborate with them without this,” she said. “From what I’ve witnessed, they seemed to be similarly excited. Having students wanting to learn about what we do . . . and some even considering maybe pursuing a doctorate in our field because they were now exposed to it, and are like, ‘Whoa, here’s how I can blend engineering with coding with game design.’”
Building a VR curriculum to solve real-world health care problems
From the start, this project has been a gratifying experience for Dennis, who noticed an industry-wide decline in approved grants for VR and game projects due to many projects failing to develop.
“Making a game is very difficult,” Dennis acknowledged. “I think it turned a lot of funding agencies away from games for a while, too, because they would give grants for people to make games and they would not deliver — because everyone learns, wow, it is way harder than you think.”
Fortunately, the GEL team was prepared for the challenge.
“We got a small amount of money through the startup funds to develop a functional prototype to show it could be done,” Dennis explained. “That led us to a slightly larger grant from Trifecta to keep developing it. That built us a pretty robust prototype that we then took to the National Institutes for Health for their cancer project — and I thought that was one of the best things we did,” he said.
“That prototype . . . when we went to them, they could see it, they knew it existed. They knew we had the capabilities of making what we described. We weren’t just describing something we wanted to do; we were describing expanding on something we had already done.”
From there, the curriculum development process was a multiyear effort, collaborating with Mary Kay Smith from the Learning and Assessment Center and MSU College of Osteopathic Medicine, Peter LaPine from CSD, and Gayle Lourens from MSU’s College of Nursing. Dennis’ team at the GEL Lab worked closely with health care faculty and graduate students throughout the summer months to create various modules and refine the VR experience based on their feedback.
“Week by week, we would feed them information and content, and they’d say yes, we can do that or no, we can’t,” Searl explained. This back-and-forth process helped ensure that the final product would be both medically accurate and easy for students to navigate.
Genoa-Obradovich took the VR immersion one step further — even getting into a motion capture suit herself to simulate patient movements. “There are certain things that would be tricky to try to emulate unless you’re familiar with the population,” she said. “As a clinician, I’m the one who would kind of know what they look like or how they’re responding. But then to have them strap you into all of these motion sensors, it’s really fun. That’s been exciting; you feel like you’re in a video game.”
As for the audio component, the team invited real patients to lend their voices to developing this curriculum.
“To make it more realistic when the students are analyzing deficits or difficulties, or the simulated patient’s challenges or chief complaints, we’ve had some individuals record their voices for us,” Genoa-Obradovich said.
The first module covers foundational knowledge, such as the anatomy of the neck and the differences between tracheostomies and laryngectomies. The second module includes a set of four patient cases in which learners work their way through different scenarios with those patients. All the while, students can interact with 3D models, assembling parts of the airway, familiarizing themselves with tools and supplies, and practicing procedures in a hands-on, immersive way.
Searl would like to see this curriculum made available as widely as possible and plans to make the VR modules free for download via the Meta App Store, allowing programs around the globe to incorporate the curriculum into their training.
“What we really hope will happen is that training programs at MSU, in the U.S. or anywhere around the world would have an instructor who knows about the app and figures out a way to position it within their own curriculum, within their own training, wherever it’s going to fit best for them, and then actually have an instructor, or several instructors, who will lead students through the experience,” he said.
Forward thinking
Looking ahead, the team at the GEL Lab see even broader applications for VR in education and health care, including mental health support.
The GEL Lab encourages project leads to contact them about including game or extended reality, or XR (all technology that combines the physical and virtual worlds, including virtual reality) in their research proposals, as developing interactive simulations is what the lab does best.
“It’s kind of fun to work with the students and the VR team because they have some ideas about how to gamify things and make it fun and interesting,” Searl said.
Genoa-Obradovich agrees.
“It’s really inspiring, but it’s also invigorating, working all together. You can feed off each other’s energy, which is nice,” she said. “Once I was exposed to VR, I immediately started to think about how to apply it to my patient populations, whether directly in therapy or with further training for students.”
Winn said most of the GEL Lab projects have introduced him to colleagues working in a variety of different disciplines, including health care — and he’s excited to see how his group can contribute to introducing XR into new and innovative spaces.
“Collectively, we can design, develop and research to advance the field forward,” he said. “This has definitely been the case when it comes to redefining education and collaboration using XR.”
