Poala Sanchez, graduate student in surgical education, wears an augmented reality HoloLens device that displays instructions for performing a surgical task. (photos by Laughing Dog)
Wenjing He, '19 PhD, fixes her eyes on the patient, contemplating her next action. Outfitted in a blue surgical gown, mask and gloves, she looks every part the surgeon. Across the table, her partner Paola Sanchez carefully reaches forward with scissors, delicately making a cut in the material in front of her. Each wears a headset with two cameras attached. One camera films what the wearer is looking at; the other records the wearer's eye itself. Together, the two are participating in a virtual surgery.
He and Sanchez are not practicing surgeons here in Canada. Both completed their medical training internationally and are now pursuing a graduate degree in the University of Alberta's Department of Surgery. The pair are part of a research team in the U of A's Surgical Simulation Research Lab (SSRL), working to improve the training of surgeons in the future.
"I would say over 90 per cent of resources in a health-care system are used to study diseases and patients. But our lab is looking at issues related to the operator," said SSRL lead Bin Zheng, associate professor of surgery at the U of A and Alberta Health Services chair in Surgical Simulation. "We take a unique approach to health care by having surgeons as our subjects in our lab. Better surgical training translates into better patient safety."
The Surgical Simulation Lab is led by Bin Zheng, associate professor, surgery and Alberta Health Services Chair in Surgical Simulation.
Over the past seven years, Zheng's mission has been to develop more impactful training for surgical residents.
The SSRL team is made up of physicians, physicists, computing scientists and engineers hailing from both Canada and countries around the world. They include graduate students, PhD candidates, visiting professors and several undergraduate students.
"They're all brilliant," said Zheng.
"We use technological equipment that is fascinating for a lot of people, such as eye tracking, augmented and virtual reality," said Sanchez, a master's student in surgical education who helps manage the SSRL. "I am more enchanted with the ideas, projects and the ways that we can employ technology in medical and surgical education."
The lab has several projects in the works, each focused on helping new surgical residents learn the skills needed to become experts in their craft.
Black box for surgical training
Among the projects is the one demonstrated by He and Sanchez. Zheng describes it as the OR Black Box, taking a cue from the black boxes in airplanes that can be analyzed to understand errors that lead to a crash.
"There are a few disadvantages behind a video-based black box. First, surgeons do not like being watched by the video in the operating room. Second, video evidence is good for analyzing the problems post hoc, but has limited power to prevent errors from happening," said Zheng.
"We believe a true black box in the OR should provide a stream of digital data that can track down the movement of surgical tools and surgeons' vision, and can be analyzed by the computer constantly without adding extra observers."
With the Black Box, a motion-tracking system monitors the surgeon's body and tool movement on multiple points. The surgeon's eye movement is also monitored with eye trackers to detect when errors happen due to a lack of surgical team coordination.
"We track the surgical team's eye movements during a procedure and then we analyze whether they're looking at the same spot or not and whether their gaze overlaps over time," said He. "What we found is for an expert team, the eye gaze overlaps at a rate of about 70 per cent. For a novice team, the team's gaze overlaps at 30 per cent. Team gaze overlap can predict team performance. If they're looking at the same spot, it means they are taking in the same visual information and collaborating."
AR in the OR
A second project underway in the lab brings augmented reality (AR) to surgical training.
AR exploded into the mainstream in 2016 with the release of the video game Pokémon Go. In the game, players use their smartphone to "find" and "capture" creatures in real-world environments. The technology, which essentially displays information over real-life settings, is being adapted in new ways in health care.
In the SSRL, researchers are using AR technology along with eye tracking to teach surgical procedures. While practising a surgical procedure, trainees wear a hands-free augmented reality device-called a HoloLens-on their heads. Zheng's team has implanted a set of three cameras to it to track the user's eye movements. Through eye tracking and augmented reality, the learner can receive instructions on the lenses of the AR goggles while performing the surgical task they experience difficulty in doing.
"For example, if I ask you to do a chest tube insertion and you find a step to be difficult, you will likely start looking around trying to think of how to continue the procedure," explained Sanchez. "At that moment, the learner is no longer engaged in the task, and we can capture this moment on the eye movements recorded by the eye tracking. By detecting the moment when the person needs some help, we can intervene. Instead of looking at the instructions again or watching the instructional video on the computer, the learner can have the information about the task displayed on the HoloLens in real time."
"We are able to detect those teachable moments and then trigger the computer to project instructional knowledge to help you to learn," added Zheng.
Other research being done, by undergraduate engineering student Bassim Nima, includes a plan to monitor the brainwaves of surgeons to detect when they become fatigued during a procedure. If fatigue is detected, surgeons can take a short break to refocus-making the surgery safer for the patient.
Going through the motions
The team is also using haptic feedback as a means of quickly training the muscle memory of young surgeons who need to perform complicated procedures with great dexterity.
The SSRL team built a system to record an expert's hand motions and translate those exact motions digitally into a second station, where a novice learner could feel what the experienced surgeon did and then try to copy the action-rather than simply try to simulate the action from video or verbal instruction. Zheng believes the technique will greatly speed up the acquisition of complicated dexterity skills.
"It's like you're putting your hands into the gloves of the expert," said Zheng.
The Royal College of Physicians and Surgeons of Canada is funding the SSRL to find evidence of how well using haptic feedback works to train surgical skills.
Coming soon
This work is in the research phase and is not yet in use training surgical residents. The researchers in the SSRL believe it's only a matter of time before the technology and techniques they are innovating become standard practice in surgical and medical education.
"I'm pretty sure once the technology is more accessible-augmented reality, virtual reality and eye tracking-we'll see many people implementing what we're doing here," said Sanchez.
Zheng believes that time is not far distant and he is already envisioning the future of surgical training in Alberta.
"We need to have a simulation lab open for our surgical residents 24 hours a day, seven days a week," said Zheng. "The goal for the future should be to have a regional simulation centre, allowing every single surgeon to practise a set amount of time before getting a surgical licence or renewing their licence for the next year-pretty much like how aviation works. We're not there yet. But we're getting closer."
Poala Sanchez and Wenjing He participate in a virtual surgery with eye tracking equipment, augmented and virtual reality technology.