Technology is reshaping all aspects of our lives. Once a week in The Future Of, we take a look at innovations in important areas, from agriculture to transportation, and what they will mean in the years and decades to come.
The case was complicated: shoulder arthroplasty, to treat an advanced case of arthritis affecting the patient’s glenoid – the spherical part of the ball joint in the shoulder. To treat the case as effectively as possible, the surgeon enlisted the help of the best. But the best was physically half a world away. What to do?
From his operating room in France, orthopedic surgeon Thomas Gregory donned a Microsoft Hololens 2 headset and called three colleagues in Brazil, Belgium and South Africa. They went through the patient’s holograms and discussed the surgery together; looking along Gregory opening the patient’s shoulder joint, Stephen Roche, Bruno Gobbato and Jean Florin Ciornohac suggested different clamps and alternative routes and observed Gregory’s technique. Together, they turned a bit of surgery into a technology showcase.
This surgery was not a sci-fi movie, however, nor a PR spinmaster’s dream. It took place just a few weeks ago, at Avicenne AP-HP Hospital in Bobigny, France – and it’s just the beginning.
Today: AR surgery is already here
“Hololens is like a smartphone for surgery, it is the performance and information tool that brings surgery into a new era”, explained Gregory, during a panel he organized with Microsoft for spread the word about the power of mixed reality in medicine. Much of what a surgeon does involves mental gymnastics, he said: Retaining mental images of 2D graphics and x-rays while walking through steps in the head and paying attention to signals in the head. ‘alarm.
“It’s almost a GPS for the surgeon.”
Augmented reality can lighten up some of it – it’s more than a fun game for Pokémon fans, in other words. AR can very effectively help doctors plan a procedure and perform it as efficiently as possible.
“It’s almost a GPS for the surgeon,” said Tom McGuiness, executive vice president of healthcare for Microsoft. Another enthusiastic supporter of the technology is Igor Sauer, head of experimental surgery at Charité Hospital in Germany. A traditional surgeon keeps 3D images in his head and mentally compares them to the real person lying in front of him, Sauer explained. This doctor has a clear need for image guidance, at the same time that she needs a clear view of the work in front of her. AR can deliver exactly that – not tomorrow or next year, but now.
There are other benefits as well, explained Massimo Chessa, interventionist cardiologist at San Donato Hospital in Italy. Pictures are much clearer than words and help overcome language barriers you probably didn’t know existed.
“Surgeons speak a different language, a real 3D language,” said Chessa. “Because they are used to working with 3D structures. But I am a cardiologist. I was trained as a 2D cardiologist, then I became a 3D interventionist. Mixed reality allows her to speak the same language as her support surgeons, improving the quality of patient care.
Indeed, Gregory believes in technology so much that he built a facility within his hospital to allow surgeons from anywhere in the world to familiarize themselves with best holographic practices and the power of mixed reality.
Tomorrow: training future doctors with entirely new tools
Assisted surgery appears to be the ultimate achievement of this technology, but it is only the first step. Any surgeon (except maybe Dr Strange) will tell you that they appreciate an extra pair of eyes during a particularly difficult surgery. But what about additional information? Sauer said he and his experimental surgery department were looking for ways to overlay things like anatomical depth information directly into the field of view, “to make a flashlight look like it’s shining through the organ. while reviewing it, “he said.
“Almost like having enlarged eyes,” Gregory added.
Other interesting advances lie in teaching, which requires observation. This is traditionally done in person: think of Rembrandt’s classic painting, “The Anatomy Lesson”. Sauer said his team had been working on volumetric recordings of the surgery, allowing prospective students to not only study the videotape, but to walk through a virtual three-dimensional recreation of the operating room itself.
But images alone are not sufficient training, especially for more specialized forms of surgery. You learn by doing, right? Virtual reality seems to be an ideal platform for a potential surgeon to hone his skills, but without a sense of touch, without being able to feel the scalpel penetrate the skin or the tinkle of a blade on a bone, a part of the skin. puzzle is lost.
“It’s almost like having enlarged eyes.”
Enter haptic feedback, the same technology that lets you feel the rumble of the road under your wheels in a driving game. FundamentalVR uses the advanced form of this – kinesthetic feedback, as opposed to simple skin haptics – to allow the surgeon to feel what surgery means with submillimeter precision, using a variety of standard devices, from gloves to haptic arms.
“The precision we need in surgery is on a different level than what you need to simulate taking a bullet or turning on a light,” CEO and co-founder Richard told me. Vincent. He also cited work on building a virtual version of the patient: ideally, by the time a surgeon is ready to operate on the physical you, he has cut the virtual version a dozen times. “If I can’t teach you what it feels like to do something wrong, and do it right, how can you possibly transfer yourself to the real world and do it?”
Further, the intersection of advanced technologies will enable a complete transformation at the literal cutting edge of the operating room.
“AI and extended reality could fundamentally change the way we perform certain procedures,” Sauer theorized. Imagine the possibilities of real-time spatial organ detection, as artificial intelligence scans a patient on a table, highlighting the outline of the liver for a surgeon to ensure that an already delicate procedure is performed so exactly as possible. New basic forms of treatment may be possible. In the near future, doctors will be able to move around a scalable 3D virtual version of the patient, studying a problem from all angles, before designing the least invasive and safest procedure possible. The actual surgery could be performed by robots, or at least by precise controlled robotic arms. It’s good on the road, though… isn’t it?
“This is of course nothing that we will do tomorrow,” Sauer said. “But maybe the next day.