Philips and Microsoft Unveil Next-Gen Mixed Reality Operating Room Powered by HoloLens 2

VCBeat (WeChat ID: vcbeat) has learned that the annual Mobile World Congress (hereinafter referred to as “MWC 2019”) was held at Fira Barcelona Gran Via from February 25 to 28. As the most influential exhibition in the global telecommunications sector, MWC attracts numerous telecom technology companies, which attend the event to showcase their latest innovations.

At MWC 2019, Royal Philips (Philips, NYSE: PHG, AEX: PHIA), a global leader in health technology, unveiled on the 26th a future operating room mixed reality application tool co-developed with Microsoft Corporation (Microsoft, NASDAQ: MSFT). Built on Philips’ industry-leading Azurion image-guided therapy platform and Microsoft HoloLens 2Mixed Reality Head-Mounted Display (also known as VR Glasses, VR Helmet), the two companies will showcase new augmented reality application tools for image-guided minimally invasive therapy.

Compared with open surgery, minimally invasive treatment requires only a small incision. Specialized instruments, such as catheters, are inserted through the incision and guided to the treatment area, which may include major organs such as the heart, blood vessels, brain, and liver. During minimally invasive procedures, physicians cannot directly visualize or palpate the treatment area; instead, they rely on advanced medical imaging technologies—such as ultra-low-dose X-ray imaging and ultrasound—as well as other navigation techniques to observe the patient’s internal anatomy and guide the movement of specialized instruments. Philips is a leading provider of these high-tech interventional suites and hybrid operating rooms.

Image source: Philips official website

The augmented reality concept developed by Philips and Microsoft for HoloLens 2 transforms real-time imaging of critical data currently displayed on large 2D screens, along with data from other sources, into a 3D holographic augmented reality environment that can be easily and intuitively controlled by physicians in an ergonomic manner. This concept is being used to gather additional clinical insights to support the development of commercially available augmented reality solutions in the future, ultimately enabling their application in image-guided procedures.

“The transition from open surgery to image-guided surgery has driven improvements in patient outcomes and reduced costs, while significantly shortening patients’ postoperative hospital observation times,” said Atul Gupta, MD, Chief Medical Officer of Image Guided Therapy at Philips, and a practicing interventional and diagnostic radiologist. “On our Azurion platform, we seamlessly integrate a range of data sources in an intuitive and easy-to-understand manner. Through our collaboration with Microsoft HoloLens 2, we can elevate this capability to a new level, immersing physicians in a tailored augmented reality environment. This concept allows us to see the real world overlaid with real-time data and 3D medical images, guiding us in delivering precise treatments. More importantly, we can also control Azurion through voice recognition, eye tracking, and advanced gestures. All of this is designed to keep our focus squarely on the patient.”

Alex Kipman, Microsoft’s Researcher in Artificial Intelligence and Mixed Reality, stated, “Mixed reality offers people new ways to interact with both the digital and physical worlds, bringing the benefits of the digital revolution to communities around the globe. Companies across various industries are currently experimenting with products we have co-developed with our partners. Mixed reality holds immense potential in the healthcare sector, and our collaboration with Philips demonstrates that this potential is already translating into momentum for real-world development.”

On February 24, Microsoft unveiled HoloLens 2 at MWC 2019. HoloLens is a standalone holographic computer that enables hands-free, heads-up interaction with three-dimensional digital objects. Building on the breakthrough technology of its predecessor, HoloLens 2 delivers a more immersive and comfortable experience, offering greater out-of-the-box value. Leveraging existing and new Azure cloud services, along with built-in AI capabilities, HoloLens 2 stands as the ultimate intelligent edge device.

Since its global launch in February 2017, the Azurion platform has been used by physicians to treat more than 500,000 patients across over 80 countries. Powered by Philips’ proprietary ConnectOS and integrating technological innovations in both software and hardware, the platform enables the incorporation of advanced digital innovations through ConnectOS. Philips previously announced that it is developing augmented reality solutions for spinal, cranial, and trauma surgeries.

Innovation is at the core of Philips’ health technology development strategy, with the company spending approximately €1.8 billion annually on research and development (R&D). In recent years, Philips has significantly increased its investments in software, adaptive intelligence, and data science. Today, one out of every two R&D professionals at Philips is active in these fields. Philips has successfully developed the HealthSuite platform, a universal digital framework that connects consumers, patients, and healthcare providers within a cloud-connected health ecosystem comprising devices, applications, and tools.

(Compiled by Cheng Xiaoqin)