Apple Vision Pro: Catalyzing the Takeoff of XR in Healthcare

At the 2023 Apple Worldwide Developers Conference (WWDC), Apple devoted a full 40 minutes to introducing the Apple Vision Pro mixed reality headset, electrifying the audience in a way not seen in years. As Apple’s first XR device, it represents the company’s most significant product innovation since the Apple Watch. CEO Tim Cook even stated that the Vision Pro heralds the advent of the spatial computing era, suggesting it could surpass the iPhone’s prominence within the next decade.

It is widely believed in the industry that Apple’s first XR device will drive the development of the entire sector, including the niche field of MXR (Medical XR, as classified by the FDA). In fact, VCBeat (WeChat ID: VCBeat) has been closely monitoring MXR, a field that has made remarkable progress in recent years. This article aims to provide industry insights into the current state of MXR and the potential impact of Apple Vision Pro on this sector.

XR is an umbrella term for several technologies, including VR. In addition to the well-known VR (Virtual Reality) and AR (Augmented Reality), MR (Mixed Reality) has also gained momentum in recent years. Furthermore, new similar technologies distinct from the aforementioned ones may emerge in the future. All such technologies can be categorized under XR.

In layman's terms, VR uses a headset to isolate users from the real-world environment, immersing them in a purely virtual digital environment generated by computers. AR, on the other hand, adds or removes interactive virtual objects or information generated in real time by computers within the real-world environment.

MR can be viewed as a combination of VR and AR, first commercialized by Microsoft. Conceptually, it is similar to AR, involving the intermingling of real and virtual environments; however, its method of integration resembles the principles of VR, combining real and virtual environments through digitally generated, interactive digital interfaces. Apple’s Vision Pro belongs to this category of technology.

MXR has undergone a considerable period of development. However, it is only recently, with the gradual maturation of technology and a significant decline in hardware costs, that MXR has begun to experience rapid growth and mass adoption, and is regarded as having substantial market prospects. According to a report by Fact&Factors, the global MXR market size is projected to surge from $2.7 billion in 2020 to $40.98 billion in 2026, with a forecast compound annual growth rate (CAGR) of 34.89% from 2021 to 2026.

As early as 2015–2016, the FDA approved several medical devices utilizing XR technology, primarily for 3D reconstruction in medical imaging. Subsequently, the application scenarios of MXR expanded into surgical navigation and further into disease treatment. Notably, such MXR medical devices with therapeutic functions are generally considered a form of digital therapeutics.

However, current regulatory strategies lack sufficient evaluation of this new technology. For instance, regulatory authorities do not have appropriate methods to assess the role of MXR platforms in various critical medical scenarios, nor can they evaluate their safety and efficacy in surgical and diagnostic applications. Furthermore, as MXR is largely based on consumer-grade platforms, the various sensors employed by these devices—including accelerometers, inertial measurement units (IMUs), and cameras—have not been validated for clinical use environments.

South Korea was among the earliest countries and regions to explore the establishment of industry-specific regulatory frameworks for Mixed Reality (MR). On June 29, 2018, South Korea’s Ministry of Food and Drug Safety issued the “Guidelines on Review and Approval of AR/VR-Based Medical Devices,” which defined and classified medical devices utilizing VR and AR technologies. The aim was to facilitate rapid product development by enterprises and support industry growth.

For instance, the Guidelines state that XR devices utilizing surgical navigation and biosignals such as electroencephalography (EEG) to assist in patient rehabilitation should be classified as medical devices, whereas XR devices used for medical training or daily low-risk rehabilitation activities, such as memory training, are not subject to medical device regulations. In April 2021, the Ministry of Food and Drug Safety of South Korea further updated these Guidelines.

In contrast, although the FDA has not issued specific guidelines for MXR, MXR medical devices have continued to receive approval, with a clearly evident growth trend. According to FDA statistics, from 2015 to September 2022, the FDA approved 39 MXR medical devices. Among these, 20 were approved between 2021 and 2022, accounting for more than half of the total.

FDA Has Approved MXR Medical Device (Graphic by VCBeat)

In this process, the FDA also recognized that the evaluation of MXR technology was significantly lagging. Consequently, the Office of Science and Engineering Laboratories (OSEL) under the FDA included the Medical Extended Reality program, which studies regulatory issues related to MXR medical devices, as one of its 20 research projects. This initiative aims to provide a scientific basis for regulatory requirements and decision-making in support of innovative MXR devices.

It is foreseeable that the FDA will inevitably release regulatory guidelines specifically targeting MXR in the near future.

Currently, China has not yet established specific regulatory strategies for MXR medical devices. However, in 2020, the National Medical Products Administration approved VR-based cognitive rehabilitation medical devices. Since then, several MXR products targeting different diseases have successively obtained medical device approval.

In terms of clinical efficacy, the effectiveness of MXR has gradually gained recognition. In April 2022, the Ophthalmic Inspection and Testing Group of the Ophthalmology Professional Committee of the China Association of Medical Equipment (CAME), in conjunction with the Ophthalmic Innovative Diagnosis, Treatment Technologies and Equipment Group of the same committee, convened a multidisciplinary panel comprising ophthalmologists, optical instrument experts, and VR/AR device development engineers. The panel reached a consensus on hardware configuration and software functional parameter settings for VR/AR devices used in visual health, indications and contraindications for wearing such devices, ophthalmic medical examination metrics for device assessment, and management strategies for visual fatigue. Subsequently, they released the Expert Consensus on the Use of Virtual Reality and Augmented Reality Devices Based on Visual Health (2022).

On the critical front of payment, MXR has also achieved breakthrough progress. In May 2023, the U.S. Centers for Medicare & Medicaid Services (CMS) established a unique billing code, E1905, for RelieVRx, AppliedVR’s VR-based therapy for chronic lower back pain, enabling reimbursement for this eight-week treatment program.

Prior to this, MXR had already been included in Germany’s directory for digital therapeutics. This MXR solution, named Invirto, treats agoraphobia and social anxiety disorder through exposure therapy and has qualified for temporary reimbursement (€620). Once Invirto completes long-term clinical trials and demonstrates its effectiveness in clinical treatment or health economics, it may become eligible for permanent reimbursement.

In certain regions of China, MXR has also been included in the medical service fee schedule. For instance, in November 2020, Jiangsu Province added “Virtual Reality Cognitive Behavioral Therapy (311503035)” to its catalog of priced medical service items. The specific description states: “Using a virtual reality cognitive behavioral therapy device, one or more patients are immersed in scenarios they fear or reject, allowing them to engage in natural interactions with these environments.” Although currently classified as a new item under trial and categorized as a Class C self-pay service, this represents a promising exploration.

In summary, MXR has gained recognition primarily due to its ability to remotely deliver standard and novel content in a highly immersive and realistic manner, as well as its capacity to tailor solutions to various clinical settings, thereby reducing costs and enhancing efficiency within healthcare processes.

This capability was first applied in the field of surgical procedures to address the common pain points of “poor visibility, inaccuracy, and lack of visualization” associated with conventional planar medical imaging used for intraoperative guidance: endoscopic images offer a narrow field of view and lack subcutaneous information; ultrasound images suffer from significant noise interference and lack three-dimensional information; and X-ray images exhibit overlapping accumulations of multiple tissues and lack three-dimensional information.

MXR can reconstruct 3D images of human tissues and organs based on digitalized planar images, reducing physicians' reliance on imagination; by assisting physicians in decision-making and preoperative planning, it significantly shortens intraoperative time, thereby accelerating the learning curve for junior physicians. With technological advancements, MXR has also overcome challenges in real-time intraoperative procedures and navigation.

This category of products enjoys high recognition abroad. According to statistics from VCBeat, among the 39 MXR medical devices approved by the FDA, as many as 30 are related to surgical navigation assistance and 3D image reconstruction, accounting for a significant 76.9%.

As the application of MXR continues to be explored, more products are entering fields such as disease diagnosis, treatment, and recovery. VCBeat has found that MXR is currently involved in multiple areas, including pediatric diagnosis and treatment, pain management, mental health, sleep, neurological disorders, ophthalmic diagnosis, telemedicine, virtual care, and postoperative and other rehabilitative therapies.The good news is that domestic enterprises have not been absent in these fields; they have all made strategic deployments.

Yet, despite all changes, the fundamental principle remains the same: regardless of how indications evolve, MXR practitioners must clearly recognize the core key to success in the realm of serious medical care—namely, constructing virtual reality scenarios and integrating them with critical healthcare pain points.

Leveraging its leading motion capture technology, Noitom has begun to expand into the medical field and launched an orthopedic surgery visualization system. Zhou Feng, Head of Medical Technology at Noitom, explained the core essence of MXR to VCBeat: “I believe the essence of MXR lies in the organic integration of the physical world with digital space, and how this integration can enhance the efficiency and experience of both doctors and patients in real-world settings. This is the core of MXR.”

“Once we digitize the real world, on one hand, we can provide more valuable real-time guidance; on the other hand, the accumulation of digital data holds significant implications for scientific research, future new product development, and improvements in clinical practice itself. The accumulation of data will generate an ‘avalanche effect’ in the era of intelligence, leading to ever-increasing value creation,” he added.

Meanwhile, as with any emerging digital health technology, clinical effectiveness is the key to gaining recognition. Dr. Wang Silun, Founder and CEO of Yiwei Medical, noted in an interview with VCBeat that MXR must ultimately return to scientific clinical validation: “Healthcare is a highly serious endeavor. No matter how advanced or groundbreaking your technology may be, whether there is a genuine medical application scenario, or indeed whether the technology is effective, must be verified through evidence-based medicine.”

“We need to continuously refine our product design through large-scale pilot studies before we can proceed to formal clinical trials and ultimately obtain regulatory approval. This process requires substantial investment of time and resources, with no guarantee of final success. XR is a technology that evolves rapidly; those accustomed to the fast pace of this industry may find it difficult to adapt to the slower rhythm of the healthcare sector. However, for a product to gain clinical acceptance, it must undergo these rigorous processes,” he added.

Xuzhishi Technology is among the first companies in China to obtain regulatory approval for its MXR medical devices, and it has spent several years exploring product commercialization. Kang Cheng, founder and CEO of the company, stated that MXR must be genuinely adopted by physicians to help them address practical challenges in their daily work, thereby reducing burdens and enhancing efficiency, rather than serving merely as a superficial gimmick.

“Different hospitals have distinct needs and customer profiles, requiring tailored solutions to meet their specific requirements. Therefore, it is essential to expand into more hospitals, engage in in-depth communication with clinicians, and commit to long-term operational engagement. Market entry should not be considered the end goal; hospital procurement is merely the beginning. The greater challenge lies in ensuring that physicians actively adopt the product and that it genuinely addresses their clinical problems,” stated Kang Cheng.

Despite the continuous progress and breakthroughs in Mixed Reality (MR), certain key technologies remain imperfect. These shortcomings include risks related to device usability, such as neck pain caused by overly heavy headsets, as well as issues concerning the content and image quality generated by XR devices, including low-contrast images, display errors, information overload, dizziness, fatigue, and potential impacts on vision.

It is precisely for this reason that MXR often faces criticism in real-world applications. So, can Apple’s Vision Pro address these pain points associated with existing MXR solutions? Judging by its demonstrated features and capabilities, the Vision Pro—developed over nearly seven years despite repeated delays—indeed holds such potential.

Compared to existing mainstream XR devices, Vision Pro maintains Apple’s consistently high standards in industrial design. Furthermore, Vision Pro boasts three major advantages.

Apple Vision Pro’s design maintains its consistently high standards (Image source: Apple’s official website)

First, it offers high precision. The Vision Pro headset features a total resolution of 23 million pixels, delivering high-precision visuals to each eye equivalent to a 100-inch 4K display. It is also equipped with up to 12 cameras, 5 sensors, and 6 microphones. Furthermore, Zeiss optical inserts can be added to further enhance the accuracy of eye tracking, ensuring precise data collection.

Judging from the demo video, users wearing Vision Pro can interact solely through gesture controls and eye tracking, representing a significant advancement over the controller-based interaction of traditional XR devices. This major evolution in interaction methods inevitably recalls the original iPhone, which introduced revolutionary interactions that fundamentally changed the history of smartphones.

Next is low latency. The Vision Pro is equipped with the high-performance M2 chip, the same one found in the latest generation of MacBook Pro laptops, along with the newly developed R1 chip designed to address data latency issues. This combination significantly reduces the motion sickness often caused by latency in previous XR devices, providing a superior immersive experience. It is worth noting that in past negative evaluations of MXR devices, dizziness induced by latency has consistently been a criticized shortcoming.

Finally, the integration of artificial intelligence and brain-computer interfaces. According to Apple’s R&D personnel, Vision Pro can acquire user data from sensors, functioning as a hidden, non-invasive “brain-computer interface.” By leveraging AI technology to perceive user emotions and predict user behavior, it creates more suitable virtual environments to enhance the user experience.

Although the Vision Pro has not yet been officially launched, many mixed reality (MXR) practitioners are considering leveraging the new capabilities it offers. XRHealth, which utilizes MXR to address autism, stress and anxiety, chronic pain, and neurological disorders, recently announced a U.S. patent for a learning system technology that can adjust training and treatment protocols based on patients’ biometric and motion data.

This patented technology will initially be used to integrate the Apple Watch with the Vision Pro, collecting vital signs such as blood pressure, heart rate, and respiratory rate through various sensors. It displays the patient’s psychological or physiological status during treatment, adjusts the therapeutic regimen accordingly, and pushes the updated plan to the Vision Pro, thereby enabling closed-loop AI therapy.

In addition to MXR enterprises, other companies in the healthcare industry have also expressed significant interest in Vision Pro. Some firms have shown strong enthusiasm for leveraging Vision Pro’s eye-tracking technology to collect data; with minimal effort, this can yield valuable insights into user habits, including medication consumption patterns and biometric identification, which are highly useful for digital marketing.

MXR practitioners in China have also paid close attention to Vision Pro.

Wang Silun stated that the Vision Pro’s high precision, low latency, and advanced intelligence make it highly suitable for applications in brain science. These include dementia screening, assessment of cognitive development in children, treatment of neurological disorders such as attention deficit disorders in children, and research in neurotechnology, all of which represent significant advancements in the field of brain science.

“Its visual analysis technology is relatively robust, employing non-contact visual analysis techniques that are theoretically superior to current eye-tracking technologies based on scanning. Of course, its accuracy still needs to be validated, but I believe this represents a trend,” introduced Wang Silun.

“However, I believe the Vision Pro still needs to enhance its data analytics capabilities, including data interpretation. Of course, Apple is primarily building a platform and cannot incorporate extensive functionality in vertical domains; we can expect third-party ecosystem partners to refine these use cases.” He also highlighted areas where the Vision Pro could be improved.

Kang Cheng believes that Apple’s launch of the Vision Pro will lay the foundation for greater recognition of mixed reality (MXR) in China: “Currently, the performance of XR devices is somewhat criticized by physicians, as they fail to fully meet their needs in many aspects. The Vision Pro offers significant improvements over existing products in terms of interaction convenience, comfort, and clarity, effectively integrating the best features of current VR, AR, and MR devices. Therefore, I believe this will greatly promote the entire industry.”

“With a solid hardware foundation and Apple’s robust software ecosystem, recognition among doctors and experts will increase manifold compared to the past. Furthermore, the launch of Apple’s XR products will help educate consumers and patients, significantly enhancing patient awareness and leading them to view XR as an advanced therapeutic approach,” he added.

Certainly, Kang Cheng also highlighted concerns regarding several details of the Vision Pro: “The external battery pack and Apple’s closed ecosystem pose certain challenges for medical applications, particularly given the relatively short standby time. If the battery were removed, would this compromise the headset’s lightweight advantage? Furthermore, the stability of bare-hand interaction and its dexterity remain to be verified through hands-on experience with the actual device. If it can deliver performance comparable to that demonstrated, I believe it would be sufficient to meet many practical needs.”

Zhou Feng also acknowledged Apple’s role in driving supply chain advancements: “A crucial aspect of Apple is that, beyond the product itself, it catalyzes a series of improvements across its supply chain. This overall enhancement of the supply chain, in turn, drives domestic XR manufacturers, providing opportunities for technological upgrades throughout the entire ecosystem.”

Regarding ecosystem development, Zhou Feng believes that the level of support Apple provides to developers will impact its application in the healthcare sector: “Relatively speaking, Apple’s previous product strategy has been more closed compared to Android. However, healthcare applications may have greater demands for low-level access or customized adaptation. Therefore, I think a definitive judgment on this issue can only be made after the official product launch and further research by relevant professionals.”

Despite its high price tag, Zhou Feng believes that the cost of over $3,000 may not pose a significant barrier to its adoption in the medical field, given the relative price insensitivity of MXR healthcare applications. “While the Vision Pro is indeed more expensive than ordinary XR products, this is not a fatal flaw for the professional medical market. What matters more for healthcare is the boundary of technical capabilities endowed by its existing technological architecture and framework.”

Years ago, the iPhone ushered in the era of smartphones, triggering a chain of butterfly effects. Since then, digital healthcare initiatives—including internet-based medical services, smart healthcare, digital therapeutics, 5G, and the Internet of Things (IoT)—have gradually emerged and grown stronger. It is no exaggeration to say that without the iPhone back then, today’s digital healthcare landscape would not exist.

Although Vision Pro arrived late and still has room for improvement, Apple’s entry into the market has undoubtedly had a positive impact on mixed reality (MR). Whether in terms of market acceptance, ecosystem development, or empowerment of specific medical application scenarios, Vision Pro will serve as a significant accelerator. While it is premature to claim that a new era of digital healthcare systems built on this foundation has been fully established, it clearly represents a major step forward compared to the past.

VCBeat will continue to closely monitor developments in related fields and deliver first-hand reports.