Applications of Virtual Reality and Digital Twins in Healthcare Metaverse: Innovations, Platforms, and Emerging Solutions
Since announcing its entry into the metaverse business in the second half of last year and renaming its parent company from Facebook to Meta Platforms, Mark Zuckerberg has single-handedly propelled the concept of the “metaverse”—which had existed in science fiction for 30 years—into mainstream online discourse.
Abroad, some tech companies—including Roblox, the “first metaverse stock,” and global chip giant NVIDIA—had already begun laying the groundwork for the metaverse well before Mark Zuckerberg. Recently, Microsoft’s $68.7 billion acquisition of Blizzard further signaled its ambition to capture a share of the metaverse market. Both before and after Zuckerberg announced his metaverse initiatives, several Chinese tech giants had either already entered or decided to enter the metaverse space. Tencent invested in Roblox; ByteDance invested in game developer Code Universe and VR company Pico; Baidu launched Xi Rang, a metaverse-focused social app; and so on.
Amidst the widespread discussion, the gaming industry is widely recognized as the sector most closely resembling the early form of the metaverse. Although applications of the metaverse in healthcare are not yet common, we can already observe some startups taking action toward developing a metaverse-shaped healthcare industry. This article explores what healthcare integrated with the metaverse might look like. VCBeat hopes to spark further discussion and contribute to the better development of the healthcare sector.
There is still no universally accepted description to define the metaverse. The term “Metaverse” first appeared in the 1992 novel Snow Crash by American science fiction author Neal Stephenson, referring to a next-generation internet based on virtual reality. In Mark Zuckerberg’s vision, the metaverse can transform the relationship between people and technology, enabling users to complete experiences within or alongside virtual content, rather than simply interacting with digital products and solutions.
Let us now attempt to envision the metaverse within the context of healthcare. Currently, the internet-enabled healthcare industry offers products and solutions that allow patients and healthcare providers to view, share, exchange, create, or otherwise interact with digital content. Examples include entering patient data into electronic health record (EHR) systems, transmitting payment information through online portals, watching physical therapy demonstrations on applications, or sharing videos during telehealth consultations. In the metaverse era, conceivable healthcare experiences may include patients attending group therapy sessions via virtual reality (VR), surgeons planning surgical procedures through holographic anatomy, and pregnant women practicing breastfeeding techniques using augmented reality (AR) technology.
There is currently a consensus on several elements that the metaverse may involve: extended reality (including VR, AR, and mixed reality MR), 3D imaging technology, artificial intelligence, blockchain and cryptography, and even brain-computer interaction. Rock Health, a U.S.-based digital health fund, believes that healthcare in the metaverse holds broad potential, but at present, two major categories of application scenarios and initiatives are the most common:
1. Immersive Environment: A virtual world created through extended reality, in which healthcare providers and consumers engage for the purposes of medical education, assistance, or treatment
2. Digital Twin: A virtual representation of physical entities in the real world, which can be used to assist in healthcare-related decision-making
Examples of Digital Health Application Companies in the Metaverse (Source: Rock Health)
According to Rock Health statistics, digital health startups that integrated VR or AR technologies secured $198 million in financing across 11 deals in 2021, more than double the $93 million raised in 2020. Although this amount accounted for less than 1% of total investment in the digital health sector in 2021, it signaled early market and capital interest in the metaverse. Additionally, according to incomplete statistics from VCBeat, there are approximately 21 startups in China involved in VR/AR plus healthcare, six of which have received financing since 2019, with a total funding amount of RMB 131 million.
Building on these two dimensions, VCBeat has expanded into related fields and scenarios to curate the following overseas digital health product forms, which may represent the initial stage of healthcare’s evolution toward the metaverse.
Digital Medical Library
On June 30, 2013, American surgeon Rafael Grossmann performed the first surgery using Google Glass. He stated that from the moment he first saw Google Glass, he recognized the disruptive innovation wearable devices could bring to the healthcare industry. During this procedure, Grossmann conducted a “live broadcast,” enabling students to learn without needing to gather around the surgeon. Instead, they could observe and hear everything happening in the operating room from the surgeon’s precise first-person perspective at a distance, while also asking questions and receiving responses in real time.
This opens up new directions for surgical education. Traditionally, surgical training for medical students and practicing physicians has largely depended on their access to the operating room, where they could learn new surgical techniques from top experts; however, such opportunities are often quite limited. The application of VR technology brings a completely different experience to medical education. In this regard, GIBLIB, a streaming video library for medical education, is one such example.
GIBLIB offers a rich library of 4K high-definition video resources, including medical teaching materials, lectures and conference content, and surgical procedures, earning it the nickname “the Netflix of medical education.” GIBLIB records real surgeries in 4K, 360-degree VR format, providing precise camera angles from the surgeon’s perspective as well as 360-degree panoramic views. Users need only don VR headsets to immerse themselves in authentic operating room activities. GIBLIB also provides Continuing Medical Education (CME)-accredited content in 360-degree VR format for surgeons and healthcare professionals. CME accreditation refers to educational activities that help medical practitioners maintain and enhance their professional skills, knowledge, and competitiveness; in some U.S. states, it is a mandatory requirement for many healthcare providers to maintain their licensure. Traditionally, medical professionals have even needed to travel worldwide to attend medical conferences to obtain such certifications, imposing significant demands on time, money, and location. With GIBLIB’s online video library, however, users can engage in immersive learning using just their smartphones and computers.
GIBLIB Official Website
Osso VR Surgical Training Platform
While also leveraging VR to achieve educational and training objectives, Osso VR has pioneered a distinct model. Unlike GIBLIB, which streams real-world filmed videos into VR headsets, Osso VR employs highly realistic visual technology to recreate an operating room environment for trainees, surgeons, and medical device specialists within the VR space. This allows users to practice hands-on surgical procedures in a virtual setting using VR controllers. The Osso platform delivers exceptional visual fidelity, ensuring that every aspect—from anatomical details to the operating room environment—mirrors reality.
Screenshot of Osso VR Official Website Demo
Osso’s founder, Justin Barad, was an orthopedic surgeon and software developer who established Osso VR after years of experience in the operating room. He once stated, “New medical devices are constantly emerging, and we have genuinely found ourselves using Google to search for instructions while the patient was already on the operating table. Learning curve data indicate that one must perform a new surgical procedure 100 times to achieve proficiency. Therefore, for safety reasons, we typically avoid using new devices, as we lack the time to thoroughly master all aspects of their use.”
Traditional training methods have limited the speed at which surgeons can master new techniques and medical devices, while also lacking effective means for providing objective feedback and measurement of surgical performance. With Osso VR, physicians can schedule realistic practice sessions for surgeries and device handling anytime and anywhere. Osso also analyzes the precision of doctors’ simulated training, tracks performance, and enables users worldwide to join collaborative training remotely. A study conducted by the David Geffen School of Medicine at UCLA found that Osso VR improved participants’ surgical performance by 230% compared with traditional training methods. Currently, orthopedic industry giants such as Johnson & Johnson, Stryker, and Smith & Nephew have partnered with Osso VR as their virtual reality training provider.
According to incomplete statistics from VCBeat, medical teaching and surgical training platforms and systems are among the most frequently targeted scenarios for domestic VR/AR healthcare companies. These solutions cover disciplines such as obstetrics, traditional Chinese medicine acupuncture, and embryo transplantation, with key players including Yuyuan Technology, Yuran Intelligence, Zhichu Computer, Yiweixun, Zhonghui Technology, Cube Fantasy, and Chuhuan Technology.
The first recorded instance of robot-assisted surgery occurred in 1985, when the PUMA 560 robotic surgical arm was used for a delicate neurosurgical biopsy. Two years later, the first laparoscopic procedure involving a robotic system—a cholecystectomy—was performed. By 2000, the da Vinci Surgical System became the first surgical robotic platform approved by the FDA for general laparoscopic procedures, thereby opening new frontiers in robotic surgery. Today, surgical robots are more widely recognized and commonly used by physicians due to advantages such as smaller incisions, higher precision, and greater operational flexibility, while also holding significant potential for market penetration. So, when surgical robotics converges with VR, will this become the developmental direction for the next generation of surgical robots?
Founded in 2014, Vicarious Surgical is developing surgical robots that combine humanoid robotic arms with virtual reality technology, enabling surgeons to achieve 360-degree visual access. The company initially focused on refining its surgical robot design, allowing its robotic arms to enter the abdominal cavity through an incision as small as 1.5 centimeters or even smaller, while moving freely in all directions. Each robotic arm is equipped with 28 sensors capable of measuring force, motion, and positioning, thereby fully replicating the surgeon’s natural movements from the shoulder and elbow to the wrist. This ensures maximized surgical precision, visual imaging, and control. By wearing VR headsets and using handheld controllers, surgeons can experience an immersive, 360-degree panoramic view, creating the sensation of directly entering the patient’s abdominal cavity.
Screenshot of Vicarious Surgical's Official Website Example
Notably, Vicarious Surgical’s surgical robot has only received the FDA Breakthrough Device designation and has not yet completed its final market launch; furthermore, VR equipment will not be included in the first-generation product. Whether the envisioned integration of surgical robots and VR will be widely accepted by surgeons remains uncertain.
Compared with VR-assisted surgery, the application of AR technology may be more practical and feasible. In interventional procedures, surgeons typically rely on intraoperative X-ray, CT, and surgical navigation systems to determine the course of the operation, requiring them to look up at a separate screen to view imaging results. With the aid of AR devices, navigational images and anatomical structures can be projected in real time directly into the surgeon’s field of view. This not only allows surgeons to focus more on the procedure and the patient, making it immediately clear where surgical instruments are located anatomically and what the next steps should be, but also reduces radiation exposure risks for both medical staff and patients.
Augmedics’ xvision Spine System (XSV) is the first FDA-approved augmented reality (AR) navigation system for use in surgery. By wearing AR headsets, surgeons can directly visualize the patient’s 3D spinal anatomy, as well as the position and trajectory of surgical instruments and implants, in their field of view—akin to performing a CT scan with the naked eye—while avoiding shifts in attention. The XSV system achieves a percutaneous implant placement accuracy of 98.9%.
Screenshot from Augmedics' Official Website Promotional Video
In China, Weizhuo Zhiyuan’s holographic imaging system, Miaozhi Technology’s VR imaging platform, and Lin Yan Medical’s AR orthopedic surgical navigation system are all representative examples in this scenario.
If there is any medical field that has proven the use of VR as a therapeutic tool, then the digital therapeutics in the two areas of VR-enabled mental health and chronic pain management can be said to have set a good precedent for other fields.
VR technology recreates 3D environments, where visual, auditory, and even sensory interactions in virtual reality induce a sense of being in different locations, times, or even alternate realities. By constructing such immersive and interactive environments, VR can alter individuals’ attention. Academic research over the past three decades has demonstrated that this shift in attention plays a significant role in managing severe conditions such as pain and anxiety. For instance, among patients undergoing chemotherapy or burn wound care, immersive VR experiences facilitate treatment procedures, reducing both their perceived pain and subjective perception of treatment duration. In the realm of mental health, virtual reality also helps alleviate stress and anxiety. Exposure therapy is often a key approach in treating anxiety disorders, phobias, and post-traumatic stress disorder (PTSD). VR-based exposure therapy provides patients with the opportunity to immerse themselves in fear- or anxiety-inducing environments, thereby enabling psychotherapists to deliver more effective treatment.
For such therapies, AppliedVR, a chronic pain VR service provider, prefers the term “immersive therapy.” Its VR-based product for chronic lower back pain, EaseVRx, is the first FDA-approved virtual reality therapy. In other areas, Luminopia One’s VR-based treatment for pediatric amblyopia has also received FDA marketing clearance. BehaVR combines meditation, exposure therapy, and cognitive behavioral therapy with VR and AR to offer treatment solutions for anxiety management, postpartum health, and pain management. TRIPP uses games within VR and AR environments to guide users through exercises that follow auditory frequencies and visualized breathing rhythms, addressing mental health and emotional well-being.
Medical innovation enterprises in China are also leveraging VR and AR technologies for disease treatment, including applications in rehabilitation, visual function training, and mental health therapy. Such companies include 7invensun, Noitom, Xinjing Technology, Shuangqi Medical, Gelun Medicine, and Yikang Medical.
In addition to the digital health companies mentioned above that leverage VR/AR technologies to create metaverse-style platforms, another major scenario category within the medical metaverse is digital twins. A digital twin is a virtual mapping, or “twin,” of real-world entities—such as organs, individuals, or patient populations—that can be used to support medical decision-making. Digital twins represent a form of synthetic data, an AI-derived information framework that models real-world entities and typically maintains a continuous connection with their physical counterparts. This hybrid connectivity places digital twins directly within the metaverse.
Siemens Healthineers is pioneering digital twins of the heart—complex digital simulations that reflect an individual patient’s cardiac molecular structure and biological functions. Physicians can simulate how a patient’s heart responds to medications, surgeries, or catheter-based interventions, enabling them to test various treatment options in advance before making any real-world clinical decisions. Virtonomy has developed digital twins of bones and muscle groups to facilitate improved implant design and to simulate how medical devices or implants degrade within the patient’s body over time.
Beyond individual organs, startups are also creating whole-body digital twins of individuals. Q Bio’s Gemini platform can generate complex simulations of a patient’s entire anatomy and physiology by integrating full-body scans, vital signs, medical history, and genetic test results. Consumers can share these insights with experts, trainers, nutritionists, and researchers to enable personalized care. Whenever new test or scan results are shared, the consumer’s digital twin is updated accordingly. AI-powered telehealth company Babylon Health is also building exclusive digital twins for users based on their health data, while Twin Health has developed digital twins focused on metabolism. These tools allow individuals and their healthcare providers to plan lifestyle changes that help prevent or reverse metabolic diseases. Clinical research results published in the American Diabetes Association’s journal *Diabetes* show that Twin Health’s digital twin technology enabled 92% of participants to discontinue diabetes medications, with over 90% achieving remission of type 2 diabetes.
Screenshot of Q Bio's Official Website Example
Since digital twins can simulate individual health data and medical conditions, the next step is for healthcare professionals and researchers to use them to model population responses to disease outbreaks or new drugs, thereby paving the way for clinical trials in the metaverse. For example, Unlearn uses machine learning models to create digital twins of clinical trial participants. Integrating their prognostic information into randomized controlled trials not only accelerates enrollment and reduces monitoring requirements but also maintains rigorous evidence standards as well as the randomization and blinding integrity of the study.
Regarding the metaverse, some are firmly convinced of its potential, others dismiss it with contempt, some feel inspired by its world, others oppose its arrival, and still others maintain a cautious watch. But one thing is certain: before we truly choose to embrace (or not embrace) it, there are many issues that need to be sorted out. The first is definition—what the metaverse represents, whether it can become a reality or has already arrived, who owns the metaverse, and how it should be built, among other questions.
As a staunch advocate of the metaverse, Mark Zuckerberg stated during an earnings call with investors, “While our direction is clear, the path forward does not appear to be fully defined.” Annual reports revealed that Meta’s exploration of the metaverse had already driven operating losses in its Reality Labs division—which focuses on consumer hardware, software, and content related to augmented and virtual reality—to over $10 billion in 2021, against revenues of $2.3 billion. Following the release of the annual report, Meta’s stock price plummeted by 27%. Beneath the hollow facade, investors remain wary of Zuckerberg’s metaverse ambitions.
What stands out more clearly than the nebulous concept itself are the underlying technologies on which the metaverse will rely. It is certain that, regardless of whether the concept of the metaverse prevails, the development of VR/AR, artificial intelligence, machine learning, computer modeling, 3D imaging, and related fields will only accelerate. In the healthcare sector, these cutting-edge technologies are already driving the advancement of precision medicine. Creating more advanced products by strategically combining these technologies depends on the technical innovation and imagination of scientists and healthcare professionals, and human imagination should never be underestimated. Recognizing the growing number of medical devices and therapies employing extended reality (XR) technologies, the FDA’s Center for Devices and Radiological Health (CDRH) has established the Medical Extended Reality (MXR) Program to conduct regulatory research on such devices, ensuring alignment with the rapidly evolving industry. Focusing on breakthroughs in underlying technologies and matching them with technological needs represents the most pragmatic approach to realizing the healthcare metaverse.
Third, the establishment of an ecosystem. Although there is currently no unified consensus on the metaverse, the party that can lead the construction of the underlying ecosystem early on will have a say and market space in the infrastructure construction of the metaverse. NVIDIA launched the public beta version of Omniverse (the "Metaverse for Engineers") in 2020. Engineers, designers, and others collaborate on Omniverse to build and exchange 3D assets and scene data, construct and render virtual worlds, and enable access to these virtual environments. After attracting 100,000 users during the public beta phase, NVIDIA announced last November that Omniverse would be freely available to all creators using NVIDIA Studio. In the medical field, IT companies are perhaps the most likely to become builders of the metaverse ecosystem, creating an underlying ecosystem that aligns with the needs and data characteristics of the healthcare industry.
Furthermore, issues regarding the necessity, accessibility, and data privacy of the medical metaverse remain. While the metaverse in healthcare scenarios may offer patients more precise and efficient treatment plans, it also implies that such solutions will favor those with access to extended reality (XR) and internet resources, potentially exacerbating inequalities in healthcare resource distribution. Faced with more cost-effective alternatives and economically viable hardware and software already available on the market, investors, healthcare professionals, entrepreneurs, policymakers, and others are likely to question its necessity. It is also worth carefully considering how to define, regulate, and protect patient data and privacy within interactions in the virtual world.
Science fiction writers are arguably the most prescient futurists in the world, yet it is the continuous innovation of science and technology that serves as the medium to turn such prophecies into reality. Without the metaverse, there might still be a “Garden” universe or a “Circular” universe, but none of these would represent the endpoint of scientific innovation, for scientific innovation has no end. The pioneers—the first to dare—will face skepticism, but they will also garner corresponding attention. It is the fate of innovators to steadfastly follow the path they believe in.