NVIDIA's Hardware-Software Synergy and Self-Evolution Powering the Next Generation of AI in Healthcare

In recent years, AI technology has been reshaping the cognitive boundaries of the healthcare industry at an unprecedented pace. Particularly in 2022, China’s regulatory policies for AI-based medical devices achieved significant breakthroughs, leading to a record high in the approval of AI medical imaging devices. Throughout 2022, the National Medical Products Administration (NMPA) issued more than 20 Class III certificates for AI medical devices, marking the year with the highest number of such approvals to date and accounting for nearly half of all currently approved AI Class III certificates.

The continuous improvement of various policies, including regulatory frameworks, has undoubtedly provided a solid foundation for the advancement of AI in healthcare. Meanwhile, as a representative of digital health, technological development is also indispensable in driving the progress of AI healthcare. Advancements in these areas rely on the collective efforts of the entire industry ecosystem.

In summary, the core elements determining the differentiation of AI products primarily include data, algorithms, and computing power. As a leading global provider of AI computing infrastructure, semiconductor giant NVIDIA continuously intensified its efforts in the healthcare sector in 2022, launching multiple solutions tailored for AI-driven healthcare. These solutions are expected to be gradually integrated into various scenarios where AI and healthcare converge in the future.

Through successive generations of product improvements, NVIDIA has continuously provided cost-effective, high-performance computing solutions to the AI industry, helping to effectively deploy these computational resources into practical application scenarios. This has driven significant progress in AI applications in recent years and established NVIDIA’s hardware and software solutions as one of the most critical “infrastructure” components of the artificial intelligence industry.

Nevertheless, NVIDIA has not relaxed its efforts in self-iteration. Taking AI accelerator cards for data centers as an example, NVIDIA had already secured an absolute market share in data center AI acceleration with its previous two generations of data center GPUs. However, at GTC 2022, NVIDIA unveiled a new generation of data center accelerator cards, which, through deployment by cloud service providers in data centers, are expected to elevate global cloud-based AI computing power to an entirely new level.

Certainly, in addition to the enhancement of cloud-based AI computing power, the improvement of edge AI computing power may be more readily perceived. With the rapid proliferation of the Internet of Things (IoT), the practical implementation and integration of artificial intelligence with IoT will undoubtedly propel human society into an era of “intelligent interconnectivity of everything,” leading to an explosive surge in data generation. This deluge of data places immense pressure on existing network bandwidth and poses significant challenges to traditional cloud-based AI acceleration.

The good news is that, powered by the robust computational capabilities enabled by cloud-based AI acceleration, artificial intelligence and machine learning have made tremendous progress. This has provided the prerequisite for continuously adapting and compatibilizing network architectures and tools—such as those used for machine learning and neural network training—to embedded systems. An increasing number of AI applications can now run directly on edge devices, making edge AI a current development trend.

Edge AI refers to AI algorithms that are processed locally on hardware devices, enabling data processing without an internet connection. This means that operations such as data generation can be performed without the need for streaming or storing data in the cloud. To achieve these objectives, Edge AI leverages deep learning in the cloud to generate data, while model inference and prediction are executed on the device itself (at the edge).

Compared with cloud-based AI acceleration, edge AI acceleration offers at least several advantages in terms of bandwidth, latency, cost-effectiveness, reliability, and privacy.

First, edge AI can reduce network bandwidth requirements. Since edge devices process a portion of the generated transient data, there is no longer a need to upload all data to the cloud, which significantly alleviates pressure on network bandwidth and reduces the demand for computational and storage resources.

Second, edge AI processes data near the source, significantly reducing system latency and improving service response times. This is critically important for latency-sensitive application scenarios, such as autonomous driving.

Third, edge AI offers better cost-effectiveness in specific scenarios. Even if technical solutions can address bandwidth and latency issues to enable cloud-based AI acceleration, performing computations at the edge may be more economical.

Fourth, edge AI offers superior reliability. Given that cloud network connections are not always reliable, edge AI is clearly more suitable for scenarios requiring continuous operation. For instance, smart door locks feature facial recognition for unlocking. Users naturally expect this function to remain operational even when the network is disconnected.

Fifth, edge AI can provide the infrastructure for the storage and use of critical privacy-sensitive data, enhancing data security and thereby addressing privacy concerns in specific applications.

For this very reason, edge AI has become a fiercely contested battleground in recent years. Data from ABI Research shows that the market size for edge AI accelerator chips is projected to reach $12.2 billion by 2025, surpassing the $11.9 billion market size for cloud-based AI accelerator chips.

At GTC22, NVIDIA unveiled the IGX platform for high-precision edge AI, delivering advanced, proactive safety capabilities to industries such as healthcare and enhancing human-machine collaboration. The IGX platform provides secure, low-latency AI inference to meet clinical demands for real-time data processing from a range of medical devices and sensors during medical procedures, such as robot-assisted surgery and patient monitoring systems.

NVIDIA IGX Edge AI Platform (Image provided by NVIDIA)

The IGX platform is a powerful combination of hardware and software. In addition to the IGX Orin, a robust, compact, and energy-efficient AI supercomputer hardware solution, it also provides support for a range of software solutions, such as Clara Holoscan, a real-time AI software solution designed for medical devices. It empowers medical device developers to integrate edge computing, on-premises data centers, and cloud services, enabling the rapid development of new software-defined devices that bring the latest AI applications directly into the operating room.

Currently, three leading medical device startups—Activ Surgical, Moon Surgical, and Proximie—have chosen to leverage the combined power of IGX and Clara Holoscan to support their surgical robotic systems.

For example, Activ Surgical leverages IGX and Clara Holoscan to accelerate the development of its AI-powered VR/AR solutions for real-time surgical guidance. This U.S.-based company uses augmented reality technology to enable surgeons to visualize critical physiological structures and functions, such as blood flow, that are invisible to the naked eye, and integrates this information into surgical imaging systems, thereby reducing the incidence of surgical complications, improving patient care, and enhancing patient safety.

French company Moon Surgical is designing Maestro, an easy-to-use, adaptive surgical assistance robotic system that integrates seamlessly with existing medical devices and workflows in the operating room. With the support of NVIDIA IGX and Clara Holoscan, Maestro’s imaging pipeline, management system, and hardware design engineering cycle was shortened by at least six months, allowing the company to redirect valuable engineering resources toward artificial intelligence algorithms and other unique features.

UK-based Proximie is building a telepresence platform to enable real-time remote collaboration among surgeons. The combination of IGX and Clara Holoscan allows it to process local video in the operating room, enhancing performance for users while safeguarding data privacy and reducing cloud computing costs. To date, Proximie has been deployed in more than 500 operating rooms worldwide and has recorded tens of thousands of surgical procedures.

Including these three companies, more than 70 medical device manufacturers, startups, and medical centers are currently leveraging Holoscan to drive the deployment of AI applications in clinical settings and transform medical devices into Software-as-a-Service (SaaS) business models. Undoubtedly, supported by the NVIDIA IGX platform, the application of edge AI in the healthcare sector is poised for an explosive growth, mirroring the earlier boom in AI-powered medical imaging.

Among NVIDIA’s decades of successful experience, software solutions that enhance hardware have been a key weapon in its competitive advantage over rivals. For this reason, NVIDIA has always placed high importance on its software ecosystem and introduced the concept of building an “AI foundation” in 2021. The NVIDIA AI Enterprise platform (NVAIE) is the outcome of this initiative.

NVAIE is designed to address the challenges enterprises face in AI application development by providing a comprehensive toolchain, enabling businesses to build and deploy AI applications efficiently and securely. This toolchain should include model deployment tools, model management platforms, model monitoring tools, and data privacy protection tools, thereby helping enterprises better manage and control the AI application development process to ensure the availability and reliability of AI applications.

With continuous iterative updates, the newly released NVAIE 3.0 has finally come close to achieving this goal in terms of functionality. This one-stop AI development platform, akin to an operating system, enables the rapid creation of AI applications, covering the entire lifecycle from model training, inference optimization, and deployment to model management and cloud-native management. AI applications that previously took months to develop can now be completed in just a few hours on the NVAIE 3.0 platform.

To accelerate AI application development efficiency and enhance the performance of final AI applications, NVAIE 3.0 also includes a large number of pre-trained models with unencrypted and fully open weights, available for direct user access.

In addition to NVAIE 3.0, the Clara platform, specifically designed for healthcare scenarios, was launched as early as 2018. NVIDIA has continuously optimized and expanded the platform to strengthen its footprint in the healthcare sector. Initially, Clara served solely as a software development toolkit for medical imaging AI researchers, aiming to standardize imaging data and accelerate AI training.

Subsequently, through collaboration with industry partners, Clara began to expand into genomics. After all, the genome represents a far larger data source; processing billions of base pairs requires more ideal computing power sources to ensure that experiments remain cost-feasible.

As NVIDIA deepens its understanding of healthcare application scenarios, an increasing number of medical industry solutions are being integrated into the Clara platform. Just as “GeForce” established NVIDIA’s initial prominence in the gaming industry, the company clearly aims to tightly associate “Clara” with healthcare. Positioned as an intelligent computing software platform for medical developers, Clara provides efficient and user-friendly data analysis tools for pioneers seeking to explore the healthcare sector.

At GTC22, NVIDIA announced new progress—it will collaborate with the Broad Institute of MIT and Harvard to provide AI algorithms and acceleration tools required for rapid analysis of massive medical data on the Broad Institute’s Terra cloud platform.

As a cloud platform jointly developed by the Broad Institute, Microsoft, and Verily, Terra enables biomedical researchers to securely share, access, and analyze data at scale. Currently, the platform includes more than 25,000 biomedical researchers from academia, startups, and large pharmaceutical companies, all of whom will benefit from this collaboration.

According to the disclosure, this collaboration will focus on the following three key areas.

First, NVIDIA will provide Clara Parabricks, a GPU-accelerated software suite for secondary analysis of sequencing data, on the Terra cloud platform. It can significantly reduce the time required for genomic analysis to just over one hour, compared to the 24 hours needed in previous CPU-based Clara environments. Additionally, Clara Parabricks can reduce the cost of whole-genome sequencing analysis by 50%.

Second, NVIDIA also released the BioNeMo framework for training and deploying supercomputing-scale large biomolecular language models (LLMs), helping scientists better understand diseases and find treatments for patients. The BioNeMo framework will support chemical, protein, DNA, and RNA data formats, and it is also part of the Clara Discovery drug development framework, applications, and AI model suite.

Schematic Diagram of the NVIDIA BioNeMo Framework Application (Image provided by NVIDIA)

Third, NVIDIA is also committed to building new deep learning models for the industry-standard GATK toolkit from the Broad Institute, which is used by over 100,000 researchers, to help them identify disease-associated genetic variants. This will empower new drug developers in their research into novel therapies.

This collaboration is poised to elevate biomedical cooperative research to an entirely new level by connecting researchers with one another through an open cloud platform, and linking them to the datasets and tools necessary for achieving scientific breakthroughs.

In addition, users of the Terra platform can also access MONAI, an open-source deep learning framework for medical imaging AI, and RAPIDS, a GPU-accelerated data science toolkit that can accelerate data preparation for genomic single-cell analysis.

When it comes to MONAI, this open-source AI development framework is a crucial component of the Clara ecosystem in model building. MONAI features automated annotation tools to assist developers in labeling data, and enables automated model selection and hyperparameter tuning. Additionally, MONAI supports self-supervised learning, allowing models to be trained on unlabeled data, thereby reducing annotation time.

Furthermore, MONAI has been specifically optimized to address the unique requirements of medical data, enabling it to handle formats, resolutions, and metadata specific to medical images. Developers can leverage its specialized data transformations, neural network architectures, and evaluation methods tailored for the healthcare domain to assess the quality of medical imaging models.

Due to its open-source nature and ease of use, MONAI has been well received since its launch, with downloads surpassing 650,000—a significant increase from the 50,000 monthly downloads recorded in February 2022.

MONAI’s capabilities continue to be further enhanced—NVIDIA released the MONAI Application Package (MAP) at the Radiological Society of North America (RSNA) annual meeting in December 2022, which will make it easier for MONAI to integrate models into clinical workflows.

In the past, deploying several AI models in the imaging department to assist experts in identifying more than a dozen different conditions or to enable semi-automated generation of medical imaging reports required substantial time and resources to secure appropriate hardware and software infrastructure for each model. While this approach was “possible,” it was not “practical.”

The MAP provided by MONAI Deploy is an AI model packaging method that can significantly simplify this process. If developers package an application using MAP, hospitals can easily run the application locally or in the cloud. Meanwhile, the MAP specification also integrates healthcare informatics standards, such as the DICOM standard for medical imaging interoperability.

Currently, healthcare institutions, academic medical centers, and AI software developers around the world are adopting MAP.

For example, Cincinnati Children’s Hospital Medical Center in the United States is developing Model Access Packages (MAPs) for an AI model capable of automatically segmenting total heart volume from CT images, thereby supporting pediatric heart transplant patients through a project funded by the National Institutes of Health. Additionally, the University of California, San Francisco is also developing MAPs for several AI models, including those for hip fracture detection, liver and brain tumor segmentation, and classification of knee joint conditions and breast cancer.

Qure.ai, which has developed AI models for medical imaging in use cases such as lung cancer, traumatic brain injury, and tuberculosis, is using MAP to package solutions for deployment, thereby accelerating their clinical impact. SimBioSys has created 3D virtual representations of patients’ tumors and leverages MAP for precision medicine AI applications that help predict how patients will respond to specific treatments.

Furthermore, prominent cloud service providers such as Amazon, Google, Microsoft, and Oracle are progressively integrating MONAI Application Packages (MAPs) to empower researchers and enterprises adopting MONAI Deploy. This integration enables them to run AI applications on their own platforms via containerized or native application integration, thereby delivering enhanced value to users.

It is evident that, across both hardware and software, NVIDIA has maintained close collaboration with the industry, continuously gaining insights into sector needs and refining its offerings based on feedback. Through iterative self-improvement, NVIDIA continues to enhance its AI solutions, thereby boosting the performance and efficiency of AI in medical applications while reducing costs. This will further solidify NVIDIA’s position within the AI healthcare ecosystem.

References:

THU Data Pie: [Original] A Comprehensive Guide to Edge Computing and Edge AI