NVIDIA Unveils Clara Holoscan AI Computing Platform for Real-Time Sensing in Medical Devices

Technological innovations in AI-powered medical devices provide healthcare professionals with enhanced decision-support tools, offering assistance in areas such as robot-assisted surgery, interventional radiology, and radiotherapy planning.

To achieve this in clinical applications, AI medical devices must feature an accelerated pipeline for real-time data processing, prediction, and visualization. NVIDIA Clara Holoscan is a new computing platform for the healthcare industry, built on NVIDIA AGX Orin, that provides the necessary computational infrastructure for medical devices requiring scalable, software-defined, end-to-end streaming data processing.

Image from NVIDIA

As an end-to-end platform that seamlessly connects medical devices to edge servers, NVIDIA Clara Holoscan empowers developers to create AI microservices for running low-latency streaming applications on devices while offloading more complex tasks to data center resources.

Almost every smart medical device follows a similar processing workflow. This process begins with sensors, proceeds through the data domain, and culminates in visualization to facilitate human decision-making. The computational requirements at each stage of the workflow vary depending on the specific device used, such as CT scanners, endoscopes, or ICU room cameras.

NVIDIA Clara Holoscan can accelerate each of the following stages:

1. High-Speed I/O: NVIDIA GPUDirect RDMA enables direct data streaming to GPU memory via NVIDIA ConnectX smart network interface cards or third-party PCI Express cards, facilitating ultra-low-latency downstream processing.

2. Physical Processing: After data is transmitted to the GPU, CUDA-X and the NVIDIA Triton Inference Server accelerate physics-based computations or AI processing, converting sensor data into the image domain. Examples include image reconstruction in X-ray and CT, or beamforming in ultrasound.

3. Image Processing: Input image data into AI models using NVIDIA Triton for detection, classification, segmentation, or object tracking.

4. Data Processing: Using the NVIDIA cuCIM library to combine sensor image data streams with previously acquired images, developers can register or enhance the data using supplementary information such as electronic health records.

5. Rendering: Through the Clara rendering server, developers can visualize device data and prediction data in 3D in real time, perform interactive cinematic-quality rendering using NVIDIA Omniverse, or enhance reality using CloudXR. For example, this provides clinicians with improved images of organ or tumor segmentation.

As a scalable architecture, Clara Holoscan can extend from medical devices to NVIDIA-certified edge servers, and further to NVIDIA DGX systems in data centers or the cloud. Developers can use this platform to flexibly add or reduce computing and input/output capabilities in their medical devices as needed, thereby balancing the requirements for latency, cost, space, performance, and bandwidth.

Many medical device companies are integrating AI and robotics, leveraging NVIDIA’s accelerated computing platform in robotic surgery, mobile CT scanning, and bronchoscopy.

NVIDIA Clara Holoscan is designed to help device manufacturers scale vertically from devices to data centers, and achieve horizontal scaling by leveraging NVIDIA’s extensive AI solutions, thereby better supporting such applications.

To accelerate the development of real-time medical devices with multi-sensor inputs, the Clara Holoscan platform supports I/O cards from members of the NVIDIA Inception AI and data science startup accelerator program, including: AJA Video Systems—video capture cards for endoscopy and surgical visualization applications; KAYA Instruments—video capture cards for microscopy and scientific imaging instruments; and us4us—research-grade front-end devices for developing software-defined ultrasound solutions.

Verasonics, a leader in front-end hardware for ultrasound research, will enable Clara Holoscan to stream data directly to NVIDIA GPUs using high-speed networking technologies.

With Clara Holoscan, developers can customize applications to run as a series of modular microservices on devices and servers. As Clara Holoscan is software-defined, medical device companies can continuously upgrade and improve their solutions over time.

The Clara Holoscan SDK supports this effort through accelerated libraries, AI models, and reference applications for ultrasound, digital pathology, and endoscopy, helping developers leverage embedded and scalable hybrid cloud computing. With an end-to-end deployment platform, enterprises can more easily upgrade applications, bringing new research breakthroughs to daily medical practice. Developers can visitNVIDIA Clara Holoscan Website。

To learn more about the applications of AI in healthcare, please watch the onlineNVIDIA GTC Conference(closed on November 11). Watch NVIDIA founder and CEOJensen Huang’s GTC Keynote Live Stream and Replay on November 9Watch the presentation by Kimberly Powell, Vice President of Healthcare at NVIDIA, delivered at 2:30 a.m. (Beijing Time) on November 10Special Speech on Healthcare。