KeYi Medical's Deep Learning System Achieves Over 90% Accuracy in Differentiating COVID-19 from Community-Acquired Pneumonia on Chest CT, Published in Radiology

Starting from the lockdown of Wuhan, within just one week, AI-assisted lung algorithms developed by various companies for COVID-19 were successively completed and deployed in frontline hospitals, including Wuhan Central Hospital and Wuhan Union Hospital.

However, in terms of functionality, during the pandemic, many AI systems were primarily tasked with assisting physicians in reviewing patients’ CT scans and identifying abnormal areas in the lungs, but they struggled to further differentiate which specific type of pneumonia a patient had.

This is a highly intriguing question, concerning the capacity of medical imaging to aid in the detection of infectious diseases such as COVID-19. Previous studies have applied deep learning to pediatric chest X-rays, demonstrating that this technology can detect and differentiate between bacterial and viral pneumonia in children. However, in the field of CT imaging, there is still a lack of large-scale validity studies and high-quality publications addressing these issues.

Prompted by this issue, many scholars have launched their own research. Some have attempted to start with lung segments, leveraging differences in lung segment parameters across various types of pneumonia to classify the disease; others have sought to integrate etiologic data, correlating pathogen test results one-to-one with imaging findings. Recently, a paper titled “Artificial Intelligence Distinguishes COVID-19 from Community Acquired Pneumonia on Chest CT,” which relies entirely on deep learning analysis of chest CT images to differentiate COVID-19, was accepted by Radiology, a top-tier international radiology journal. The algorithm employed in this study appears to have identified key CT imaging features characteristic of COVID-19.

In brief, this study was a retrospective analysis of CT images from patients with COVID-19. The researchers developed a specialized 3D detection neural network and applied it to analyze 4,356 CT datasets. The results showed that the algorithm achieved a sensitivity of 89.76% and a specificity of 95.77% for differentiating COVID-19, with an area under the receiver operating characteristic (ROC) curve (AUC) of 0.96.

This paper was jointly researched and published by Keya Medical, a medical AI imaging company, in collaboration with six domestic hospitals. As the first enterprise in China to obtain a Class III medical device registration certificate for artificial intelligence, Keya Medical has been dedicated to the practical application of big data and AI technologies in the healthcare sector. Its certified product, "DeepVessel FFR," has been officially recognized by the National Medical Products Administration (NMPA) as having "significant economic benefits and social value." Compared with similar CT-FFR products both domestically and internationally, "DeepVessel FFR" demonstrates leading performance across all key metrics.

Keya Medical stated to VCBeat: “The significance of this paper lies in its pioneering use of deep learning-based technology to conduct a three-class classification study on chest CT images for Coronavirus Disease 2019 (COVID-19), Community-Acquired Pneumonia (CAP), and Non-Pneumonia, yielding exciting results. Previously, researchers had conducted studies on pneumonia classification using two-dimensional X-ray chest radiographs, while the classification of pneumonia on three-dimensional CT lung images remained unexplored. During the national and global fight against COVID-19, Keya Medical developed COVNet, a deep learning neural network that extracts various imaging features from lung CT scans to differentiate COVID-19. Furthermore, heatmaps were employed to visualize the critical regions influencing the model’s decisions, thereby highlighting the key pathological areas identified by the model.”

Validated by data from multiple medical centers, the COVNet network model demonstrates high sensitivity and specificity in detecting COVID-19, community-acquired pneumonia (CAP), and non-pneumonia cases. For COVID-19 detection, the area under the receiver operating characteristic (ROC) curve reaches as high as 0.96. This paper initiates research on multi-class classification of pneumonia in lung CT images based on deep learning technology, providing a framework for rapid AI-based imaging screening of infectious pneumonias such as COVID-19, SARS, and MERS, with the hope of contributing to the global fight against COVID-19.

This study collected 4,536 three-dimensional chest CT scan images from 3,506 patients across six hospitals. After excluding contrast-enhanced CT scans and examinations with a single-slice thickness greater than 3 mm, the study constructed a dataset comprising 4,356 three-dimensional chest CT scan images from 3,322 patients. The mean age of these patients was 49 ± 15 years, including 1,838 males and 1,484 females. In this dataset, 1,296 cases (30%) were images from patients infected with COVID-19 (all confirmed positive by RT-PCR testing); 1,735 cases (40%) were from patients with community-acquired pneumonia (CAP); and 1,325 cases (30%) were from non-pneumonia patients.

All examination data were randomly split into a patient training set and an independent test set at a 9:1 ratio. The training dataset was then further divided at a 9:1 ratio for model training and internal validation. The independent test set was not used for training or internal validation.

To minimize the potential for experimental bias, these data were sourced from six different hospitals, with the enrollment period restricted to August 16, 2016, through February 17, 2020; specifically, the COVID-19 subset included only data collected between December 31, 2019, and February 17, 2020.

Due to variations in CT equipment manufacturers across different hospitals, all images used in this study were exported from DICOM files. The CT slice thickness was controlled within the range of 0.5 mm to 3 mm, with an image matrix size of 512 × 512 and an in-plane resolution of 0.29 × 0.29 mm.²to 0.98*0.98 mm²between.

Fig. 1: Algorithm Flowchart

Fig. 1 is the overall flowchart of the algorithm proposed in this paper, which mainly consists of four steps:

Step 1: Construction of the model training and testing databases. Process according to the data description above.

The second step is dataset preprocessing, which includes data augmentation and segmentation of the lung region. This paper employs a U-Net-based network for lung region segmentation to eliminate the influence of non-lung areas on subsequent detection algorithms.

Step 3: Input the preprocessed dataset into the COVNet network to train the deep learning algorithm.

Step 4: Preprocess the independent test set (lung region extraction) and input it into the COVNet model trained in Step 3 for predictive classification.

Keya Medical named this 3D deep learning framework for COVID-19 detection COVNet. This convolutional neural network, with ResNet50 as its backbone, can simultaneously extract 2D local and 3D global representative feature maps from input CT images (as shown in Fig. 2). COVNet uses max pooling layers and fully connected layers to comprehensively analyze these feature maps, ultimately generating probability scores for each category (COVID-19, community-acquired pneumonia [CAP], and non-pneumonia).

Fig. 2: Framework diagram of COVNet, a neural network for COVID-19 detection

COVID-19: COVID-19; CAP: Community-Acquired Pneumonia; Non-Pneumonia: Other Non-Pneumonia Conditions

Statistical data show that when COVNet is deployed on a workstation (NVIDIA Quadro M4000 8GB GPU, 16GB RAM, and Intel Xeon Processor E5-1620 v4 @ 3.5GHz), the average time required for pneumonia prediction per CT examination is 4.51 seconds, which is substantially faster than the reading speed of physicians for individual CT image sets.

In terms of diagnostic accuracy, the algorithm achieved a sensitivity and specificity of 90% and 96%, respectively, for COVID-19 detection; 87% and 92%, respectively, for community-acquired pneumonia (CAP) detection; and 94% and 96%, respectively, for non-pneumonia imaging. Detailed results are presented in Table 1.

Table 1: Performance of the Deep Learning Framework COVNet on an Independent Test Set

To enhance model interpretability, Keya Medical employs the Gradient-weighted Class Activation Mapping (Grad-CAM) method to visualize the critical regions that drive decision-making in deep learning models. The following three images (Fig. 3a, Fig. 3b, Fig. 3c) present heatmaps highlighting the specific areas guiding the deep learning algorithm’s decisions for COVID-19, community-acquired pneumonia (CAP), and non-pneumonia cases, respectively. This approach allows for easy identification of the imaging features focused on by the deep learning model, thereby providing researchers with clear direction for further investigation into the radiological characteristics of COVID-19.

Fig.3a COVID-19 Prediction Output Heatmap, with Red Highlighting Activation Regions Associated with the Predicted Class

Fig. 3b CAP prediction output heatmap, with red highlighting activation regions associated with the predicted class

Fig. 3c Heatmap of non-pneumonia prediction output

Many pulmonary diseases exhibit substantial overlap in their imaging characteristics. These conditions are influenced by host factors such as patient age, drug responsiveness, immune status, and underlying comorbidities. Consequently, it is difficult to differentiate all pulmonary diseases based solely on chest CT imaging findings. Furthermore, the current study has only performed classification predictions for COVID-19 and has not yet classified the severity of infection. Keya Medical stated, “Next, we will attempt to further refine the trials to predict both the presence of COVID-19 and the severity of patient infection, thereby assisting physicians in better monitoring, treating, and managing patients.”

Keya Medical’s research demonstrates sufficient innovation. Its clinical application will not only improve the accuracy of CT image-based diagnosis for COVID-19 but also assist clinicians in making early confirmed diagnoses for infected patients. This will significantly enhance diagnostic efficiency for frontline physicians, optimize COVID-19 screening processes, enable efficient and precise screening, reduce physicians’ workload, and facilitate the rational allocation of medical resources.

As one physician noted, “Radiologists tend to favor the diagnosis of ‘space-occupying lesions’ and rarely diagnose pneumonia. However, the term ‘space-occupying lesion’ is too broad for patients, who require more precise diagnostic conclusions—judgments that are seldom provided by radiology departments nowadays. Currently, many AI systems are engaged in quantitative data analysis, but this is not their core strength, as conventional data-processing software can also collect and mine data. Therefore, to realize its true value, AI must leverage quantitative data to deliver qualitative diagnostic judgments.”

Keya Medical has taken a valuable step in exploring the untapped potential of AI. The precise role that radiology departments can play remains to be further elucidated by AI companies through deep learning-based exploration of the unknowns within pixels; the answer may well lie hidden among the dots.