Tencent Achieves Breakthrough with Eight Papers Accepted at MICCAI 2019 Across Pathology Image Classification, Medical Image Segmentation, CT Lesion Detection, and Machine Learning

With the MICCAI 2019, a top-tier international academic conference on AI in medical imaging, approaching, paper acceptance notifications are being released sequentially. Tencent has had eight papers accepted, covering areas such as pathological cancer image classification, medical image segmentation, CT lesion detection, and machine learning.

In recent years, AI in medicine has flourished, with AI-driven medical solutions represented by “Tencent Miying” rapidly entering the core diagnostic processes for various diseases. However, industry-wide challenges such as inconsistent quality of medical imaging data and the difficulty of manual annotation have posed significant obstacles to the learning and application of AI in the medical field.

At MICCAI 2019, two major AI laboratories under Tencent—Tencent AI Lab and Tencent Youtu Laboratory—conducted innovative research from the perspectives of deep learning efficiency enhancement and clinical medical applications, respectively, with four papers accepted from each lab.

A major challenge in applying AI to medical image classification is the scarcity of training data. One solution is to combine various training strategies, such as transfer learning, multi-task learning, and semi-supervised learning. Researchers at Tencent AI Lab have integrated these three approaches into a unified framework, thereby accumulating their respective contributions and enabling a fair comparison of each method’s impact across different scenarios.

Figure Caption: State-of-the-art consistency-constrained algorithms for semi-supervised learning (including VAT and the PI model) and widely adopted multi-task learning algorithms (hard parameter sharing) are integrated into a single network. End-to-end training is performed using an alternating task strategy, with transfer learning incorporated through the initialization of starting points.

A series of experiments on the classification of benign and malignant gastric endoscopy images demonstrate that transfer learning yields the most significant performance improvement when used alone. Building upon transfer learning, multi-task learning can further enhance performance during the early stages of a project when data volume is limited, whereas semi-supervised learning provides continuous improvements with larger datasets. The combination of all three approaches, leveraging large-parameter networks, achieves superior performance. These research findings can guide the individual or combined application of transfer learning, multi-task learning, and semi-supervised learning to improve the accuracy of medical classification models.

Meanwhile, research from Tencent Youtu Laboratory promises to offer new approaches to addressing the challenges of annotating medical images. Medical imaging data, such as CT and MRI scans, are typically three-dimensional (3D), posing significant difficulties in annotation and making annotated information hard to obtain. As a result, the quantity of annotated 3D medical images is usually insufficient for effectively training deep learning networks. Tencent Youtu Laboratory has proposed a self-supervised learning method that deeply mines information from raw data by simulating the process of solving a Rubik’s Cube, enabling the network to autonomously extract useful features from the raw data.

This study is expected to reduce the data requirements of deep learning networks and improve the accuracy of subsequent supervised tasks. Meanwhile, it represents the first proposal of a 3D self-supervised learning method in the industry, with the potential to fill existing research gaps.

Figure Caption: A Self-Supervised Learning Framework for Simulating the Rubik's Cube Restoration Process

This framework includes two operations: shuffling the order of cubes and rotating the orientation of cubes.

Furthermore, another study from Tencent Youtu Lab proposed a pairwise segmentation framework. By effectively mining the relationships between medical image slices and imposing constraints through proxy supervision, this approach significantly increases the volume of annotated data via inter-slice pairing, while enhancing the smoothness and consistency of prediction results across adjacent slices. This method proves effective in real-world semantic segmentation scenarios where annotated data is scarce, leveraging prior knowledge to compensate for insufficient data volumes.

Dr. Yao Jianhua, Chief Scientist at the Medical Center of Tencent AI Lab, previously outlined three primary research directions in pathology AI: first, AI-based pathological diagnostic models; second, pathomics; and third, AI-based pathological prognosis prediction models. These three directions respectively help physicians improve diagnostic efficiency, consistency, and accuracy, while also adding the capability to predict treatment efficacy. At MICCAI 2019, Tencent AI Lab published three consecutive papers on pathological analysis, detailing its new breakthroughs in AI-driven pathology analysis.

One study, conducted in collaboration between Tencent AI Lab and the Sixth Affiliated Hospital of Sun Yat-sen University, proposes a method that uses only clinical diagnostic results as weak supervision signals to train a classifier for automatically classifying patches extracted from pathological images. This novel classification approach can accelerate the development of pathological image classifiers and, by integrating with smart microscopes, provide real-time diagnostic insights to clinicians, thereby reducing the incidence of misdiagnosis.

Figure Legend: HE-stained section of colorectal cancer pathological tissue

By integrating smart microscopes, real-time diagnostic insights can be provided to clinicians, thereby reducing the incidence of misdiagnosis.

The second study, conducted in collaboration between Tencent AI Lab, South China University of Technology, and the Sixth Affiliated Hospital of Sun Yat-sen University, proposed an unsupervised domain adaptation algorithm to train deep neural networks with domain invariance. This approach addresses the classification task of unlabeled microscopy images by leveraging annotation information from whole-slide digital pathology images. The deep neural network trained using this method achieved exceptional performance on microscopy tasks without utilizing any labeled microscopy data, even surpassing networks trained with partially labeled datasets.

Study 3 focuses on color normalization of pathological images. Due to variations in tissue preparation and staining protocols, as well as differences among pathology scanners, digital pathological images exhibit significant color discrepancies. Therefore, color normalization is required prior to nearly all digital pathology-related analyses. Tencent AI Lab proposed a cycle-consistent generative adversarial network optimized for the characteristics of pathological images. By introducing additional inputs, this approach guides the generator to produce pathological images with specific color styles, thereby stabilizing the cycle-consistency loss function during training. Compared with other color normalization methods, the color normalization network developed by Tencent AI Lab, when used as a preprocessing step for cancer classification tasks, can better enhance the performance of subsequent tasks.

Figure legend: (Top) Test data were sourced from five different medical centers. The method proposed in this paper can normalize the color styles of these pathological images to a similar appearance. (Bottom) The trained model was directly tested on other pathological datasets without parameter fine-tuning, and it still achieved accurate color normalization.

Dr. Zheng Yefeng, Medical AI Director at Tencent Youtu Lab, who has been deeply engaged in intelligent medical image analysis for many years, stated when sharing his experience in AI algorithm research: “The most important insight is not to alter doctors’ existing workflows, but to seamlessly integrate AI technology into their diagnostic procedures.” At MICCAI 2019, Tencent Youtu Lab presented two research outcomes closely aligned with clinical diagnostic needs.

One study focuses on the application of CT lesion detection. To improve the accuracy of detecting lesions of varying sizes, Tencent Youtu Lab proposed a multi-scale detector leveraging channel and spatial attention mechanisms. This approach achieved more accurate results on 2D detection networks than on 3D detection networks for the first time, thereby enhancing detection efficiency and laying a solid foundation for subsequent research on lesion detection methods. Notably, this method can also provide clinicians with real-time lesion detection results, improving their efficiency in analyzing CT images.

The second study focuses on fundus vessel segmentation. As the only vessels in the human body that can be directly observed non-invasively, retinal blood vessels are significantly affected by many systemic diseases and cardiovascular or cerebrovascular disorders, which alter the morphology of retinal arteries and veins. Therefore, automated vessel segmentation and arteriovenous classification hold substantial clinical significance. Tencent Youtu Lab has introduced deep learning and multi-task models to address vessel segmentation and arteriovenous classification, achieving end-to-end segmentation and classification of arteries, veins, and the entire vascular network simultaneously.

This research achievement significantly improves the accuracy and prediction speed of fundus vessel classification, achieving end-to-end vessel segmentation and arteriovenous classification. It lays the foundation for precise quantification of fundus vessels, thereby facilitating research on fundus biomarkers related to systemic diseases and cardiovascular and cerebrovascular disorders.

Figure Caption: Visualization of Arterial and Venous Vessel Segmentation and Classification

(A) Original image; (B)(E) Vessel segmentation and artery-vein classification labels; (C)(F) Model prediction results; (D)(G) Magnified views of local details

As the technology provider behind “Tencent Miying,” Tencent Youtu Lab and Tencent AI Lab have continuously translated their research achievements into real-world clinical studies and application explorations. Currently, “Tencent Miying” can leverage AI-based medical image analysis to assist clinicians in screening for early-stage lung cancer, fundus lesions, colorectal tumors, cervical cancer, breast tumors, and other diseases. It also utilizes an AI-assisted diagnostic engine to help physicians identify and predict risks for over 700 diseases.

Meanwhile, the clinical efficacy of AI technology is being progressively validated. In cities such as Beijing, Shanghai, Guangzhou, Wenzhou, Nanning, and Deqing, “Tencent Miying” has conducted clinical trials and research collaborations focused on early-stage lung cancer, gastrointestinal tumors, and fundus diseases under the leadership of discipline heads, aiming to leverage technology to help address the imbalance in medical resource distribution.