Zhinuo MedMark PD-L1 Quantitative Scoring System Passes Turing Slide-Reading Test, Demonstrating Human-Level Accuracy in Pathological Assessment
From September 18 to 20, 2020, the 2020 Annual Conference of the Pathology Branch of the China International Exchange and Promotive Association for Medical and Health Care was successfully held at the Shangri-La Hotel in Suzhou Industrial Park. Centered on the application of artificial intelligence (AI) in pathological diagnosis, the conference featured insightful keynote addresses by leading experts, including Professor Bian Xiuwu, Director of the Department of Pathology at the General Hospital of the Chinese People's Liberation Army and Academician of the Chinese Academy of Sciences; Professor Bu Hong, Vice President of Sichuan University; and Professor Ding Yanqing from Southern Medical University. They discussed the cutting-edge advancements and broad prospects of AI applications in the pathology sector. Additionally, directors of pathology departments from numerous Grade A tertiary hospitals across China delivered academic lectures and engaged in discussions on recent progress in molecular pathology of tumors, such as lung cancer, breast cancer, and lymphoma. It is believed that the in-depth application of AI in pathology will significantly promote the development of the pathology industry in the future.
Figure 1. Professor Liang Zhiyong, Director of the Department of Pathology at Peking Union Medical College Hospital, delivers the opening address for the “Turing Slide Reading” Star Competition.
The PD-L1 Pathology “Turing Reading” Star Competition is one of the flagship events at the conference’s lung cancer session. A key component of this event involves conducting a Turing test on the Tumor Proportion Score (TPS) for PD-L1 as interpreted by artificial intelligence (AI) algorithms. As an important tumor biomarker discovered by the two 2019 Nobel Laureates, PD-L1 has garnered significant attention in the field of immunotherapy. The efficacy of immune checkpoint inhibitors targeting the PD-1/PD-L1 pathway has been extensively validated and widely recognized. However, companion diagnostic testing for PD-1/PD-L1 immune checkpoint inhibitors is characterized by challenges in quantification, time-consuming interpretation, and poor result stability, making accurate PD-L1 assessment a new pain point in the daily practice of pathologists. Therefore, AI-based precise quantitative interpretation systems have emerged as the optimal solution to address this challenge.
Figure 2. Schematic diagram of AI-based precise quantitative interpretation. Through steps such as tumor region segmentation, cell segmentation, and positivity determination, automated and precise quantitative reading of the Tumor Proportion Score (TPS) can be achieved; (a) Original image before slide processing; (b) Results of tumor region segmentation, with red areas indicating cancerous regions predicted by the AI model; (c) Results of cell segmentation and positivity determination, where green highlights represent PD-L1-negative cancer cells and red highlights represent PD-L1-positive cancer cells. The TPS value can be precisely calculated based on the counts of these two cell types.
Turing Test[1]Proposed by the British mathematician Alan Mathison Turing (1912–1954) in 1950, the Turing Test is a famous thought experiment designed to determine whether a machine can think. It assesses whether a machine can exhibit intelligence equivalent to, or indistinguishable from, that of a human. It is generally accepted that if more than 30% of evaluators fail to distinguish whether the subject is human or machine after multiple tests, the machine is considered to have passed the test and is deemed to possess human-level intelligence. Drawing on the concept of the Turing Test, this event aims to evaluate the performance and application scenarios of current artificial intelligence technologies in the field of pathological image interpretation by asking pathologists to differentiate between diagnostic results provided by human pathology experts and those generated by AI.
The “Turing Slide Reading” Star Competition was conducted through a hybrid online-offline format, featuring digital pathology slides of tumors (non-small cell lung cancer and urothelial carcinoma). Prior to the event, two independent sets of interpretations were generated separately by a star team of pathologists and an AI system. The star team of pathologists was led by eight experts and professors, including Professor Lin Dongmei, Director of the Department of Pathology at Peking University Cancer Hospital; Professor Fan Xiangshan, Director of the Department of Pathology at Nanjing Drum Tower Hospital; and Associate Chief Physician Li Yuan from the Department of Pathology at Fudan University Shanghai Cancer Center. This team provided expert pathological interpretations for the digital slides used in the competition.
Participants in the “Turing Slide Reading” Star Match are required to identify which of the two sets of results was interpreted by the star team of pathology experts. In addition to providing the TPS score, the AI-based interpretation must also display its specific segmentation results for cancerous regions and cells on the images when the results are announced.
As the provider of the AI model participating in the “Turing Slide Reading” Star Competition, ZhiNuovision employed its independently developed “Biomarker Evaluation Scoring Tool”—the iPathology™ BEST system, which is based on deep learning convolutional neural networks—to perform cancer region prediction and quantitative TPS assessment on a set of non-small cell lung cancer (NSCLC) digital pathology slides provided by the conference, without prior training or optimization.
Nearly 100 pathologists on site actively served as “Turing Judges” in the event. During the nearly one-hour session, participants were given three minutes to review and interpret each whole-slide image, after which they submitted their judgment on whether the reading was provided by a “pathology expert” or “AI.” The results of the discrimination test for non-small cell lung cancer (NSCLC) digital slides showed that over 30% of the pathologists could not distinguish whether the Tumor Proportion Score (TPS) results were generated by pathology experts or AI. For one slide with minimal discrepancy in results, nearly 50% of the pathologists mistakenly identified the AI-generated reading as that of a pathology expert. The event demonstrated that the readings provided by the AI model are already very close to those manually determined by pathology experts. With advantages such as precision, reproducibility, speed, and tirelessness in the actual quantitative interpretation of PD-L1 in tumors, AI is poised to play a significant role in addressing the challenges posed by the demand for precise pathological diagnosis in future tumor immunotherapy.
Figure 3. On-site participants reviewed images on a large screen and then carefully evaluated the results.
The iPathology™ BEST system demonstrates ZhiNuo WeiSi’s R&D strength in the application of artificial intelligence (AI) to pathology, particularly its breakthroughs in AI technologies for the precise analysis of companion diagnostic biomarkers. It is believed that in the near future, ZhiNuo WeiSi’s AI technology will not only assist pathologists in performing more efficient and accurate interpretations of PD-L1 in pathological assessments but also explore additional application scenarios, thereby making new contributions to the precision and integrated diagnosis of tumors in pathology.
ZhiNuoweiSi is a pioneer in AI-driven big data solutions for precision oncology. Its “Intelligent Precision Cancer Diagnosis (iPCD)” platform integrates multiple functionalities, including intelligent analysis of tissue and molecular pathology, automated report generation, and integrated data management, thereby enabling medical testing institutions to achieve quantitative tissue pathology, precise molecular pathology, standardized pathology reporting, and intelligent diagnostic and therapeutic decision-making.
References:
[1] TURING IBYAM. Computing machinery and intelligence-AM Turing[J]. Mind, 1950, 59(236): 433.