128 Imaging Cases, 32 Medical Experts Explore Human-AI Collaboration: AI-Powered Lung Nodule Diagnosis Takes Center Stage at 2018 Thoracic Surgery Conference

On March 31, the event “I AI Read Scans, Co-Creating a Smart Future – Special Session on AI-Assisted Diagnosis of Small Pulmonary Nodules” was held in Chengdu. The event was hosted by the Thoracic Surgeons Branch of the Chinese Medical Doctor Association, co-organized by West China Hospital of Sichuan University, and supported by LinkDoc Technology as the AI technology provider.

Unlike previous “human-versus-machine” competitions, this event compares the efficiency, time consumption, and accuracy of manual image interpretation versus AI-assisted interpretation, with a view to exploring and envisioning future application scenarios for artificial intelligence in the diagnosis of small pulmonary nodules—a relatively novel initiative within the industry.

In addition, the event invited renowned clinical experts, imaging specialists, and AI technology experts from West China Hospital, Xiamen University Affiliated Hospital, Peking University Cancer Hospital, Sichuan Cancer Hospital, The First Affiliated Hospital of Soochow University, and Henan Cancer Hospital to provide technical and academic guidance. Professor Zhang Xun from Tianjin Chest Hospital, Honorary President of the Thoracic Surgeons Branch of the Chinese Medical Doctor Association, and Professor Liu Lunxu, Vice President of West China Hospital and Vice President of the Thoracic Surgeons Branch of the Chinese Medical Doctor Association, served as witnesses to this conference.

In his address, President Liu stated that AI, as a technology leading the future, is gradually being validated.The integration of AI and big data with healthcare has already yielded tangible results in areas such as the diagnosis of pulmonary nodules and dermatological conditions. If primary care hospitals were equipped with AI, they would have the potential to achieve diagnostic accuracy comparable to that of top-tier specialists in China.The purpose of this seminar is to promote the application and development of AI technology in the healthcare sector.

Professor Liu Lunxu, Vice President of the Thoracic Surgeons Branch of the Chinese Medical Doctor Association and Vice President of West China Hospital 

The experience consisted of two rounds with identical tasks, each involving 16 participating physicians. The 128 cases used in the event were randomly selected from case pools provided by West China Hospital of Sichuan University, Tianjin Chest Hospital, and the First Affiliated Hospital of Xiamen University, all confirmed via pathological examination. Dr. Sun Ling from the Department of Radiology at West China Hospital of Sichuan University unsealed the cases on-site. Staff members copied all case materials to the host computer and to the computers stationed in front of each participating physician. Based on a random draw, physicians selected their assigned workstations. Each physician was required to interpret 16 cases of small pulmonary nodules within 13 minutes, with eight cases interpreted using AI assistance (i.e., human-AI collaboration) and eight interpreted manually without AI support.

Perhaps the term “human-machine” struck a chord with the physicians. The contestants did not let their guard down despite the diagnostic exercise being merely experiential; on the contrary, they were exceptionally engaged. Many participants wore solemn expressions, as if they had returned to their hospital departments, fully immersed in interpreting medical images and fulfilling their duties as physicians. As time ticked by, the host began to remind the contestants of the remaining time. Yet even those who had not completed their image interpretation showed no signs of impatience. Clearly, accuracy, rather than the time spent, was their primary consideration.

After two rounds, the final outcome was unsurprising.For pulmonary nodules measuring either 1–3 cm or 0.3–1 cm, the accuracy of benign versus malignant diagnosis by the human-AI collaboration group was higher than that by physicians alone.In terms of diagnostic speed, it is also faster than diagnosis by physicians alone.(The AI-assisted diagnostic system used in this event took only 4.7 seconds to analyze cases and provide results for expert reference.) Specifically, in the diagnosis of small pulmonary nodules measuring 1–3 cm, the human-AI collaboration group achieved a 5.42% higher accuracy rate and reduced diagnosis time by 6.67% compared to physicians working alone.In the diagnosis of pulmonary nodules measuring 0.3–1 cm, the human-AI collaboration group achieved a 15.75% higher accuracy rate and a 25.01% reduction in time compared to physicians working alone.

Regarding the results of this competition, Professor Zhang Xun stated,The true application of artificial intelligence should be at the grassroots level, with the aim of improving the diagnostic capabilities of primary care physicians. This is a crucial step in the national tiered diagnosis and treatment system.. Merely relying on training to enhance the competencies of primary care physicians requires a considerable amount of time. Therefore, by leveraging AI technology to establish a teleconsultation center affiliated with tertiary hospitals, primary care physicians can upload imaging studies to the center for preliminary AI-based screening. Specialists from tertiary hospitals can then provide diagnostic opinions within a short timeframe and feed them back to the primary care physicians, thereby improving the diagnostic capabilities of primary healthcare institutions.

Honorary President of the Thoracic Surgeons Branch of the Chinese Medical Doctor Association, Professor Zhang Xun from Tianjin Chest Hospital

“Earlier, I was nervously reviewing the images, my heart actually trembling. Every day, work involves constantly flipping through scans, focusing on those few glass-slide shadows. With the help of AI, this process can save us a great deal of mental effort,” commented Professor Liang Naixin from Peking Union Medical College Hospital, one of the contestants.

Professor Liang Naixin of Peking Union Medical College Hospital

Additionally, he believes that the development of AI should go through three stages:The first stage is to diagnose the presence or absence of a condition, and the second stage is qualitative analysis.This does not merely entail determining whether a tumor is benign or malignant; experts also expect AI to identify tumor subtypes and mutations. Even if it cannot achieve 100% accuracy, it can still provide substantial reference information for physicians, ultimately benefiting patients.

Phase III,Integrating AI into MDT.Not only radiologists, thoracic surgeons, and internists, but also pathologists should be involved. The future integration and cross-disciplinary collaboration of multiple specialties is the direction we are striving toward and a task we should actively pursue.

Another young committee member, Professor Xu Shiguang from the General Hospital of the Shenyang Military Region, stated that AI has provided significant assistance to physicians, enabling contestants to locate lesions with ease and make judgments based on its prioritized prompts, thereby substantially improving clinical efficiency.

Professor Xu Shining, Department of Thoracic Surgery, Jinan Military Region General Hospital

For a subset of the images,Professor Xu did not endorse the AI-generated suggestions, instead placing greater trust in his own clinical judgment.However, the results indicate that AI still achieves higher accuracy. He believes that physicians need to engage in more learning during their clinical practice, particularly by validating their findings against pathology results, to determine the accuracy of their diagnoses and accumulate experience.

Professor Xu also shared his views on the relationship between AI and physicians. He believes that while AI serves as a significant aid to doctors, it will not replace them. The ultimate outcome remains human-centric, with physicians making the final diagnostic decisions.Undoubtedly, AI can help physicians improve efficiency, reduce workload, and lower the rate of missed diagnoses. In this way, surgeons, pathologists, and radiologists will have more time to study and enhance their professional capabilities.

“How can the canal be so clear? Because fresh water flows from its source.” Dr. Liu Xiaohua, Chief Scientist at LinkDoc Technology—the AI technology provider for this human-AI collaboration experience event—and one of the world’s pioneering experts in early AI research, used this poetic line to illustrate the relationship between AI and medical specialists.

Dr. Liu Xiaohua, Chief Scientist at LinkDoc Technology

He believes that,The advantage of AI lies in its powerful computational capabilities and tireless operation. However, no matter how sophisticated the algorithms are, without the benefit of expert experience, even the most intelligent systems risk becoming “artificial stupidity.” Therefore, to truly empower AI, it is essential to rely on the support of expert knowledge.Each training iteration of the AI aims to ensure that the identified shape and size of nodules align with expert annotations, thereby learning from the experts’ clinical reasoning.

At the conclusion of the competition, the attending experts engaged in discussions on the latest research advances in AI-enabled early diagnosis of lung cancer, the current status of AI application and development in clinical medicine in China, and future innovations in clinical diagnostic and therapeutic scenarios as well as explorations of implementation models.