World AI Conference Establishes Standard Medical AI Datasets; Data Challenges Remain for AI Development

On July 9, the three-day 2020 World Artificial Intelligence Conference opened at the Shanghai Expo Center. Themed “Intelligent Connectivity for a Shared World, Our Common Home,” this year’s conference focused on three key topics: “AI Technology Trends,” “AI Empowering the Economy,” and “AI Warming Our Homes.”

As the core driving force behind a new wave of industrial transformation, artificial intelligence is reshaping every stage of economic activities—including production, distribution, exchange, and consumption—thereby generating new demands for intelligence across all sectors, from the macro to the micro level. This shift is spurring the emergence of new technologies, products, industries, business formats, and models. Particularly in the healthcare sector, AI is fundamentally redefining the broader health ecosystem.

Compared with the previous two editions of WAIC, this year’s conference is more comprehensive. Ning Guang, Sun Chunlan, Zhang Wenhong, Wu Fan, and many other leaders who have been on the front lines of the COVID-19 pandemic response have gathered here, making the topics of the conference more diverse.

Following the conference, VCBeat conducted a comprehensive review of WAIC 2020’s impact on the healthcare sector and interviewed select participating enterprises and institutions, aiming to identify the development trajectory of medical AI amidst the wealth of information.

In April 2019, the State Council issued the Development Plan for New-Generation Artificial Intelligence, which proposed five new directions in the era of big data: swarm intelligence, cross-media intelligence, human-machine hybrid augmented intelligence, and autonomous intelligent systems. This has undoubtedly laid a solid foundation for the development of artificial intelligence technology in China. With technological advancements, AI is continuously expanding its scope of applications, with the healthcare sector being one of them.

Among the many specialized fields, “AI + Medical Imaging” has garnered the most attention due to its close alignment with physicians’ daily workflows, while also being subject to the most stringent regulatory oversight. However, whether in the now-substantial field of medical imaging, the ongoing development of smart hospitals, or the exploratory efforts in new drug R&D, all AI-related domains cannot escape the issue of data standardization.

To address data standardization issues and ensure the system security, compatibility, reliability, and other capabilities of artificial intelligence applications, institutions such as the Center for Medical Device Evaluation of the National Medical Products Administration and the China Academy of Information and Communications Technology have been deeply involved in the formulation of labeling standards and product reviews, aiming to regulate the development of medical artificial intelligence.

Over the past two years, the National Medical Products Administration (NMPA) has successively formulated technical specifications and evaluation frameworks, including the “Review Points for Deep Learning-Assisted Decision-Making Software,” the “Appendix to the Good Manufacturing Practice for Medical Devices: Standalone Software,” the “Technical Review Guidelines for Medical Device Software Registration,” and the “Technical Review Guidelines for Cybersecurity of Medical Devices.” In parallel, the NMPA established the Artificial Intelligence Medical Device Innovation Cooperation Platform. Since January 2020, multiple AI software products have obtained Class III medical device certifications through these pathways.

The China Academy of Information and Communications Technology (CAICT) is also actively advancing initiatives such as the standardization of artificial intelligence data and the development of standardized databases. At this year’s World Artificial Intelligence Conference (WAIC), Wang Aihua, Executive Deputy Secretary-General of the China Artificial Intelligence Industry Alliance, presented three recent achievements of the innovation platform in sequence.

The first achievement is the establishment of an AI database for diabetic retinopathy, developed by the Real-World Data Application Working Group led by Peking Union Medical College Hospital. The significance of benchmark datasets and algorithm evaluation testing platforms cannot be overstated; they serve as the foundational guarantee for promoting the effectiveness, reliability, robustness, and safety of artificial intelligence models in healthcare applications, and are also a critical pillar for advancing the sustainable development of the AI-driven medical industry.

The second achievement is that Peking Union Medical College Hospital obtained third-party certification for diabetic retinopathy, based on established standards for relevant data and third-party databases. Meanwhile, the China Academy of Information and Communications Technology (CAICT) established a public service platform for medical AI evaluation and issued two sets of performance indicators and testing methodologies for artificial intelligence products.

The third achievement is the establishment of a public service platform for the artificial evaluation of medical technologies. Designed to provide the public with an authoritative, secure, and traceable platform, it aggregates data on various diseases from different hospitals and medical institutions to validate corresponding artificial intelligence (AI) products. Furthermore, the platform developers are required to devise specific testing methodologies for different products and systems and generate corresponding reports.

Traditional medical devices must undergo a series of tests and evaluations before market launch, as well as a six-month clinical trial, before they can be approved for marketing. Wang Aihua stated, “For traditional medical devices with slower update and iteration cycles, the rationality of this process has been validated over many years. However, for emerging technologies like artificial intelligence, which feature rapid updates and iterations, this approach presents significant challenges. If every update were required to undergo corresponding clinical trials under the traditional device regulatory framework, no product would successfully navigate the approval process. With this platform, we can help enterprises rapidly supplement clinical data, allowing clinical trials during product iterations to be completed quickly. This is the key differentiator of the platform.”

As a third-party platform, security is the most critical prerequisite for its normal operation. Wang Aihua believes that the security of third-party platforms encompasses two aspects. The first relates to intellectual property issues concerning data provided by third parties; therefore, when building the platform, the China Academy of Information and Communications Technology (CAICT) establishes virtual private channels to encrypt data and products, strictly adhering to the three principles of “data ingress only, no egress,” “no manual intervention,” and “deletion immediately after testing.” Only in this way can the security of third-party data and related test products be adequately ensured. The second aspect concerns the traceability of tests, facilitating the investigation and correction of any problematic tests.

Beyond standard-setting, under the guidance of the Department of Science and Technology of the Ministry of Industry and Information Technology (MIIT), the China Academy of Information and Communications Technology (CAICT), the Artificial Intelligence Industry Alliance, and the MIIT Key Laboratory for Medical AI Research and Validation jointly announced the results of the “Evaluation of AI-Assisted Diagnostic Products for Pneumonia Imaging.” The evaluation comprehensively assessed various indicators of the participating products through on-site testing and document review, strictly adhering to national standards, industry standards, and the relevant technical review principles issued by the National Medical Products Administration (NMPA). Furthermore, a specialized expert committee was established to develop authoritative evaluation criteria.

Ultimately, the AI-assisted diagnostic imaging products for pneumonia developed by seven medical AI companies—Shanghai Yitu Healthcare Technology Co., Ltd., Shanghai Deepwise Medical Technology Co., Ltd., Beijing Ande Yizhi Technology Co., Ltd., Hangzhou Jianpei Technology Co., Ltd., Ping An Technology (Shenzhen) Co., Ltd., Shanghai Haohua Information Technology Co., Ltd., and Shanghai United Imaging Intelligence Healthcare Technology Co., Ltd.—stood out among the 17 participating medical AI firms.

This evaluation has broken the previous pattern in the medical AI industry, where companies operated in silos and made unsubstantiated claims. It marks not only the first official head-to-head competition of medical AI-assisted diagnostic imaging products but also a significant assessment of “AI-powered anti-epidemic products” at a special historical juncture, carrying profound implications.

To ensure the authority of this assessment, the evaluation is divided into three parts: functional review, performance testing, and expert review. The functional review primarily assesses the functional completeness, system security, reliability, usability, and compatibility of pneumonia AI products. The performance testing mainly employs annotated test data for objective testing, evaluating the clinical diagnostic performance of pneumonia AI products using metrics such as accuracy, sensitivity, specificity, and the Kappa coefficient. The expert review involves inviting authoritative clinical experts from China to evaluate the products in terms of user experience, practicality, innovation, and rationality, while also comprehensively considering the evaluation of special innovative features.

According to relevant experts from the China Academy of Information and Communications Technology (CAICT), the test data constituted another major highlight of this evaluation. Specifically, the evaluation utilized 1,000 cases of pneumonia, including COVID-19 and other types of pneumonia, which were tested using the respective models (products) of each participating medical AI enterprise. The data were sourced entirely from seven designated medical institutions for COVID-19 treatment, including the Shanghai Public Health Clinical Center. During data collection, careful consideration was given to three dimensional indicators: acquisition equipment and configuration, data diversity, and case distribution.

Meanwhile, to enhance robustness and resistance to interference, 70 cases of non-standard imaging data frequently encountered in clinical practice were incorporated. Furthermore, regarding data annotation standards, the organizer established a draft annotation protocol, which was implemented by the participating hospitals. The annotated data were then consolidated and uniformly validated by a specialized committee, thereby effectively mitigating potential inconsistencies arising from heterogeneous annotation practices.

“Performance testing is the primary factor driving the disparity,” stated a relevant official from the China Academy of Information and Communications Technology (CAICT). Performance testing encompasses the determination of “presence of pneumonia,” “presence of COVID-19,” and “lesion location.” Evaluation results indicate that while most enterprise products demonstrate robust capability in screening for “presence of pneumonia,” only three companies achieved accuracy, sensitivity, and specificity rates exceeding 80% in the differential diagnosis of COVID-19, with some metrics surpassing 90%.

On June 19, the Department of Science and Technology of the Ministry of Industry and Information Technology released a list of artificial intelligence enterprises that demonstrated outstanding performance in leveraging technology to support the fight against the COVID-19 pandemic. Taking Ande Yizhi, which is included on the list, as an example, according to public reports by People’s Daily during the epidemic: “This AI-based qualitative diagnostic system for COVID-19 can not only perform ‘pneumonia diagnosis,’ but also further differentiate COVID-19 from other types of pneumonia (such as viral pneumonia and bacterial pneumonia).” “While achieving a sensitivity close to 100% in detecting pneumonia, the accuracy of differential diagnosis for COVID-19 (consistent with positive nucleic acid test results) can reach over 97.8%.”

According to the overall evaluation by the China Academy of Information and Communications Technology (CAICT), the pneumonia AI products from domestic medical AI companies that stood out in this assessment have reached a relatively high level, with product functions being relatively mature and comprehensive. A small number of products have achieved recognition capabilities specifically for COVID-19 pneumonia. However, it should be noted that only a few products can identify more than ten types of pneumonia. The precision of lesion localization still needs further improvement, and performance and functionality require training on larger volumes of annotated data.

As a key driving force behind the advancement of artificial intelligence, various medical AI companies have also participated in the discussion, offering thought-provoking insights. VCBeat has selected four key directions and extracted their core viewpoints.

Huang Feng, Dean of the Neusoft Medical AI and Clinical Innovation Research Institute:The integration of artificial intelligence and the internet aims to address critical pain points in the current healthcare sector, including insufficient and unevenly distributed medical resources, a shortage of high-caliber professionals, and high operational costs. In response to these challenges, the government has proposed strengthening primary care and implementing a tiered diagnosis and treatment system as the strategic direction, with “AI + Internet Healthcare” serving as the solution.

From a technical perspective, the primary beneficiaries of AI assistance are grassroots healthcare institutions with limited diagnostic and treatment capabilities. However, at present, high-quality AI software and workstations often cost millions of yuan, a price point that is unaffordable for grassroots medical facilities. Consequently, directly deploying artificial intelligence at the grassroots level faces significant challenges. Internet hospitals offer a new business model that can amplify the impact of AI and reduce regional healthcare deployment costs through medical consortiums.

From another perspective, while the internet connects healthcare institutions across all tiers, it cannot provide round-the-clock service, as physicians still require rest. In light of this reality, there is a need for AI-powered intelligent applications that operate 24/7 in the cloud to assist physicians with intelligent diagnostic support and analysis.

In summary, the Internet and AI are two technologies that mutually reinforce and complement each other; only through their synergy can they better serve healthcare.

Liang Weimin, Chairman of Ande Medical Intelligence:The impact of AI and 5G on healthcare is primarily reflected in three aspects. First, the decentralization of high-quality medical resources, with a focus on hospital consultations. Second, advancing the development of smart hospitals and promoting education for primary care physicians through methods such as remote teaching. Third, public health prevention and early warning systems can naturally integrate with “AI + 5G”; therefore, we should promote the establishment of artificial intelligence-based early warning platforms to enhance capabilities for early monitoring and alerting of infectious diseases.

Qiao Xin, CEO of Deepwise Medical:When a major epidemic outbreak occurred at the Mudanjiang border port, Deepwise Medical rushed to the frontline and joined forces with Heilongjiang Unicom to deploy a 5G medical private network in Heilongjiang’s border cities. This initiative enabled information interoperability with local hospitals, facilitated the development of smart hospitals, and supported various intelligent medical services such as remote consultations. In retrospect, without the 5G infrastructure in Heilongjiang’s border cities, this collaboration would not have been possible. From this perspective, while 5G and AI can work synergistically, it does not mean that they currently constitute a disruptive variable capable of transforming healthcare; rather, 5G and AI each follow their own distinct developmental paths.

Dai Ying, Chief Innovation Officer at GE Healthcare:The sudden outbreak caused brief chaos but also accelerated technological advancement. In this process, we have identified further room for development in the role of artificial intelligence in healthcare, primarily across three directions.

First, we will promote the development of smart hospitals. In the near future, traditional equipment in hospitals will be transformed into intelligent devices. More AI will be embedded to help these devices collect data and perform imaging scans with greater precision and efficiency.

Second, intelligent hospital operations. Top-tier hospitals in China are often overcrowded with patients, making efficient operational management a significant challenge for hospital administrators. Artificial intelligence can analyze big data on hospital operations to facilitate more effective coordinated management of medical equipment, patient and staff flow, and departmental administration.

Third, assisted diagnosis and treatment. From the perspective of intelligent healthcare, artificial intelligence applications can be integrated throughout the entire process—from disease screening, diagnosis, and treatment to subsequent efficacy evaluation and follow-up—helping us enhance efficiency, improve healthcare accessibility, and achieve precision medicine.

These three areas are also the strategic focus of GE Healthcare. In the post-pandemic era, the construction of a next-generation AI-driven healthcare system is an urgent priority.

Mao Xinsheng, Chairman of Shukun Technology:The role of AI in enhancing healthcare quality is beyond doubt, even capable of driving disruptive innovations. During the fight against the pandemic, we have begun to glimpse AI’s potential.

When the epidemic broke out, nearly 3,000 patients at Wuhan Central Hospital were awaiting imaging scans, diagnosis, and results. However, there were insufficient physician resources to handle such a massive workload, and AI played three critical roles in this context.

First, if a patient presents with fever and cough, how can we determine whether they have COVID-19? While it may take several minutes for a physician to perform an imaging-based assessment, AI can complete the analysis in just ten seconds. By integrating epidemiological data and nucleic acid test results, the patient’s condition can be rapidly evaluated.

Secondly, what treatment approach should be adopted for a patient with confirmed COVID-19? How can the effectiveness of the treatment be assessed? Generally, physicians need to compare changes in lesions between consecutive imaging examinations, which poses certain challenges. In contrast, AI can provide quantitative information.

Finally, how is the decision made regarding whether a patient can be discharged? This also relies on quantitative data analysis conducted by AI during follow-up.

Through these three functions, Shukun Technology helped alleviate significant pressure on Wuhan Central Hospital during the pandemic. However, this is not the end point; we aim to leverage AI to make more hospital scenarios intelligent, as we are still far from reaching the ceiling of AI’s potential.

Su Xiaoming, Vice President of Yitu Healthcare:From what we have observed so far, the successful deployment of all AI products must be grounded in customer value, aligned with scenario-specific needs, and capable of creating added value for customers; only then can they sustain viability. Based on this, we have identified three clear development trends.

The first trend is the shift from single-point tasks to closed-loop scenarios. Initially, artificial intelligence was capable of performing only single-point tasks, such as the detection of pulmonary nodules or the assessment of bone age. However, applications based on such isolated tasks fail to meet the clinical demand for closed-loop workflows. Consequently, we have witnessed AI evolving from single-task capabilities toward multi-task functionalities and integrated, closed-loop scenarios.

The second trend is the shift from top-tier hospitals to primary care settings. Initially, the emergence of artificial intelligence (AI) was closely tied to specialists at leading medical centers. However, the most significant practical application scenario for AI lies not in these top-tier institutions, but in primary care—specifically, in helping grassroots physicians with limited or insufficient diagnostic capabilities achieve standardized levels of diagnostic accuracy.

The third trend is the shift from purely software-based services to new scenario-based solutions that integrate both hardware and software. Initially, artificial intelligence was delivered solely in the form of software. However, we have found that in many application scenarios, AI requires hardware support. This is particularly true at the primary care level, where lightweight, high-performance hardware is needed to assist in delivering diagnostic and treatment services. To address this, Yitu Technology has developed intelligent chip systems and smart terminals to support primary care institutions in meeting their clinical diagnostic and therapeutic needs.

Meanwhile, we have found that in the field of pediatric growth and development, existing digital radiography (DR) involves excessively high radiation doses and requires shielded rooms, thereby limiting its application for periodic follow-up examinations in children. To address this, we developed a dedicated low-dose bone age imaging system, which, when integrated with our AI-assisted diagnostic platform, enables low-dose radiation exposure and can be deployed in kindergartens, schools, and communities. This represents the third trend we have identified.

Li Jingjue, CEO of Ande Medical Intelligence for the Greater China Region:AsHow to Understand the Core of Artificial Intelligence? Its Essence Lies in the Two Characters “Zhi” (Intelligence) and “Neng” (Capability). The “intelligence” of medical AI should derive from the extensive clinical experience accumulated over many years by top physicians at prestigious hospitals. While such medical expertise was non-replicable in traditional medicine, AI can aggregate and integrate these data-driven experiences.

After creating “intelligence,” we must empower grassroots healthcare by enhancing the diagnostic and treatment capabilities of primary care physicians, enabling patients to access high-quality medical services from specialists at prestigious hospitals right in their communities. This represents the core developmental imperative for artificial intelligence, rather than merely addressing issues of efficiency and time.

In the early stages, medical artificial intelligence (AI) primarily addressed single diseases, isolated scenarios, and individual tasks. However, medicine itself is grounded in complex evidence-based logic; it does not exist in isolation by single disease entities, nor is it satisfied by single-task solutions. Therefore, for the future development of AI, we must address the challenge of simultaneous multi-disease analysis. Simulating the human brain’s capacity to resolve complex medical problems represents the true direction for the future advancement of artificial intelligence.

AstraZeneca's Global Executive Vice President, Lei Wang:From AstraZeneca’s perspective, artificial intelligence not only plays a role in drug discovery and clinical research but also extends its capabilities across the entire end-to-end process of pharmaceutical manufacturing and operations.

For example, chronic kidney disease is a multi-target disorder with a highly complex genetic landscape, in which numerous receptors are implicated. Therefore, artificial intelligence can be employed to analyze the pathophysiological essence of the disease and to screen compounds via AI-driven approaches.

Meanwhile, in drug development, the rise of wearable devices and health apps enables patients to complete follow-up visits at home. The resulting data can also be leveraged for real-world evidence studies, which will help accelerate future drug approvals.

Furthermore, artificial intelligence can be applied to drug research, development, and manufacturing, thereby advancing the construction of smart factories and intelligent logistics systems. During the pandemic, AstraZeneca leveraged extensive automation in its resumption of work and production, ensuring high-quality drug manufacturing while preventing employee congregation.

In the future, AstraZeneca aims to establish an AI Open Innovation Laboratory in Shanghai, inviting partners to join us in collaboratively exploring how to translate backend technologies into practical applications across various disease areas and care pathways. Our goal is to deliver tangible value to patients within real-world clinical scenarios. I believe this vision represents the ultimate outcome that AstraZeneca and its AI partners will achieve together, and we hope that such a laboratory will ultimately be realized to benefit the entire ecosystem.

Overall, as an emerging technology, AI’s innovation does not lie in creating previously non-existent medical scenarios; rather, it operates within nearly every existing medical scenario, synergizing with technologies such as the Internet, 5G, and IoT to empower the new healthcare landscape.

Therefore, as Dai Ying, Chief Innovation Officer at GE Healthcare, stated: “In the future, no AI enterprise can stand alone. Much like the concept of a community with a shared future for mankind, AI requires the development of a comprehensive ecosystem. Whether it involves data collection or the construction of smart hospitals, collaborative efforts are essential; thus, we need to build an ecological ecosystem.”

This also reflects the original intention behind the establishment of the World Artificial Intelligence Conference: “Intelligently Connecting the World, Building a Shared Home.” Whether in healthcare, security, or commercial applications, as long as big data is involved, collaborative efforts remain essential. Precisely because of this, artificial intelligence may not advance as rapidly over the next decade...