Can Multi-Disease AI Redefine the Era of Medical Artificial Intelligence? Insights and Product Overview

In 2020, Shen Si (pseudonym) held a temporary review meeting with the National Medical Products Administration (NMPA) to discuss an innovative AI for auxiliary diagnosis that covers three types of fundus diseases. At the meeting, 30 experts attended, and one of the reviewers told Shen Si:This is the most complex product they have ever seen.

"The AI products we have approved so far are all single-disease AIs, meaning one AI can handle one specific disease. Such AIs are relatively easier in clinical trials since the similar cases that need to be handled may only involve one condition. However, developing multi-disease AIs makes clinical trials more complex. It not only requires us to validate each single-disease AI but also the intersection of two diseases and the areas where neither of the two diseases exists," explained Shen Si.

"At this point, the number of similar cases and exclusive cases becomes particularly high, covering nearly 90% of ophthalmic diseases."

The existing regulatory process is not applicable to such a complex medical device. In order to proceed with the review and approval, Shen Si's team and the drug administration began exploring the regulatory process for multi-disease AI. From scratch, the two groups spent a full two years.

On September 1, 2022, the team'sFundus Lesion Fundus Image Assisted Diagnosis SoftwareFinally, it obtained the Class III certificate issued by NMPA, capturing the first multi-disease AI Class III certificate in China, marking a "milestone" in the history of domestic AI registration and access.

The only regret stems from the coverage of disease types. Compared with the initial plan, the approved third-class certificates only retained two disease types: diabetic retinopathy and glaucoma.

Besides Baidu's Lingyi Zhihui, which backs Shen Si, China's UNITED IMAGING, Vistel, and Airdoc have also made achievements in the review and approval of multi-disease AI. Can multi-disease AI, which is being advanced by many leading companies, change the era of medical artificial intelligence?

If we ignore the security and explainability of the software, and only create a reasonable algorithm to make probabilistic judgments on images, most companies can achieve fairly good results with sufficient data for deep learning training. However, evidence-based medicine requires evidence, and under the black box, we cannot know how the results are generated.

Shen Si told VCBeat: "The partial probabilities provided by AI under big data can indeed reflect, to a certain extent, the potential correlations between symptoms and conditions. However, these correlations are merely statistical, and the process by which the algorithm reaches its conclusions is not interpretable. It must be emphasized that AI is primarily a product with medical attributes, meaning...""If you only calculate a probability through an algorithm, and this probability is irrelevant to the doctor's diagnosis and treatment pathway, then such artificial intelligence will be difficult to pass the review and approval."

As mentioned earlier, the clinical trial design for multi-disease AI is a complex process. For instance, when constructing datasets and validating algorithms for multi-disease AI (using N=2 as an example), it is necessary to not only build Disease A and Disease B databases but also create an A∩B database. Additionally, medical knowledge must be incorporated into the model to enable it to explain, based on medical principles, the probability derivation process of the intersection data.

When the number of disease types is relatively small, the difficulty of building an integrated database is still manageable. However, under the current review and approval logic,As the number of diseases increases, the styles of various disease combinations and the richness of the required datasets rise rapidly, showing an overall exponential trend.

For an enterprise, relying on piling up human resources might be able to piece together the algorithms required for multi-disease AI validation. However, when it comes to building a complete dataset, developers will face some difficult real-world challenges. For instance, data for patients with stage 0 or stage 6 diabetic retinopathy is naturally scarce, making it hard for companies to find enough data in the real world to meet the requirements of a validation dataset. If the combination of different diseases is taken into account, the complexity of related data collection will increase exponentially, eventually becoming an insurmountable challenge in practice.

Despite the challenges faced by multi-disease AI in practical implementation, the ultimate goal for many companies pursuing this technology remains the creation of an AI that can perfectly replicate the specialized abilities of human radiologists.

In theory, the practical application value, R&D difficulty, and the number of diseases covered by multi-disease AI have an exponential relationship, and algorithms with different coverage levels vary greatly in all aspects.Only when N is large enough, covering a wide range of high-flux diseases, and capable of handling multi-modal fusion imaging, can the value of AI converge towards that of doctors.

On this path, companies must carefully consider how to select the target applications for multi-disease AI and determine the number of diseases that the AI will cover.

Reviewing the breakthroughs of domestic companies in multi-disease AI, currently, the major medical device regulatory agencies have approved two Class III multi-disease AI certificates in total. One is the recently obtained retinal multi-disease AI Class III certificate by Baidu's Lingyi Zhihui, which includes two diagnosable diseases: diabetic retinopathy and glaucoma. The other is the chest multi-disease AI-assisted diagnosis product by UNITED IMAGING, which received CE certification in 2020 and covers three diseases: pneumonia, pulmonary nodules, and chest fractures. In addition, Vistel’s multi-disease fundus image-assisted diagnosis software has been under review since entering the innovative medical device channel in 2020, and the model includes a total of 13 diseases.

Airdoc's multi-disease fundus product has not yet disclosed regulatory review and approval news, but some of its research achievements have been disclosed. In July 2022, a research team led by Professor Wei Wenbin, Vice President of Beijing Tongren Hospital affiliated with Capital Medical University, collaborated with Airdoc and IKANG Group to publish a research paper titled "Artificial Intelligence for Screening of Multiple Retinal and Optic Nerve Diseases" in JAMA Network Open (IF=13.4), a subsidiary journal of the Journal of the American Medical Association.

Research results show that the AI algorithm model trained on fundus photographs performed as sensitively as senior fundus experts in screening for 10 common fundus diseases during real-world validation across multiple centers in China, while reducing screening time by approximately 75%. This trial partially validates the diagnostic algorithm capabilities and application value of retinal (fundus) imaging artificial intelligence for multi-disease diagnosis.

In the comprehensive discussion of the above four multi-disease artificial intelligence systems, UNITED IMAGING possesses the only AI serving chest CT imaging, while the other three are focused on fundus imaging. These two disease categories boast the richest data and the longest history of development, placing them at the forefront of the multi-disease era.

However, the amount of data on disease types can illustrate the development direction chosen by artificial intelligence companies when planning AI research and development for multiple diseases.But to transform it into a product, a more decisive factor is the number of diseases covered by the multi-disease AI.

The Two Multi-Disease AI Systems Approved by Regulatory Authorities Have Distinct Characteristics. The multi-disease AI for chest conditions includes pneumonia, pulmonary nodules, and fracture signs, which vary significantly and can be easily distinguished visually by either humans or machines. On the other hand, the individual single-disease AI systems for pneumonia, pulmonary nodules, and fractures are relatively mature, with lower difficulty in research and development as well as clinical trials. Therefore, this product obtained CE certification as early as 2020.

Baidu Lingyi Zhihui's multi-disease AI for fundus diseases also has relatively simple distinguishing features, and the cost and difficulty of obtaining fundus data are low. In addition, the multi-disease AI involves only two types of diseases, and the complexity of clinical trials remains within a controllable range. In contrast, although Airdoc's "10-disease" AI and Vistel's "13-disease" AI also focus on fundus diseases, the large number of diseases involved makes the design of clinical trials extremely difficult.

In this regard, some experts said: "Although multi-disease AI has entered the green channel for the review and approval of innovative medical devices,Under the current medical device review system, it is basically impossible to complete the approval."Therefore, to break through the clinical trial phase, companies must work with the Center for Device Review to explore new approval pathways, which may take a long time."

In summary, multi-disease artificial intelligence is still in its early stages of development. Certification tends to focus on areas with high throughput and where single-disease AI is relatively mature. Moreover, the number of covered diseases is strictly controlled to avoid the risk of ineffective investment during the research and development process. For AI systems covering a larger number of diseases, due to a lack of explainability, such AI systems still require innovation in the approval pathway.

The R&D process for multi-disease AI is long and arduous, but is its value able to match the investment of enterprises? The answer is currently uncertain.

A relatively simple way is to compare it with single-disease AI. When doctors review images, they usually conduct a comprehensive analysis of various signs displayed in the imaging data, while single-disease AI can only analyze one type of sign in the image, and thus can only be used in specific scenarios. Its greater value lies in assisting doctors to reduce costs, improve efficiency, and fill in the gaps.

The value of multi-disease AI depends on the quantity of N.For multi-disease AI with a smaller N and significant differences in signs, its accuracy is not much different from the combination of multiple single-disease AIs. Many artificial intelligence platforms with relatively rich product lines also integrate multiple single-disease AI products and are referred to as multi-disease AI. Therefore, when N is small, the advantage of multi-disease AI over single-disease AI in auxiliary diagnosis is relatively limited.

In terms of R&D difficulty, multi-disease AI must play an effective role in clinical practice. It not only requires research on each included disease to ensure diagnostic capabilities equivalent to multiple single-disease AIs, but also needs to guarantee that the results are sufficiently accurate and non-conflicting in complex cases involving multiple diseases. Therefore, the development of multi-disease AI is based on the capabilities of single-disease AI, with an overall R&D difficulty greater than or equal to that of multiple single-disease AIs. When N is small, multi-disease AI does not offer significant advantages in terms of benefits.When the N value is large enough, covering a sufficient number of patients, and capable of handling multimodal data, the value of multi-disease AI will quickly outpace that of single-disease AI.

However, the ideal state of a larger N value presents excessive technical and regulatory challenges. At present, most innovations brought by multi-disease AI are not about technological breakthroughs but rather business model innovations that reduce the cost for medical institutions to acquire medical AI. In this context, the relationship between multi-disease AI and single-disease AI is not one of replacement but of collaborative empowerment, using intelligent means to connect every medical scenario.

Specifically, the main application scenarios of single-disease AI currently remain in hospitals, as their price and functionality limitations mean that medical AI is mostly purchased by tertiary hospitals. After all, the role of grassroots medical institutions and physical examination centers lies in "screening and prevention," a task that a single AI system would find difficult to accomplish alone. At the same time, purchasing multiple AIs is too expensive. In this context, multi-disease AI can serve as a complement to single-disease AI, offering grassroots medical institutions a relatively affordable option with broader coverage.

Of course, since the review process includes the diagnostic capability for individual diseases, multi-disease AI could theoretically also be applied in the crucial hospital setting, initiating market competition with single-disease AI. However, the lack of a mature pricing system and actual data currently leaves this model unsupported.

Admittedly, multi-disease AI at the current stage still faces numerous challenges and does not possess the ability to change the era. However, from a long-term perspective, multi-disease AI and even full-disease AI will be the inevitable development of medical AI.

At this stage, if medical artificial intelligence companies want to develop multi-disease AI, they should not focus on multi-disease AI as the core of their research and development.

As the first company in China to obtain multi-disease AI registration certificates, UNITED IMAGING did not rush to accelerate the R&D of multi-disease AI in the following two years. Instead, it chose to perfect the creation of an "all-stack" solution, allowing its AI to cover as many diseases and processes as possible.

In this regard, a senior executive of UNITED IMAGING mentioned in a past interview: "Both single-disease AI and multi-disease AI serve clinical needs, and multi-disease AI requires single-disease AI as support. Currently, the fields covered by artificial intelligence development are limited. To further develop artificial intelligence in the future, a more solid foundation needs to be laid."

As for how UNITED IMAGING chooses its AI research and development path, he summarized the principles of UNITED IMAGING's choices with four idioms. At present, companies still tend to focus more on"Looking for a Needle in the Haystack"And"Turn a Blind Eye"Specifically, the former refers to many diseases that are very difficult to detect on imaging, such as small calcifications on mammograms or metastatic tumors in the brain on MRI. These applications will gain popularity the fastest; the latter refers to when the human eye focuses on one disease, it often overlooks another. For example, in breast DCE-MRI scans, flexible registration of time-series images is required, followed by complex quantitative analysis of contrast agent changes. These "unseen things" by the human eye are precisely where AI can play a significant role.

"Viewing Flowers through the Mist", that is, some pathological imaging manifestations are not obvious, such as lesions being partially obscured by other objects, appearing faint and indistinct, seemingly present yet absent;"Capture Shadows and Chase Shadows", that is, mimicking doctors to make some wise guesses, judgments, and decisions based on scattered patient information, various clinical information from imaging or beyond imaging. This kind of application scenario requires extra caution. Although these two scenarios are not the current focus of UNITED IMAGING, they will be explored to a certain extent on the basis of prudent consideration.

Overall, although multi-disease AI deviates from the review and approval process of single-disease AI, it heavily relies on the research and development support of single-disease AI. From the above four directions, it is not difficult to see the ideal form of multi-disease AI or even full-disease AI — this will be a complex system capable of integrating multiple modalities of information for decision-making and simultaneously handling multiple diagnostic pathways.

Clearly, we are still far from the ideal AI, but the existing AI is still in rapid development, continuously optimizing the current healthcare system. There is no need to worry too much about the future, after all, the current achievements have not made everyone stagnant, and there are always explorers who tirelessly strive along the path of dreams.