Deep Learning in Radiation Therapy: Innovations from Stanford's Prof. Lei Xing and Huiying AI Cloud Platform

As a technical and practical expert in medical imaging, Professor Lei Xing, Director of the Center for Medical Physics at Stanford University, has recently been invited consecutively to attend domestic and international academic conferences on radiology and radiation therapy, including the 2017 Annual Meeting of the American Association of Physicists in Medicine (AAPM), the International Symposium on Image Computing and Digital Medicine in Chengdu, the First Summit on Artificial Intelligence Applications in Medicine, the Annual Meeting of the American Society for Radiation Oncology (ASTRO), and events organized by the Radiology Specialist Branch of the Beijing Medical Doctor Association.

In recent years, AI technology has been widely discussed. How to integrate AI into clinical practice, help radiologists unlock greater value, and use AI to penetrate the entire chain of tumor treatment starting from imaging diagnosis are also current hot topics. This is not only a game of data and algorithms but also a synergy between diagnosis and treatment.

Dr. Lei Xing, Professor of Distinction at Stanford University and Chair of the Department of Medical Physics, presented the Huiyi Huiying AI Analytics Cloud Platform (English Version) at various summits. Professor Xing’s keynote presentation, titled “Applications of AI in Clinical Diagnosis and Treatment,” received widespread acclaim from attendees.

Professor Lei Xing is a tenured professor at Stanford University, holding joint appointments in the Department of Electrical Engineering, the Molecular Imaging Program, the Biomedical Informatics Program, and Bio-X. With over two decades of experience in teaching and research in medical imaging, medical physics, and medical informatics, he has published more than 300 peer-reviewed papers and served as principal investigator for major research projects funded by the National Institutes of Health (NIH), the Department of Defense (DOD), the National Science Foundation (NSF), the American Cancer Society (ACS), and the Radiological Society of North America (RSNA). His accolades include the American Cancer Society Research Scholar Award, the Best Paper Award from the American Association of Physicists in Medicine (AAPM), and the Google Research Award. Dr. Xing is also a Fellow of both the AAPM and the American Institute for Medical and Biological Engineering (AIMBE).

The following is a curated collection of Dr. Xing Lei’s original shares, edited without altering the original meaning:

Discussion on the Application of AI in Medicine: To begin, it is essential to clarify what medicine entails. Modern medicine is evidence-based medicine, which primarily comprises three components: clinical experience, scientific data, and patients’ actual conditions and preferences.

These three points appear simple but are challenging to implement in clinical practice. Medicine is both a science and an art, involving the quality and accumulated experience of physicians. It is precisely for this reason that many issues exist in clinical medicine, creating excellent opportunities for the application of AI in the medical field.

First, clinically collected data often carry certain biases and uncertainties. Using such empirical data to inform clinical decisions is a highly complex process, which is typically difficult or even impossible to describe using conventional simple mathematical models.

Secondly, there is currently significant overlap in services between the radiology and radiation oncology departments, resulting in high workloads.

Third, the construction of medical imaging departments is costly, and there is a significant disparity in infrastructure between major tertiary hospitals and those at the municipal, county, and township levels. In contrast, AI technology offers incomparable advantages, as its marginal costs continue to decrease with expanded application scale. The adoption of AI can liberate physicians from many repetitive and inefficient tasks, thereby raising the average proficiency and efficiency of healthcare professionals and enabling them to devote their valuable time and energy to more valuable, creative clinical work.

Furthermore, clinical trials are often time-consuming, typically requiring three to five years from the availability of results to actual clinical application. The application of AI technology can accelerate the efficiency of clinical trials. It is no exaggeration to state that AI is an indispensable technology for realizing personalized medicine.

In evidence-based medicine, clinical decisions are closely tied to evidence and data. With the rapid advancements in science and medical technology, the growing volume of data has expanded the dimensions involved in clinical decision-making. From a cognitive perspective, an individual’s capacity to simultaneously consider variables is quite limited. A person who can comfortably process ten factors at once could be considered exceptional. In reality, however, oncologists often need to consider far more factors than this, making the complexity and uncertainty of their decisions readily apparent.

Furthermore, we are living in an era of knowledge explosion. Articles are published every day, yet the half-life of this knowledge averages only a few years, making it prone to becoming obsolete and forgotten. Therefore, leveraging AI to rapidly extract key insights from data for clinical application is of paramount importance.

Experts are undoubtedly familiar with computer-aided diagnosis, or CAD. This field has been actively pursued since the 1980s, giving rise to numerous companies. Among them, R2 Technology, later acquired by Hologic Inc., stands out as a prominent representative.

With the enhancement of computational power and the advent of GPUs, deep learning has gradually come into the public eye. Today, deep learning is widely applied in our daily work and life.

The process of machine learning is highly analogous to the cognitive development of children: machines are trained on extensive existing datasets to distinguish between categories, such as cats and dogs. After processing a sufficient number of samples, the machine can accurately identify these animals in novel contexts. This capability has numerous real-world applications. A classic example is radiologists making diagnoses based on their clinical experience, while taking into account patients’ medical histories and other relevant clinical information.

Machine learning can be divided into three major categories. We must first train models using large amounts of data before we can analyze, judge, and predict new data. In recent years, deep learning has gained significant momentum, which may be largely attributed to the high-profile Go matches between humans and AI. In fact, this is not the first time machines have defeated humans. A decade ago, IBM’s Deep Blue already defeated Garry Kasparov, the world chess champion. The recent two man-machine Go matches featuring AlphaGo have pushed artificial intelligence to new heights, as Go has long been hailed as “the crown jewel of human intelligence.”

Deep learning and deep reinforcement learning are currently the most widely used techniques in the field of medical imaging. They can be employed to address many challenging problems that were previously unsolvable. A notable success story this year is the research on skin cancer detection published in Nature by S. Thrun from the Department of Computer Science at Stanford University. Based on nearly 130,000 skin cancer cases, they trained a convolutional neural network (CNN) deep learning model and then evaluated it using approximately 2,000 test samples. The performance of this model was comparable to that of experienced dermatologists.

Huiyi Huiying has consistently remained at the forefront of the industry in fully harnessing the potential of artificial intelligence to build a global intelligent medical imaging platform. The company is leveraging deep learning to simulate the human brain’s cognitive processing of three-dimensional images, achieving remarkable progress. The human brain processes images across five dimensions, including color, shape, and abstract recognition. Consequently, algorithms simulating these cognitive processes vary across different regions. Through extensive practical experience, Huiyi Huiying has made significant strides in accumulating substantial clinical data, enhancing computational efficiency, optimizing deep learning algorithms, and developing models capable of continuous self-improvement.

I. AI Predicts Side Effects of Radiotherapy for Liver and Lung Cancer in Treatment Planning

AI has numerous applications in radiotherapy, such as predicting potential side effects of radiotherapy for liver and lung cancer. Deep learning models can replace existing nomograms to achieve more accurate personalized predictions. Current methods evaluate predicted radiotherapy toxicity using certain metrics after dose distribution is provided, such as determining the mean dose threshold above which clinically unacceptable toxicity occurs. Deep learning models can supplant these conventional metrics, thereby eliminating reliance on a limited number of parameters for clinical decision-making.

We have developed the world’s first deep learning–based radiotherapy model for liver cancer, leveraging extensive data from a large cohort of patients, including imaging, treatment plans, and post-treatment toxicity. This model enables convenient and accurate prognosis prediction for new patients. Compared with actual clinical outcomes, the deep learning–based predictions demonstrate significantly higher accuracy than those of existing models. This represents the first substantial application of deep learning in translational radiotherapy medicine.

II. Applications of AI in Treatment Planning and Image Analysis and Reconstruction

Radiotherapy is a highly individualized process that requires continuous optimization based on the patient’s anatomical features and tumor location, making it a complex procedure. Consequently, treatment planning is an extremely time-consuming task. Generally, it takes an experienced technician several hours to several days to develop a treatment plan for patients with complex cases. Our department treats approximately 3,000 patients annually, which underscores the significant amount of time and effort required. The application of deep learning in treatment planning can substantially improve both efficiency and quality. Currently, we have successfully applied Google AlphaGo’s algorithm to the optimization of treatment plans. A comparison between plans generated using this algorithm and those created manually demonstrates that our approach yields superior results compared to existing methods and is readily implementable on linear accelerators.

When using MRI for imaging of moving organs such as the heart, or for MRI-guided radiotherapy, it is necessary to rapidly acquire and reconstruct MRI images in real time. Current MRI systems can generate 4–8 two-dimensional image frames per second. Deep learning models can significantly reduce the time required for three-dimensional MRI reconstruction, making “real-time” four-dimensional MRI reconstruction possible. Machine learning can also be applied to CT image reconstruction. During CT imaging, patients typically receive a radiation dose of 1–5 cGy; if the dose is reduced, noise levels increase significantly, leading to degraded image quality. We construct a model using prior patient CT data and perform joint reconstruction with new low-dose data, thereby substantially reducing the radiation dose associated with CT imaging.

III. Applications of AI and Impactomics in Image Analysis and Clinical Practice

Research on the application of deep learning in disease screening and detection is also highly active, with new methods and technologies emerging continuously. For instance, our laboratory is leveraging deep learning to improve existing methods for prostate cancer detection. Broadly speaking, when elevated levels of prostate-specific antigen (PSA) are detected, diagnosis typically relies on magnetic resonance imaging (MRI) and biopsy. By utilizing extensive patient imaging data and diagnostic outcomes, we employ deep learning to perform diagnostic analysis of prostate tumors, thereby identifying all lesions and assessing the degree of tumor progression. This approach aims to avoid or reduce the need for biopsies, thus minimizing patients’ costs and suffering.

I trust that everyone is familiar with radiomics. In this context, I would like to introduce a book titled “Radiomics and Radiogenomics,” co-authored by myself and three colleagues from Stanford University, which is scheduled for publication in early summer next year. Radiomics enables the screening and selection of feature values. We recently published an article in Radiology on the use of radiomics to study the prognosis of gliomas. Furthermore, a critical clinical challenge in glioma treatment is distinguishing between pseudoprogression and true progression. During regular follow-up examinations after treatment, the management decisions for these two scenarios are entirely different: discontinuation of therapy for the former, and continuation of treatment for the latter. This issue can be effectively addressed using neural networks.

To date, research and applications in computer vision have primarily been conducted in two-dimensional space. In contrast, medical imaging is almost entirely three-dimensional or even four-dimensional, as seen in CT, MRI, and PET scans. The field of true medical image learning and processing is only just beginning. Beyond algorithms, another key factor in the application of AI in medicine is data and how to facilitate effective data exchange.

Deep learning requires continuous model improvement, which in turn necessitates massive amounts of data. The Huiyi Huiying platform can store data in the cloud in widely accepted formats, enabling sharing among multiple experts—a capability that is crucial for big data processing and deep learning. In addition to imaging and electronic medical record (EMR) data, we aim to integrate patients’ genomics data, wearable device data, and other sources in the future, thereby facilitating more convenient multi-dimensional deep learning. Data sharing is indeed a significant challenge, involving technical, managerial, and social aspects. An article published in The Lancet Oncology explored this issue in depth, noting that the barriers to data sharing mentioned therein are also prevalent in China. I believe that with the emergence of large volumes of high-quality data, the prospects for AI in clinical applications are very promising.

To summarize, today I primarily used examples to discuss the applications of deep learning in diagnosis, image reconstruction, and radiotherapy decision-making. Currently, the application of artificial intelligence in the medical field is still in its early stages; it is estimated that it will take another three to five years before substantial clinical implementation is truly realized. There is still a long road ahead. As for the “perennial question” of whether AI will eventually replace physicians, let us revisit this topic in a more relaxed setting at a later time.