Google's Neural Network Algorithm Achieves Over 90% Accuracy in Detecting Diabetic Retinopathy

On November 29 (U.S. local time), Google researchers published a paper in the Journal of the American Medical Association (JAMA), demonstrating that Google’s deep learning algorithm, trained on extensive fundus image data, can detect diabetic retinopathy with an accuracy exceeding 90%.

Google has been making significant moves in the field of diabetes. This September, Verily, the U.S. life sciences company under Alphabet, Google’s parent company, joined forces withSanofi, a well-known French pharmaceutical manufacturer,Invest approximately $500 million to establish a diabetes joint venture integrating equipment and servicesOnduo will leverage the strengths of both parties:Verily’s expertise in microelectronics, analytics, and consumer software development, combined with Sanofi’s clinical specialization and experience in delivering innovative therapies to patients with diabetes, aims toHelping people make better decisions for daily health solutions, ranging from improved medication management and early prevention to habit formationImprovement. It appears that Google has made substantial efforts in the prevention and treatment of diabetes. What surprise does it bring us this time?

ForDiabetic Retinopathy (DR)Examples of retinal photographs. On the left, a healthy retina is visible; on the right, the retina shows damage, with hemorrhage and fluid leakage in the eye.

Diabetic retinopathy is an ocular condition affecting patients with diabetes and is the leading cause of rapid vision loss worldwide, with approximately 415 million diabetic individuals at risk. If detected early, the disease is treatable; otherwise, it is highly likely to result in irreversible blindness.

One of the most common methods for detecting diabetic eye disease involves a specialist examining photographs of the posterior segment of the eye to identify signs of the disease and, if present, assess its severity. This approach falls within the realm of intuitive and experience-based medicine. Although annual screening is generally recommended for all patients with diabetes, many individuals are unable to access adequate specialist care due to resource constraints. Consequently, millions of people are not receiving the necessary care to prevent vision loss.

A few years ago (the exact date was not disclosed), a Google research team began investigating whether machine learning could be used to screen for diabetic retinopathy (DR). At its core, this approach relies on a deep learning algorithm capable of examining retinal images for signs of pathology, potentially helping physicians screen a larger number of patients, particularly in underserved communities with limited resources.

In collaboration with a group of physicians from India and the United States, Google created a dataset comprising more than 128,000 images, of which 9,963 were retrospectively obtained from EyePACS in the United States and three ophthalmology hospitals in India. Each image was graded three to seven times by board-certified ophthalmologists and used to train a deep neural network to detect diabetic retinopathy.

To validate the algorithm's performance, we compared its results with those from another set of images evaluated by seven American Board-certified ophthalmologists. The two datasets demonstrated sensitivities of 97.5% and 96.1%, and specificities of 93.4% and 93.9%, respectively. The algorithm achieved accuracy comparable to that of ophthalmologists, delivering high sensitivity and specificity.

Prediction accuracy of the algorithm (black curve) and eight ophthalmologists (colored dots) for detecting referable diabetic retinopathy (moderate or worse diabetic retinopathy or suspected diabetic macular edema) on a validation set of 9,963 images. The overlapping black diamonds on the graph indicate the algorithm’s operating point with high sensitivity and specificity.

“These results indicate that deep neural networks can be trained using big data to identify diabetic retinopathy or diabetic macular edema in retinal fundus images with high sensitivity and high specificity, without the need to specify lesion-based features.”

The paper states, “This automated system for detecting diabetic retinopathy offers several advantages, including consistency in interpretation (as the machine makes the same prediction each time for a given image), high sensitivity and specificity, and near-instantaneous reporting of results. Because the algorithm has multiple operating points, its sensitivity and specificity can be adjusted to meet the needs of specific clinical settings, such as prioritizing high sensitivity in screening contexts.”

Clearly, this is undoubtedly very exciting news for patients with diabetes. Although other forms of machine learning have been used in the past to diagnose diabetic retinopathy, deep learning represents a purer form of artificial intelligence because it does not rely on any guidance to identify specific features. Instead, it develops self-learning mechanisms from images and data.

Regarding the automated grading capability of the algorithm, Google researchers also noted: “Automated grading of diabetic retinopathy offers potential benefits, such as improving the efficiency and reproducibility of screening programs, expanding coverage, reducing barriers to access, and enhancing patient care through early detection. To maximize the clinical utility of automated grading, an algorithm capable of detecting suspicious cases of diabetic retinopathy is indeed necessary.”

However, there is still much work to be done before Google’s artificial intelligence algorithms can be widely adopted. For instance, the interpretation of 2D retinal photographs is merely one step in the diagnostic process for diabetic eye disease; in some cases, physicians also need to employ 3D imaging techniques to examine the various layers of the retina in detail. Researcher Lily Peng explained, “In certain scenarios, doctors use 3D imaging technologies, such as Optical Coherence Tomography (OCT), to conduct detailed examinations of the retinal layers.” Google’s DeepMind division is working on applying machine learning to this approach. In the future, these two complementary methods could be used in conjunction to assist physicians in diagnosing a broad spectrum of ocular diseases. Automated, highly accurate screening methods have the potential to help clinicians perform faster health assessments for patients.

Because neural networks learn the features most predictive of inferability, the algorithm may leverage previously unknown or overlooked features. Although this study used images from diverse clinical settings (hundreds of clinical sites, including three in India, hundreds in the United States, and three in France) to mitigate the risk that the algorithm relies on anomalously low-frequency data for prediction, it remains unknown which specific features were actually used. Deep neural networks are still a relatively new area within machine learning, and they have a long way to go before they can fully replace human experts.

As it stands, Google’s algorithm has been trained solely to identify diabetic retinopathy and diabetic macular edema; consequently, it is likely to miss non-diabetic retinal pathologies that it was not trained to recognize. Therefore, this algorithm cannot comprehensively replace ophthalmic examinations—such as visual acuity testing, refraction, slit-lamp examination, and intraocular pressure measurement—which must still be performed by specialized ophthalmologists.