Deep Learning's 'Black Box' Isn't Holding Back Healthcare Adoption

Earlier this year, Sebastian Thrun, an artificial intelligence scientist at Stanford University, and his colleagues demonstrated that an algorithm known as “deep learning” can diagnose the potential cancer risk of skin lesions with accuracy comparable to that of board-certified dermatologists.

Numerous journals, including Nature, have reported this new discovery, hailing it as ushering in a new era of “software-based diagnosis,” in which artificial intelligence will assist physicians, and even replace them.

Experts state that medical imaging modalities, such as X-ray, CT, and MRI, align almost perfectly with the strengths of deep learning software, which has achieved significant breakthroughs in facial recognition and processing over the past few years.。

Numerous companies are already pursuing this technology. Verily, Alphabet’s life sciences subsidiary, partnered with Nikon last December to develop algorithms aimed at identifying the causes of blindness in diabetic patients. Radiology has been dubbed the “Silicon Valley of medicine” because it generates vast amounts of detailed images that serve as raw material for software analysis.

Although Professor Thrun’s team achieved a very high prediction accuracy, no one can definitively determine which features the “deep learning” algorithm relies on to classify skin lesions as malignant or benign. This is why the “deep learning” approach is referred to as the medical black box problem.

Unlike traditional vision software, where programmers define rules—for example, that a stop sign has eight sides—in “deep learning,” algorithms discover the rules themselves but typically leave no clues to explain their decisions.

“In the case of black-box medicine, doctors do not know what is happening, because no one can; its very nature is opacity,” said Nicholson Price, a legal scholar at the University of Michigan.

However, Price stated that this would not hinder the development of the healthcare industry. He likened “deep learning” to a drug that yields benefits through mechanisms that are not yet fully understood. For instance, lithium was approved for the treatment of bipolar disorder due to its ability to modulate emotional fluctuations, even though its specific biochemical mechanism remains to be elucidated. Similarly, aspirin, the most widely used medication worldwide, had an unclear mechanism of action during the first 70 years of its clinical use.

Price also stated that the black box issue would not pose a problem for the U.S. Food and Drug Administration (FDA), as the FDA will regulate software used for treating or preventing diseases, in addition to approving new drugs.

In a statement, the FDA indicated that over the past two decades, it has approved the development and application of several “image analysis applications relying on various pattern recognition, machine learning, and computer vision technologies.” The agency confirmed that an increasing number of software products are “powered by deep learning,” and that the FDA permits companies to maintain the confidentiality of their algorithms.

The FDA has given the green light to at least one “deep learning” algorithm. In January, the FDA approved software developed by Arterys, a private medical imaging company based in San Francisco. Its algorithm, “DeepVentricle,” can analyze magnetic resonance images of the ventricular endocardial contours and calculate the patient’s cardiac blood volume and stroke volume. This calculation is completed in under 30 seconds, whereas conventional methods typically require an hour.

The FDA required Arterys to conduct extensive testing to ensure that the results generated by its algorithms were consistent with those produced by physicians. John Axerio-Cilies, the company’s Chief Technology Officer, stated, “You need to use statistical methods to demonstrate that your algorithm’s outputs are consistent with expectations and align with the claims made in marketing materials.”

To test their software, the team led by former Google vice president Thrun evaluated 129,405 expert-reviewed images of skin conditions. These images covered 2,032 different skin diseases, including 1,942 images of confirmed skin cancer.

Ultimately, the software outperformed 21 dermatologists in identifying which skin lesions were potentially cancerous.

Robert Novoa, a dermatologist and skin disease researcher at Stanford University, stated, “When dermatologists see the potential of this technology, I believe most will embrace it.” However, he and other team members declined to disclose whether they plan to commercialize the software.

“Any notion that doctors will soon lose their jobs is mistaken,” said Allan Halpern, a dermatologist and president of the International Society for Digital Imaging in Dermatology. “I believe the threat is quite the opposite; algorithms can significantly drive up demand for dermatology services.”

This is because individuals who test positive on screening tests still require a biopsy. Halpern said, “Deep learning software can play a role in primary care settings to serve the general population, as reliance on dermatologists for this task would be constrained by the limited number of specialists.。”

Axerio-Cilies stated that the company would tend to provide deep learning tools directly to consumers. For example, people can scan their own skin lesions to determine whether they need to see a doctor. Some non-artificial intelligence mobile applications, such as Mole Mapper, have already provided services for tracking suspicious skin lesions and recording changes over time.

However, Halpern stated that he does not believe consumers are ready to adopt this diagnostic system, as it does not inform them whether a skin lesion has a 5% or 50% probability of progressing to cancer.

“We are not good at using probabilities,” he said.

Source: www.technologyreview.com

Author: Monique Brouillette

Translation: Zuo Bangyou