The Unresolved Challenge in Medical AI: Integration, Trust, and Real-World Validation Beyond Google, Apple, and IBM

New findings from Google and Apple indicate that symptom self-reporting and artificial intelligence (AI) are challenging physicians in terms of diagnostic speed and accuracy, with similar implications for physician-led research. Nevertheless, barriers remain that could slow the implementation of AI in the healthcare sector.

Google obtained histopathology slide images from the Camelyon16 project, which challenged participants to develop cancer detection algorithms. After scanning 400 slides, Google reported that its machine learning platform was able to detect 92.4% of tumors.

Previous automated detection technologies could only identify 82.7% of tumors, whereas a group of physicians determined that only 48% of the so-called “tumors” in patients were truly confirmed as malignant.

Using gigapixel images, Google’s AI can detect tumors as small as 100×100 pixels within images measuring 100,000×100,000 pixels. It even identified tumors in two slides that had been incorrectly labeled as “normal.” In its published white paper, Google noted that missed, uncertain, or delayed diagnoses may affect up to 20% of cases, emphasizing that its AI-based image scanning algorithm is faster and more accurate than human physicians.

ResearchKit is Apple’s crowdsourcing solution for medical information, released in 2015. Studies can encompass any disease, condition, or treatment that researchers wish to track and are conducted entirely by healthcare professionals.

Regarding a recent study related to asthma, Mount Sinai Hospital stated,ResearchKit Data Can Serve as a Reliable Source for Physician DiagnosisThrough this platform, users can self-report on a variety of topics related to their disease, ranging from symptoms to treatment. Asthma was one of the initial five ResearchKit themes announced by Apple.

Fifty thousand users downloaded an app associated with the study, of whom 7,600 enrolled in the six-month research program. The study collected self-reported data on asthma management from participants and incorporated metadata on air quality and geographic location. Researchers then compared the user-submitted data with air quality reports to assess how individuals managed their condition (for example, participants in Washington experienced more severe symptoms during regional wildfire events).

There is no doubt that AI and symptom self-assessment can function effectively under appropriate conditions; however, several obvious issues remain.

Many ResearchKit studies are open to the public, so you could become one of 50,000 users who download the app and participate in asthma research. There are ways to narrow the scope of participation, such as using TestFlight or assigning username/password authentication for participants, but most studies are simply limited to the App Store without verification. The ResearchKit website invites the public to download and use apps related to concussion, asthma, hepatitis C, and postpartum depression (among other topics).

This is likely the reason why researchers at Mount Sinai initiated their study: “The feasibility of using mobile health applications for observational clinical research requires rigorous validation.” Because ResearchKit apps are open to the public, they can be easily manipulated if enough individuals are willing to do so.

Google’s AI is impressive, but it still has its shortcomings. The study used only tissue sections from living organisms that could be digitized into gigapixel images. Although Google stated that “future work will focus on improving the AI using larger datasets,”However, it did not indicate whether it would concentrate its efforts on developing images with the highest pixel count.. If AI research were limited to high-resolution scans, there would still be more promising avenues to anticipate.

Neither platform is “better” than physicians. Like many other cutting-edge technologies, Google’s and Apple’s healthcare initiatives have reshaped the demand for healthcare professionals and their services, which remains consistently high.These technologies are designed to save doctors’ time in diagnosing and treating patients, bringing an end to the model where patients pay for prolonged survival.

Conceptually, there are many reasons to use these technologies. Self-assessment of treatment symptoms is a blind spot for doctors. When you report your treatment status to your doctor every few weeks or months, you stand in front of the doctor, and they only take a brief look at you. As long as you report truthfully, through ResearchKit, those doctors can see detailed information about your daily condition.

AI can scan and analyze images faster than doctors, serving as a good pair of “eyes” for physicians before patient treatment. Because AI is objective and its results lack contextual correlation, doctors must interpret the cold, rigid data before taking further action.

Meanwhile, do not forget that healthcare is also a business.IBM’s Watson AI engine has made concerted efforts to help cure cancer, with researchers spending tens of thousands of hours training it to analyze clinical data.They also provided the supercomputer with more than 600,000 medical records and two million pages of literature from 42 medical journals and clinical trials.Furthermore, Watson can access the medical records of 1.5 million patients, including their recovery outcomes, to facilitate in-depth research into optimal treatment protocols.。

IBM has integrated its Watson supercomputing platform into various industries, most notably healthcare: New York’s Memorial Sloan Kettering Cancer Center has partnered with IBM and WellPoint to enhance Watson’s ability to process and interpret oncology data.

For a long time, IBM has been exploring how to best leverage Watson, which boasts massive data repositories and the capability to process natural language queries, holding significant potential value for researchers, physicians, and other healthcare professionals. Healthcare providers can upload their electronic health records to Watson; by integrating medical resources—such as journal articles and clinical trial data—with human-in-the-loop training, the platform can be transformed into a tool for cancer detection.

This study is promising, but the Watson supercomputer has been dismissed from its more famous role.

The University of Texas MD Anderson Cancer Center invested more than four years and $62 million in a program known as the Oncology Expert Advisor. Of this amount, $61 million was not approved by the Board of Directors. IBM received $40 million of those funds.

Some of the issues areWatson was unable to read the hospital’s new medical data recording system, and over a four-year period, it was used only 12 times.. IBM has terminated its partnership with the University of Texas, stating that its system is “not yet ready for clinical research or clinical application in humans and is prohibited from use in patient treatment.”

Auditors reviewing procurement and compliance stated that during the transition period of implementing Watson, hospitals may have failed to consider other lower bids. In some instances, hospitals made payments even though Watson failed to meet its contractual performance targets.

In response to the audit, University President William McRaven stated, “The nature of research and development inevitably causes goals and expectations to evolve over time; in terms of specific research outcomes, this often renders the original contract obsolete. Where goals and expectations change over time,Incomplete documentation prevents determination of whether the revised milestone targets can be achieved.。”

Such developments do not help IBM, Google, Apple, or any other health-related technology service or company find solutions that benefit patients; they also raise questions about whether healthcare and commercial interests can be appropriately aligned.

Unfortunately, at times, those most affected by medical conditions do not have the luxury of waiting for regulatory agencies to resolve systemic issues, which leads to greater controversy when technologies are ready to transition from the laboratory to everyday use.

"In the healthcare sector,"Many great ideas stall at the technology stage, or more specifically, struggle to be integrated into existing systems..” John Sung Kim, founder of Five9 and DoctorBase, wrote in a new column for TechCrunch. “Regardless of whether Watson is marketed to small clinics or large hospitals,Healthcare institutions of all sizes are managing multiple software systems, many of which are incompatible.。”

While many experts attribute the shortcomings of the medical IT industry to the lack of integration between medical databases and software platforms, regulatory issues also exist.

Every application that interacts with patient data must comply with the Health Insurance Portability and Accountability Act (HIPAA), which safeguards data both during transmission between different databases and while at rest. Hospitals and other entities handling such data must ensure adherence to necessary privacy and security standards.

According to Kim,Startups in the healthcare IT sector face intense competition from electronic health record (EHR) vendors., executives at these companies do not want to see their businesses disrupted by a small micro-enterprise with an innovative platform.

Whether working for a small startup or a large vendor, technical professionals interested in the medical IT field must not only be well-versed in the fundamental building blocks of all software platforms (such as programming languages like C# and Python, as well as efficient management methodologies) but also familiar with the creative thinking that enables people to tackle complex problems.

That said, much of the software used in healthcare is complex and industry-specific, making it difficult for technical experts to master most of its functionalities without years of hands-on experience.

Health Level 7 (a framework and standard for retrieving electronic health data) and DICOM (imaging protocols) are just two platforms with which one should be familiar. However, given the importance of data protection, perhaps the most critical skill is a thorough understanding of all matters related to HIPAA.Regardless of the nature of your startup, nothing is more important than ensuring the protection of patient data.