Ambient Intelligence Illuminating the Dark Spaces of Healthcare: A Dialogue Between Shen Nanpeng and Li Fei-Fei

On March 26, at the 2021 Sequoia Global Healthcare Industry Summit held in Shanghai, distinguished speakers—including experts from top hospitals worldwide, new drug developers, pioneering scientists, and Chinese and American entrepreneurs—engaged in discussions on topics such as the construction of public health systems, leveraging technology to combat the pandemic, innovative approaches to medical diagnosis and treatment, and the evolution of digital healthcare. Neil Shen, Global Executive Partner of Sequoia Capital and Founding and Managing Partner of HongShan (Sequoia China), held an insightful discussion with Fei-Fei Li, Sequoia Professor and Co-Director of the Stanford Institute for Human-Centered Artificial Intelligence (HAI), on the topic of “AI + Healthcare.” VCBeat (WeChat ID: Vcbeat) has edited the transcript of the dialogue without altering its original meaning.

Shen Nanpeng:Congratulations on your election as a member of both the U.S. National Academy of Engineering and the U.S. National Academy of Medicine in 2020. Last year, you published an article in Nature titled “Illuminating Healthcare’s Dark Spaces with Ambient Intelligence.” Therefore, I would like to invite you to explain the connotation of ambient intelligence and how it illuminates the dark spaces in healthcare.

Li Feifei:First of all, thank you, Nan Peng, and thank you to HongShan for the invitation. Even though we are separated by the Pacific Ocean, I still want to wish everyone a good morning!

You mentioned that the article in Nature was co-authored by myself, my collaborator—Stanford Medical School Professor Prof. Arnold Milstein—and my student Albert Haque, and published in October 2020. In fact, this work has been ten years in the making. It highlights how healthcare settings involve a multitude of human behaviors. While we often focus on extensive trial data, X-ray images, or cellular conditions, human behavior and actions are ultimately critical to patients’ physical conditions, as well as their recovery and healing processes within the healthcare context.

This is, in fact, a particularly overlooked source of information. In medical settings, we tend to focus on blood tests, medical imaging, and cellular data; however, to truly enable patients to achieve long-term health and well-being, we must ultimately return to human behavior.

Under this premise, my collaborators and I identified a new opportunity brought by artificial intelligence a decade ago. Through sensors, AI enables us to collect such data. Most importantly, it not only gathers information on the environment and human behavior but also performs intelligent analysis, allowing us to determine whether a patient’s condition has changed and whether the actions of doctors and nurses have influenced the patient’s recovery. This information is critically important.

The greatest inspiration I drew was from autonomous driving. A decade ago, Silicon Valley was the birthplace of this technology. As the then-director of the Stanford Artificial Intelligence Laboratory, I recognized that autonomous driving emerged from the integration of sensors, AI algorithms, and overall system architecture. We applied this concept to healthcare scenarios, giving rise to the idea of ambient intelligence.

Shen Nanpeng:Where can this be applied in today’s healthcare scenarios? In China, we have seen AI applications in diagnostic medicine. For instance, in chest X-ray interpretation, artificial intelligence is increasingly capable of replacing the work of physicians, even highly experienced ones, enabling hospitals and the broader medical community to share previously accumulated knowledge. What other significant applications have you observed in different settings? Which specific scenarios do you believe will see breakthroughs in the near future?

Li Feifei:This is an excellent question. Our commitment has never been to replace humans, but rather to augment human capabilities. In our article published in Nature, we illustrated numerous scenarios, including those in hospital and home settings.

In hospitals, we have long focused on the Intensive Care Unit (ICU). In the United States, annual ICU expenditures account for 1% of the entire GDP, making it a critical healthcare setting. Patients in the ICU are engaged in a life-and-death struggle, placing immense pressure on medical staff. Under such circumstances, many protocols in U.S. ICUs must be strictly enforced. For instance, how can bedridden patients with limited mobility be prevented from developing pressure ulcers? Pressure ulcers pose a significant threat to patient safety in hospitals. Caused by muscle stiffness and poor blood circulation, pressure ulcers cause severe suffering for patients and increase medical costs. How can we address this issue?

We have identified that “movement” is the most critical factor. Patient mobility plays a vital role in pressure ulcer prevention and is particularly important in the Intensive Care Unit (ICU). But how can patient movement be measured? Placing sensors under the bed often fails to yield accurate measurements. Currently, one common approach relies on manual observation—for instance, having nurses document patient repositioning in the electronic medical record every two hours. However, this method is highly inaccurate and rudimentary. By leveraging AI-enabled smart sensors, we can monitor patients’ repositioning and movement in real time. This information is crucial for both nursing care and clinical treatment. As this small example demonstrates, such technology can have a significant impact.

Let me discuss another home-based scenario, which is the medical context I am most concerned about. Global aging will result in an increasing number of elderly people spending extended periods at home during their later years; moreover, old age is a life stage characterized by a higher prevalence of chronic diseases. In the context of global aging, how can we enable older adults to achieve better health and maintain their independence in daily living?

For instance, if chronic conditions in the elderly are addressed promptly, issues that can be resolved with antibiotics would not require a visit to the emergency room. But how can we detect whether an older adult is developing an infection early on? Are there changes in heart rate or respiration? Have their eating and sleeping patterns changed, especially if they have been largely inactive throughout the day? You might even observe that they have withdrawn from their usual social activities.

Where does this information come from? Generally, there are only two sources. The first is caregivers, whether family members or domestic care workers; however, the information they provide is often inaccurate and unsustainable. The other source is wearable devices, which I believe hold significant promise for future development. Nevertheless, wearable devices have their own limitations. They are not particularly popular among the elderly, and they cannot observe many behavioral issues in older adults as comprehensively as human observation can. Through devices and sensors, we can continuously monitor changes in the behavior of the elderly and capture critical health-related information, transmitting it promptly to family members and healthcare professionals. As mentioned earlier, an elderly patient with a chronic condition might require only antibiotic intervention, rather than having their condition deteriorate to the point of needing emergency room visits or hospitalization two weeks later.

Shen Nanpeng:The application of artificial intelligence in the healthcare industry is likely a long-term trend. Another factor is COVID-19. While it was a short-term event, how did it drive innovation within healthcare systems? When humanity faces such disasters, we must address the immediate challenges while also leveraging the opportunity to foster innovation in the healthcare sector. What lessons can be shared with everyone?

Li Feifei:The COVID-19 pandemic has had a profound impact on everyone present, affecting personal lives, daily routines, and careers alike.

First, I believe a profoundly significant impact has been that the world has undergone a human-centric and deeply humbling experience. No matter how advanced our science and technology have become in the 21st century, we still face many insurmountable challenges posed by nature and our own physical health. Although my remarks are not directly related to science and technology itself, as a technologist, I have always emphasized to my students that this focus on people and their health is of paramount importance. The COVID-19 pandemic has delivered a profound shock to us all.

Regarding specific technical points, I believe the following are key.

First, telemedicine. As a patient who has lived in the United States for many years and sometimes is so busy that I can only communicate with my doctor remotely, I have always wondered why telemedicine has not been widely adopted. The COVID-19 pandemic, however, rapidly accelerated its adoption. Therefore, I believe it has promoted the development of the entire ecosystem related to telemedicine.

At the same time, we are equally attentive to ambient intelligence. I have clearly observed a marked shift in attitude among my partners—whether Stanford School of Medicine or another major U.S. healthcare system—toward leveraging ambient intelligence to sustainably capture patient data and even make technical assessments for patient management. This approach has been warmly welcomed.

The public health crisis you mentioned is essentially an “infodemic.” Many argue that while COVID-19 was not the first major crisis, it was the first infodemic. The term “infodemic” refers to the rapid spread of both accurate and misleading information. This phenomenon has had a profound impact on technology and society. Many of my colleagues in medical school have observed that technology plays both beneficial and detrimental roles in information dissemination: the internet enables the swift transmission of information, yet AI also facilitates the spread of misinformation. Thus, the COVID-19 pandemic has exerted far-reaching influences across multiple dimensions.

Shen Nanpeng:You spoke about the infodemic in the context of the COVID-19 pandemic. Now, as you lead the Human-Centered Artificial Intelligence Institute, I find this initiative highly commendable. As an institute, it focuses not only on technological advancements but also on the societal changes driven by AI, particularly its ethical implications and broader social impacts. Could you share some insights and interpretations from HAI (Stanford University’s Institute for Human-Centered Artificial Intelligence) since its establishment to the present day, especially regarding this interdisciplinary field?

Li Feifei:This is indeed a critically important issue. As a scientist and technologist, I have undergone significant personal and professional growth since entering the scientific field two decades ago. I never anticipated that science, which I hold in such high regard, would ultimately become a driving force for social change.

In this process, we recognized at the time of HAI’s establishment two to three years ago that AI is not merely a technical field; its sociological and ethical implications are equally profound. Within this institution, we have established critical areas of study and research spanning from fundamental research and education to policy.

The first is economics. Economics is a social science, but it is also a highly significant discipline closely related to human life. Particularly in the digital economy and the human capital market, AI has brought about substantial changes. We currently have several world-leading economists driving this research forward.

Another critical domain is law. While law intersects with ethics, the advent of AI—from autonomous driving and healthcare to government operations—is challenging many fundamental assumptions underpinning traditional legal frameworks. Faculty members from our law school have been actively involved in numerous initiatives at HAI (Human-Centered Artificial Intelligence). On one hand, they are examining how governments can leverage AI technologies to enhance operational efficiency. On the other hand, they are exploring how to formulate sound policies and laws that both foster innovation and address the myriad novel issues raised by AI.

Another area intersecting with social ethics is the arts. We place significant emphasis on the arts, which constitute a most distinctive component of human civilization. They embody human expression, the condition of human nature, and the direction in which society ought to evolve. At HAI, we maintain deep collaborations with artists across the Department of Art, spanning music, visual arts, and literary arts.

Shen Nanpeng:Can you provide an example? Even if it has not yet been implemented, how does artificial intelligence interact with artists, musicians, and painters?

Li Feifei:Certainly. About two years ago, the world’s most renowned auction house sold the first AI-generated painting. This marked the first time globally that an artwork created by algorithms was purchased at a high price. Whether in visual arts or music, AI algorithms can produce highly intriguing works. This poses a challenge to human artists: What is the role of human artists? My AI can continuously generate replicas of Van Gogh’s The Starry Night. If audiences also appreciate art created by AI, what do human artists represent? Is it the expression of inner voices, or some other form of expression? Therefore, we are currently exploring many avenues to expand the artistic landscape. With such algorithms, we can further develop and enhance human expression and emotion—this serves as one example.

The final example I wish to discuss pertains to healthcare. Ten years ago, Professor Arnold Milstein and I jointly established a research laboratory. While our primary focus has been on technology-driven research, we formed an ethics committee several years ago. We have found that AI research in the healthcare sector raises numerous novel ethical issues, such as privacy concerns, data ownership, and fairness. These complex questions cannot be fully addressed by individuals with backgrounds solely in technology or medicine.

Therefore, we invited a law professor, an ethics philosophy professor, and two bioethics professors. These four professors formed the committee to engage in high-frequency, real-time interactions with us, helping us refine our research direction. Our goal is to advance technology while respecting universal values and human nature, ensuring that science creates benefits rather than causing unintended harm to patients or healthcare workers.

Shen Nanpeng:HAI also launched the world’s first “Artificial Intelligence Index Report.” Could you share it? This is a highly forward-looking initiative.

Li Feifei:This could indeed be the first of its kind globally. It is a project led by a senior professor at the Stanford Artificial Intelligence Laboratory in 2017. After merging the AI Index project in 2019, HAI has continued to support this initiative. Therefore, this is the fourth year we have released the AI Index.

The AI Index is dedicated to providing a fair, impartial, and comprehensive report on the global progress of artificial intelligence. It covers impacts and changes across various sectors, ranging from research and education to industry, manufacturing, and commerce.

2020 was undoubtedly a fascinating year, as the COVID-19 pandemic gave rise to several new trends. For instance, we observed significant changes in the application of artificial intelligence in drug discovery and design, marking a substantial impact.

Second, industrialization continues to advance robustly, with AI becoming increasingly industrialized. Many Ph.D. candidates and even professors are entering the industry. Additionally, AI faces challenges related to diversity. The AI workforce remains predominantly male, a persistent challenge that has yet to be adequately addressed.

Shen Nanpeng:I believe everyone is eagerly anticipating that the annual Index Report will continue to provide guidance for the industry. Let’s return to your early work. As a world-renowned AI expert and a leading figure among Chinese AI professionals, could you please share what motivated you to initiate the ImageNet project? How did it drive progress and bring about revolutionary leadership in deep learning within artificial intelligence? What was your original intention behind undertaking this endeavor?

Li Feifei:The ImageNet project was launched in 2006; I recall that I didn’t have any gray hair back then. Let me first explain why this project later garnered significant attention.

Since 2010, ImageNet has hosted an annual ImageNet Challenge within the academic community. The competition requires participants to use AI algorithms to classify one million images across one thousand categories. In 2012, Canadian Professor Geoff Hinton and his students secured first place in the ImageNet Challenge by employing what was then a relatively conventional algorithm known as the convolutional neural network. This milestone marked a historic turning point, heralding the “second spring” of neural network algorithms, sparking revolutionary advances in deep learning, and driving significant transformations over the past decade.

Why did they participate in ImageNet, and why did I create ImageNet? To answer this, we must go back to around 2006. At that time, AI was still a playful pursuit, a small niche within computer science. I had just completed my Ph.D. and become a professor—a relatively young one—and I kept pondering what the “North Star” of the AI field was. The North Star represents the ultimate aspiration of scientists. With my background in physics, I placed the greatest emphasis on identifying the most critical questions. For me, the paramount North Star was visual learning—the ability to recognize thousands of objects. This is the most fundamental capability; without it, humans would be unable to do anything else. We would not be able to stroll through streets or shop in stores.

Therefore, identifying the core challenge of object recognition was what I considered the “North Star” in the field of visual intelligence at that time. However, pinpointing this “North Star” proved futile. I found that although this issue garnered attention in 2006, we had no clear idea how to approach it. I was merely tweaking various model parameters for one or two categories. At the time, I felt this approach was highly unnatural and unreasonable. Human intelligence does not develop through a child’s day-to-day learning efforts; rather, it forms a relatively comprehensive understanding of the visual world during the first few years of rapid growth, driven by exposure to vast amounts of data. This cognition stems from learning based on big data.

From this point, I came to believe that we might have been on the wrong path all along—previously, we had been painstakingly tuning model parameters to recognize only one or two categories of objects. We adopted a new approach: leveraging big data to drive the learning of visual intelligence. I thought of using a dictionary; at the time, the largest set of visual object categories could be identified by extracting nouns from a lexical database. This particular dictionary is called WordNet. It contains 80,000 noun synsets, although some nouns do not represent physical objects—for instance, abstract nouns such as “anger” do not correspond to tangible entities.

Therefore, I extracted the noun labels for two to thirty thousand objects. Fortunately, 2007 was also a period of rapid growth for the internet, providing us with both the internet infrastructure and data sources. Our laboratory undertook substantial work, spending three years to consolidate over a billion images into a dataset of 15 million images. Our original intention was to use this dataset to reach for the “North Star.” This is the origin story of ImageNet.

Shen Nanpeng:"I believe this could be a story written into textbooks, as it has indeed driven much of the development in AI."

Li Feifei:You and HongShan have always had a keen sense for cutting-edge technology. As someone in the tech industry myself, I’m particularly eager to ask: as a Global Executive Partner at HongShan, what is your outlook on the development and application of AI in healthcare over the next decade?

Shen Nanpeng:As investors, our focus aligns with that of research scientists: we concentrate on the industry’s overarching challenges, including its fundamental questions, and identify its most critical pain points. This is the key to investing in high-quality companies.

Second, this aligns closely with your earlier remarks. Over the past one to two years, the field of new drug development has witnessed multiple platform-level breakthroughs in artificial intelligence. We are observing a burgeoning intersection between IT (Information Technology) and BT (Biotechnology), with their integration accelerating at an unprecedented pace in history. From online healthcare services, which have long been a focus of our attention, to the application of AI across the entire spectrum of diagnostic and therapeutic services, as well as in new drug development, these advancements collectively constitute a new revolution within the healthcare industry.