Xuhui AI-Powered Family Doctor Voice Assistant Project Files Prospectus

According to statistics from the National Health and Family Planning Commission, the number of patients with chronic diseases in China has exceeded 300 million, and the incidence rate continues to rise rapidly. Chronic diseases, represented by hypertension and diabetes, have severely impacted residents' physical health and quality of life, making the continuous and effective management of these conditions an urgent priority.

Under the leadership of the Xuhui District Health Commission and with the guidance of the Xuhui District Science and Technology Commission, the Xuhui District Health Development and Management Center has partnered with the Artificial Intelligence Research Institute of Shanghai Jiao Tong University and iFlytek, an AI technology enterprise, to explore pilot initiatives that leverage artificial intelligence voice interaction technologies to enhance health management applications for residents.

On August 28, 2019, the Xuhui District Health Development and Management Center held the launch ceremony for the “Smart Family Doctor, Healthy Households” – Xuhui Family Doctor Intelligent Voice Assistant Project, through collaborative efforts among the government, universities, and enterprises. Leaders in attendance included Qin Liping, Deputy District Mayor of Xuhui District, Shanghai, and Zhang Ansheng, Member of the Standing Committee of the Party Committee and Vice President of Shanghai Jiao Tong University.

Figure:Opening Remarks by Qin Liping, Deputy District Mayor of Xuhui District, Shanghai

Figure: Address by Zhang Ansheng, Member of the Standing Committee of the Party Committee and Vice President of Shanghai Jiao Tong University

At the ceremony, Sun Ya, Director of the Xuhui District Health Administration and Development Center, and Professor Yu Kai, Co-founder and Chief Scientist of AISpeech, introduced the project from the perspectives of the demand side and the technology provider, respectively. Leveraging key technologies such as robust speech recognition and structured electronic medical record (EMR) generation, the project explores a software-hardware collaborative multimodal interaction model. It integrates pre-consultation appointment scheduling, documentation of clinical encounters, post-consultation medical orders, and legacy EMR systems within healthcare institutions to create a patient-centric comprehensive health management platform. This platform enables voice-mediated access to health information and enhances the quality of medical services through voice-based interactions.

Subsequently, Qin Liping, Deputy District Mayor of Xuhui District; Zhang Ansheng, Member of the Standing Committee of the Party Committee and Vice President of Shanghai Jiao Tong University; He Aixing, Secretary of the Party Working Committee for Health Work of Xuhui District; Ren Xuelei, Deputy Director of the Xuhui District Health Commission; Zhang Lei, Deputy Director of the Xuhui District Health Commission; Sun Ya, Director of the Xuhui District Health Care Management and Development Center; Song Haitao, Deputy Dean of the Institute of Artificial Intelligence at Shanghai Jiao Tong University; Yu Kai, Co-founder and Chief Scientist of AISpeech; and Zhang Shouguo, Vice President of AISpeech, among other leaders, took the stage to jointly press the launch button, officially inaugurating the Xuhui Family Doctor Intelligent Voice Assistant Project.

Figure: Launch Ceremony of the Xuhui Smart Family Doctor Assistant Project

Following the launch ceremony, Zhang Shouguo, Vice President of AISpeech, presented the preliminary outcomes of the project development, providing an initial demonstration of various functionalities—including chronic disease follow-up, medication reminders, and health education—within home-based resident scenarios. The Xuhui Family Doctor Intelligent Voice Assistant Project is being developed in three progressive phases: “Health Data Collection,” “Health Guidance and Analysis,” and “Digital Family Doctor Preliminary Diagnosis.” Through continuous self-learning powered by artificial intelligence, the system will ultimately assume the role of an intelligent assistant for family doctors in Xuhui District.

During the second half of the event, Sun Ya, Director of the Xuhui District Health Administration and Development Center, and Zhang Shouguo, Vice President of AISpeech, held a signing ceremony for the Xuhui Family Doctor Intelligent Voice Assistant Project, formally executing the project cooperation agreement.

Figure:Signing Ceremony for the Xuhui Smart Family Doctor Assistant Project Collaboration

Subsequently, Zhang Lei, Deputy Director of the Xuhui District Health Commission; Zhu Qiong, Deputy Director of the Xuhui District Health Development and Management Center; Song Haitao, Deputy Dean of the Artificial Intelligence Institute at Shanghai Jiao Tong University; Yi Chuntao, Director of the Fenglin Community Health Service Center; Zhu Lan, recognized as one of China’s Most Beautiful Family Doctors; and Zhang Shouguo, Vice President of AISpeech, participated in a roundtable discussion. They exchanged views on topics such as “How Intelligent Voice Assistants for Family Doctors Empower Residents’ Health Management,” exploring the application of intelligent voice technology in settings including medical institutions and households.

Finally, Ren Xuelei, Deputy Director of the Xuhui District Health Commission (presiding over work), delivered the closing remarks. He stated that the Xuhui Family Doctor Intelligent Voice Assistant project has explored a novel model for residents’ self-managed health care, imbuing artificial intelligence technology with greater empathy and warmth, with the aim of providing higher-quality medical and health services to residents. Meanwhile, advancing “AI + Health” requires collaborative support from prestigious universities such as Shanghai Jiao Tong University and leading AI enterprises like AISpeech, to jointly promote the development of “Healthy Xuhui.”

Figure: Roundtable Discussion Session

The Xuhui Family Doctor Intelligent Voice Assistant Project represents a significant endeavor by Xuhui District to explore the development of “AI + Health” application scenarios. It is believed that, through a model featuring government leadership, academic collaboration, market-oriented operations, and co-construction and sharing, Xuhui can pioneer an innovative path for empowering resident health management with artificial intelligence.

The artificial intelligence technologies adopted by the project include:

Medical Content Speech Recognition

Different intelligent hardware is adopted for different scenarios: smart speakers serve as the hardware interface in home settings, while wearable devices are used in clinical diagnosis and treatment settings. Software-hardware collaborative voice processing is implemented based on the corresponding hardware. Since most patients with chronic diseases are elderly individuals whose spoken language often features heavy dialectal accents and unclear articulation, it is necessary to develop accent-adaptive speech recognition models, fault-tolerant spoken language understanding models, and transfer learning methods tailored to medical terminology, thereby maximizing the validity of voice input at the perceptual level.

Structured Information Extraction

After implementing speech-to-text transcription, post-processing of the text is required. First, effective information must be extracted from disorganized spoken language through slot filling. Second, casual spoken expressions should be smoothed and converted into standardized terminology, followed by structured generation based on existing medical templates. Corresponding knowledge graphs should be established for different departments and disease types, and metadata normalization should be applied to varying expressions across different medical institutions, thereby achieving a universal, patient-centric health record.

Conversational Interaction for Health Management

The medical knowledge base serves as the foundational resource for dialogue systems oriented toward health management, while dialogue management acts as the core scheduling module of such systems. Therefore, it is essential to investigate robust management strategies and evolvable dialogue systems. Research should focus on a unified dialogue management framework and fusion algorithms that integrate task-oriented dialogues with health knowledge question-answering dialogues, statistical and rule-based dual-driven cold-start methods for health management, and adaptive learning methods for dialogue interaction strategies aimed at ontology expansion.