Beyond Human Perception: AI-Powered Objective Movement Assessment for Parkinson's and Neurological Disorders

“This is a movie clip. Can anyone spot an issue with the actor’s gait?” Dr. Fan Wei, Dean of Tencent Medical AI Research Institute, opened his speech with this question at the Smart Healthcare Special Session of the Tencent Global Digital Ecosystem Summit held in Kunming on May 22.

“In fact, the frequency and amplitude of his left arm’s swing were smaller than those of a normal person, and he was later diagnosed with Parkinson’s disease,” Fan Wei further explained. The actor in question is Michael J. Fox, who starred in the hit sci-fi comedy Back to the Future and was just 30 years old when he was diagnosed with Parkinson’s disease.

Figure: Dr. Fan Wei, Dean of Tencent Medical AI Research Institute, Delivers Keynote Speech

China is home to over 4 million patients with Parkinson’s disease, accounting for approximately 50% of the global total. In Fan Wei’s view, the impact of Parkinson’s disease on society and families is immeasurable. “A patient with Parkinson’s disease incurs annual expenses of nearly RMB 30,000, which constitutes up to 48% of some households’ total income.” Nevertheless, even more unbearable than the financial burden is the mobility impairment and loss of self-care ability caused by the disease. “Parkinson’s disease is classified into four stages: patients at stages 1 and 2 can care for themselves, while those at stages 3 and 4 cannot, struggling even with basic activities such as eating and using the restroom. One caregiver at home is often insufficient; some families require two.” Consequently, in households with Parkinson’s patients, the disease may have already plunged the entire family into a state of “paralysis” before the patient themselves becomes physically paralyzed.

In fact, early detection, diagnosis, and treatment, combined with scientific and effective disease management, can effectively delay and control the progression of Parkinson’s disease. However, this is no easy task. As Fan Wei noted, “In China, the prevalence of Parkinson’s disease among individuals aged 75 and older is as high as 10%, yet only 3.75% seek medical attention during the initial stages of onset.” The early diagnosis and comprehensive management of Parkinson’s disease face at least “two major hurdles”:

First, the long-term imbalance in the “seesaw” ratio of doctors to patients. In China, there is a large number of Parkinson’s disease patients, but only slightly more than 1,000 specialists dedicated to Parkinson’s disease. As population aging becomes increasingly severe, this disparity between the number of patients and available physicians will continue to widen.

Second, traditional Parkinson’s disease assessment methods are not only time-consuming and labor-intensive but also highly subjective. Unlike chronic conditions such as diabetes and hypertension, which can be quantitatively measured using medical devices, the primary assessment approach for Parkinson’s disease is relatively rudimentary. Physicians rely on manual observation, scoring patients based on the fluency with which they perform specified movements and conducting a comprehensive evaluation of their condition. “Currently, a single clinical assessment takes 30–40 minutes, during which specialists visually observe and score different motor tasks. Due to variability in human visual judgment, different experts may arrive at different assessment results for the same patient.”

To overcome these two major hurdles in Parkinson’s disease management, Dr. Fan Wei and his team aim to leverage AI technology to transform motor function assessment into a quantifiable, objective standard, thereby enabling more effective patient management by physicians through productized solutions. Furthermore, beyond Parkinson’s disease, many other conditions are closely linked to motor function assessment, including cerebral palsy, scoliosis, stroke, and multiple sclerosis. The lack of tools and methods for objective measurement currently represents a blind spot in medical practice. In response, Dr. Fan Wei proposes developing a set of objective, quantifiable motor assessment methods applicable to multiple diseases, utilizing AI capabilities to address this gap in medical care.

Figure: Analyzing gait in videos using AI technology

“Measure everything that can be measured, and make the unmeasurable measurable.” By quoting this famous saying by Galileo, Fan Wei shared his inspiration for exploring motor function: “Some things cannot be measured directly, so you need to find ways to measure them. Accurately measuring what is measurable is a fundamental principle of science.” Just as thermometers, blood pressure monitors, and glucometers have become everyday monitoring devices, the integration of artificial intelligence (AI) technology has made Fan Wei’s “beyond-cognition” vision possible. Dr. Fan Wei explained that in terms of vision, the human eye can recognize approximately 24 frames per second, whereas AI applications operate at 60–240 frames per second; in terms of sound sampling, the human ear hears up to 20 kHz, while AI applications require rates greater than 44 kHz.

Figure: AI Helps Measure Beyond Cognitive Limits

Dr. Fan Wei’s team at the Tencent Medical AI Research Institute collaborated with Professor Wang Jian’s team from the Department of Neurology at Huashan Hospital to perform MDS-UPDRS scoring by analyzing videos of patients performing prescribed movements, leveraging video analysis technology. Since motor function assessment in patients is inherently time-consuming, data collection has been inefficient. To address the scarcity of training data, Dr. Fan Wei’s team innovatively introduced AR-based synthetic data generation, enabling a thousands-fold expansion of the training dataset within a short period. Video data annotation is even more labor-intensive. To tackle this challenge, the technical team incorporated virtual sensing technology, which embeds annotation points directly into the generated synthetic data. These annotated synthetic data were then fused onto real-world videos of patients’ limb movements to achieve synchronized motion. This approach broke through the limitations of manual human annotation, significantly improving work efficiency and reducing project R&D costs.

In traditional motor function assessment methods, the minimum resolution of tremor detection by the human eye is at the centimeter level. However, with the aforementioned video analysis technology, the system can now identify pixel-level tremors that are imperceptible to the human eye and perform quantitative analysis, thereby breaking through the limits of micro-motion measurement. This achievement far exceeds the limits of human visual perception.

Figure: AI Video Analysis for the Recognition and Measurement of Microtremors

In addition to AI video analysis technology, Dr. Fan Wei also unveiled for the first time at the conference the application of two new technologies from Tencent Medical AI Laboratory in the field of Parkinson’s disease assessment: mobile sensor-based motor assessment and speech function analysis. By leveraging sensors in ordinary smartphones, these technologies enhance the dimensionality and precision of motor function evaluation for Parkinson’s disease. Currently, the motor function assessment for Parkinson’s disease supports not only hand movements such as resting tremor and kinetic tremor, but also leg movements including foot tapping and toe lifting, enabling the sensors to efficiently measure subtle whole-body motions.

Beyond visual breakthroughs, speech analysis technology is poised to transcend the limits of human hearing. A common symptom of Parkinson’s disease is monotone speech, accompanied by rapid breathing or speech latency. To capture and learn these features, the system guides users to record a 5–10 second voice sample, which is then analyzed by an artificial intelligence model to assess the patient’s condition. Furthermore, as no additional hardware is required—only a smartphone is needed for rapid, simple, and user-friendly voice-based assessment of Parkinson’s disease status—daily home-based and caregiving monitoring becomes feasible.

By leveraging three core assessment technologies—video analysis, mobile sensor analysis, and speech analysis—physicians can more efficiently evaluate the condition of patients with Parkinson’s disease, stratify disease severity, and formulate more targeted treatment plans, thereby providing convenience for patients with limited mobility. Through mobile phone-based interactions, physicians can manage patients’ conditions, ensuring the administration of the most appropriate medications at optimal dosages, preserving motor function, and alleviating the burden on families.

In addition, similar motion video analysis techniques can be applied to other scenarios involving motor function assessment, such as preoperative gait analysis for patients with cerebral palsy, motor function evaluation during the rehabilitation phase for stroke patients, and assessment of motor functional status in soccer players during post-injury recovery training.

Figure: The Parkinson’s Disease Motor Function Assessment System Has Initiated International Clinical Trials

According to Dr. Fan, the Parkinson’s disease motor function assessment system has already initiated international collaborations. For instance, it has obtained clinical trial approval in the UK through its partner Medopad and is gradually commencing clinical trials. Furthermore, in a collaborative clinical study with a renowned international hospital, comparative experiments conducted with physician teams demonstrated that the system achieved a 91% consistency rate in scoring multiple motor functions, surpassing the inter-rater consistency observed among general physician teams.

As of May 2019, this motor assessment system for Parkinson’s disease had undergone pilot trials at 15 clinical institutions in cities including Nanjing, Tianjin, Xi’an, Kunming, and Guiyang, bringing significant benefits to patients with Parkinson’s disease. Among the award-winning cases announced in the Smart Healthcare session of the Tencent Global Digital Ecosystem Summit, “Intelligent Assessment of Motor Function in Parkinson’s Disease” was successfully selected as one of the “Top Ten Innovations in Technology-Enabled Healthcare” in the 2018 China “Internet + Medical Health” Excellent Cases.

Tencent Medical AI Research Institute has also conducted extensive research and exploration in disease areas such as multiple sclerosis, psoriasis, assisted diagnosis and surgical planning for cardiovascular diseases, cancer radiotherapy planning, and EKG analysis.

Taking psoriasis care as an example, Tencent led the initiation of a research project under the “Special Project on R&D of Digital Diagnostic and Therapeutic Equipment” within China’s National Key R&D Program last year. The Tencent Medical AI Laboratory, in collaboration with Professor Chen Xiang’s team at Xiangya Hospital, jointly developed an AI-powered psoriasis diagnostic and therapeutic assistant based on the WeChat Mini Program platform. This tool integrates the entire workflow of AI-assisted dermatological diagnosis and treatment, enabling patients to perform initial self-screening through photos of skin lesions and online Q&A. During clinical consultations, it quantifies disease status by scoring severity, thereby assisting physicians in continuously and quantitatively monitoring disease progression, tracking prognostic outcomes, and timely adjusting treatment plans. Currently, the accuracy of initial psoriasis screening has reached 93%.

Furthermore, Dr. Fan Wei introduced that the team has also leveraged AI technology to achieve automatic delineation of organs at risk in head and neck radiotherapy, completing organ segmentation within 0.12 seconds. Previously, it typically took physicians seven days to develop a radiotherapy planning scheme for a single patient. This technology significantly enhances the work efficiency of radiation oncologists, benefiting both doctors and patients.