ZeroUniqueMind Founder Prof. Lü Baoliang Pioneers Novel Affective Brain-Computer Interface Solutions for Depression Diagnosis and Therapy

According to the China Mental Health Survey published in The Lancet Psychiatry in 2021, about 95 million people in China suffer from depression, and 40% of the 280,000 people who commit suicide each year are affected by depression. This data is derived from the lifetime prevalence of depression.

With the population growth and rising pressures in recent years,CurrentlyThe actual number of patients is widely believed to have exceeded 100 million. However, in contrast to the vast population of patients with depression, the etiology and pathogenesis of depressive disorders are complex and may involve various factors such as physiological, psychological, and environmental aspects, which have not yet been fully elucidated.

At the same time, the World Mental Health Report released by the WHO in 2022 shows that the total number of psychiatrists in China is only 40,000, which means there are fewer than 3 psychiatrists per 100,000 people. The shortage of other mental health professionals, including psychiatric nurses and psychotherapists, is even greater. This leads to many patients with mood disorders not receiving timely and effective assessment and treatment, further exacerbating the disease burden of depression.

EmotionHelper, established on December 1, 2021, is committed to providing objective assessment, early diagnosis, and digital therapeutics for the large population of patients with depression through Affective Brain-Computer Interface. The company's main products under development currently include the Emotion "X" Ray Machine, Home Emotion Device, and the Weisi Affective Brain-Computer Interface Teaching and Experiment System.

Before discussing EmotionHelper's technical products, Professor Lü Baoliang first talked about the SEED (SJTU Emotion EEG Dataset), the world's largest and most diverse emotional EEG dataset that he and his team developed. Currently, the SEED dataset is one of the two most commonly used standard datasets in the field of affective brain-computer interface research globally.

Speaking of the original intention of starting a business, Professor Lü Baoliang said, "The laboratory I am in has been conducting research in the field of affective brain-computer interfaces since 2005, focusing on fatigue driving detection, sleep quality assessment, and emotion recognition. As the research deepened, we found that these research achievements are exactly the key technologies needed for evaluating depression."

In 2015, Professor Lü Baoliang's team officially released the SEED Emotion EEG Dataset, which was designed for EEG-based emotion classification tasks.

In 2016, China's '13th Five-Year Plan'纲要 included brain science and brain-like research in the 2030 Major Science and Technology Innovation Projects, with main contents covering research on brain disease diagnosis and treatment, the neural basis of brain cognitive functions, and brain-computer intelligence technology.Professor Lü Baoliang, as the leader of the brain-computer interface research direction at the Brain Science and Technology Research Center of Shanghai Jiao Tong University, has actively responded to the significant needs of the "China Brain Project" and officially started working with team members on the diagnosis and evaluation of depression.

As we all know, EEG signals are potentials generated by the spontaneous and rhythmic activity of brain neurons. Compared to non-physiological signals like human facial expressions, EEG signals are objective and not easily disguised.Therefore, the rhythmic changes in EEG signals can be correlated with different emotional states, serving as a representation of emotional state changes.

Professor Lü Baoliang also pointed out, "By analyzing experimental data, we found that EEG signals have a high recognition rate for positive emotions, while eye movement signals are better than EEG signals at recognizing negative emotions."Therefore, eye movement signals and EEG signals have excellent complementary characteristics. A multimodal emotion recognition model based on the fusion of these two signals will achieve higher recognition accuracy, providing a feasible multimodal affective brain-computer interface framework for the objective assessment of depression.

The SEED dataset, developed by Professor Lü Baoliang's laboratory, primarily identifies emotions in normal individuals by collecting EEG and eye movement signals. In addition to acquiring EEG signals using an EEG cap, SEED also gathers various eye movement information such as pupil diameter, gaze, saccades, and blinking through glasses-type and desktop eye trackers, enabling emotion recognition through multimodal signal fusion.

At the same time, emotion is a complex and multidimensional concept, a high-level generalization of a series of subjective cognitive experiences. The expression and experience of the same emotion may vary from person to person. Therefore, SEED also collected multimodal data from subjects of different cultural backgrounds and ethnicities to enhance the completeness of the dataset.

The SEED dataset includes six high-quality subsets: the three-category emotion dataset SEED, the four-category emotion dataset SEED-IV, the five-category emotion dataset SEED-V, the Chinese-French cross-cultural three-category emotion dataset SEED-FRA, the Chinese-German cross-cultural three-category emotion dataset SEED-GRE, and the vigilance dataset SEED-VIG.

The SEED dataset not only provides high-quality normal population data for EmotionHelper to develop objective depression assessment devices, the "X" emotion machine, but also offers a standardized multimodal dataset for global affective brain-computer interface research. According to reports, as of January 2024, the cumulative number of papers published using SEED has exceeded 1,440, with users from over 2,000 universities and research institutions across 88 countries worldwide.

In addition to the SEED dataset, Professor Bao-liang Lü has also extended various self-developed emotion interaction paradigms, emotion-inducing materials, multi-modal emotion brain-computer interface architectures, multi-modal deep learning algorithms, and large-scale EEG models to the objective assessment of depression in patient populations.

At the beginning of 2024, the research paper on the "EEG Large Model" by Professor Lü Baoliang's team and EmotionHelper stood out from 7,262 submitted papers and was selected as one of the top 5% Spotlight papers by ICLR 2024, one of the top conferences in the international machine learning field.

The general-purpose EEG large model LaBraM designed by Professor Lü Baoliang's team can effectively process various EEG data with different channels and lengths.By performing unsupervised training on a large amount of EEG data, it can quickly adapt to various downstream EEG tasks. The team has pre-trained three models with different parameter sizes, which are 5.8 million, 46 million, and 369 million respectively. This is the largest model in the field of brain-computer interface to date.

In 2020, Shanghai Jiao Tong University launched the "Jiao Da Zhi Xing" Plan "Medical-Engineering Interdisciplinary Research Fund" project, as one of the project membersProfessor Baoliang LüAfter interacting with clinical experts from Ruijin Hospital and the Shanghai Mental Health Center, I was deeply moved. "Before founding EmotionHelper, I always pursued scientific research based on personal interest. However, after engaging in medical-engineering crossover projects, I realized that if we want our research results to gain market recognition and be applied clinically to ultimately benefit nearly 100 million patients with depression, it must be achieved through entrepreneurship."

Therefore, based on the long-term research of multi-modal emotional brain-computer interfaces, EmotionHelper was officially established in 2021 to meet the significant needs of the country for early warning and early screening of depression.EmotionHelper also leverages high-quality clinical resources from hospitals such as the Shanghai Mental Health Center and Ruijin Hospital affiliated with the Shanghai Jiao Tong University School of Medicine to develop a novel objective assessment system for depression based on multimodal emotion brain-computer interface technology, aiming to achieve automated early warning and early screening for depression.

EmotionHelper currently has two main products, respectivelyEmotion "X" Machine and Family Emotion Device。

Analogous to traditional X-ray machines, the emotional "X-ray" machine is a medical device aimed at the objective diagnosis of mood disorders, assisting clinical psychiatrists in objectively evaluating depression.

Traditional depression diagnosis relies on various scales and doctors' experience, which requires significant manpower and time. This is one of the main reasons mental health institutions are overwhelmed with patients. However, through innovative multimodal emotion brain-computer interface technology, patients only need to undergo a 5-10 minute emotional interaction experiment within an emotional "X-ray" machine. The device can simultaneously collect signals such as EEG, eye movement, micro-expressions, and voice, enabling rapid and accurate quantification of depression symptoms, cognitive abilities, sleep quality, attention, and other indicators. This allows for automatic and objective evaluation of depressive states.

Emotion "X" Machine currently uses two key indicators, "low mood" and "anhedonia," in emotion interaction tasks to comprehensively collect multimodal physiological data from participants during the task process. This objective, quantifiable data enhances the objectivity and accuracy of depression diagnosis. Additionally, the device utilizes intelligent data processing, large-scale EEG models, and multimodal quantitative assessment reports to provide doctors and patients with more precise and efficient treatment solutions.

"EmotionHelper's emotion 'X'-ray machine, which has completed more than 2,700 trials on patients with depression and healthy individuals, has achieved over 80% accuracy and specificity in depression assessment. A multi-center clinical trial across China is expected to be launched in the first half of 2024. In the future, this medical-grade auxiliary diagnostic device for depression will apply for a Class III medical device registration certificate, becoming a 'standard feature' in psychiatric centers and general hospitals. It will reduce the burden on hospitals and psychiatrists, help identify groups prone to depression earlier, and enable timely intervention and treatment," said Professor Lü Baoliang.

Depression, as a common mental disorder, is characterized by its persistent presence, significantly impacting daily life and leading to long-term health consequences, with the family being one of its crucial environmental contexts.

The second-generation TOC family emotion meter from EmotionHelper has three major application scenarios: First, it functions like an "emotional blood pressure monitor" to help track the emotional states of patients with mental illnesses or chronic diseases accompanied by depression. Second, it provides emotional support for women during the perinatal period and monitors the emotional atmosphere within families. Third, it offers emotional supervision for teenagers and children along with monitoring the family’s emotional environment. It delivers targeted emotional management advice for different groups, assisting them in improving their emotional well-being. Additionally, this mobile family product supports cross-platform multi-device interconnectivity, allowing doctors to access users’ emotional data in real-time through cloud-based backend services.

In addition to these two products, EmotionHelper is also developing an educational and research tool that supports scientific research experiments and innovative science popularization — the Weisi Emotion Brain-Computer Interface Teaching and Experiment System, which provides high-quality experimental teaching support for university laboratories, corporate developers, and students in basic education.

In addition to being the founder and chief scientist of EmotionHelper, Professor Bao-Liang Lu is also a tenured professor in the Department of Computer Science and Engineering at Shanghai Jiao Tong University, an IEEE Fellow, the director of the Shanghai Key Laboratory of Intelligent Interaction and Cognitive Engineering at Shanghai Jiao Tong University, the executive director of the Qingyuan Research Institute at Shanghai Jiao Tong University, the co-director of the Brain-Computer Interface and Neuromodulation Center at Ruijin Hospital Affiliated to Shanghai Jiao Tong University, and the director of the Brain Disease Center-Mihoyo Joint Laboratory at Ruijin Hospital Affiliated to Shanghai Jiao Tong University. He has achieved abundant research results in the fields of artificial intelligence and brain-computer interfaces.

Professor Baoliang Lu serves as an editorial board member for IEEE Transactions on Affective Computing, Journal of Neural Engineering, IEEE Transactions on Cognitive and Developmental Systems, Pattern Recognition and Artificial Intelligence, and Journal of Intelligent Science and Technology. He has also received the 2018 IEEE Transactions on Autonomous Mental Development Best Paper Award, the 2020 Wu Wenjun Artificial Intelligence Natural Science First Prize, the 2021 IEEE Transactions on Affective Computing Best Paper Award, the ACM Multimedia 2022 Top Paper Award, and the 2022 Asia-Pacific Neural Network Society Outstanding Achievement Award.

Behind the wave of scientists starting businesses lies the pressing need of the industry and market for key technologies. In 2021, EmotionHelper was co-founded by Professor Bao-Liang Lu and miHoYo, with miHoYo also being the angel investor of EmotionHelper. In addition to supporting the transformation of scientific research results through capital, miHoYo has also collaborated with the Brain Disease Center of Ruijin Hospital, affiliated with Shanghai Jiao Tong University, to jointly establish the "Ruijin Brain Disease Center miHoYo Joint Laboratory," where Professor Bao-Liang Lu serves as the director.

At the end of the interview, Professor Baoliang Lü stated, "Compared to the identity of a university professor, the role of a founder of a brain-computer interface company allows me to better utilize my years of experience in interdisciplinary research and grassroots administrative management. It also enables me to truly implement over a decade of accumulated scientific research achievements, transforming them into new diagnostic and treatment methods that can help patients with depression."

Speaking about the future, Professor Lü Baoliang continued, "Currently, EmotionHelper is in the midst of a new round of financing. We hope that, on the foundation of successfully transforming scientific research from 0 to 1, we can leverage social capital to accelerate the clinical application of our first scientific achievement and bring it to patients. At the same time, we are continuously innovating and refining in the field of Emotion AI, aiming to create intelligent systems and emotional service robots suitable for all kinds of populations."