Dr. Bing Liu of Chinese Academy of Sciences: Two Decades of Pioneering Intracortical Brain-Computer Interfaces to Fill China's Visual Prosthesis Gap

Visual Reconstruction Technology Has Once Again Drawn Attention Recently.

Neuralink, Elon Musk’s brain-computer interface company, announced that its “Blindsight” technology, which utilizes implanted brain chips, has received the U.S. Food and Drug Administration (FDA) “Breakthrough Device” designation. Meanwhile, it has obtained approval in Canada to initiate the country’s first clinical trial and has begun recruiting participants.

However, visual reconstruction technology is not a novel concept. As early as the 1960s, a team at the University of Cambridge demonstrated the feasibility of achieving visual reconstruction through electrical stimulation by studying phosphenes. After decades of continuous refinement and iteration by countless scientists, visual reconstruction technology has finally become a reality.

In China, visual reconstruction technology is just beginning to emerge.Associate Researcher Liu Bing, Institute of Automation, Chinese Academy of Sciencesis one of the pioneers. Dr. Liu Bing has extensive research experience in neuroscience, brain-computer interfaces, and visual neuroscience. Since 2006, he has been closely following advancements in visual restoration technology, and he shared with us his insights on the current state and future prospects of this field.

Currently, there are two main directions for the development of visual reconstruction technology. One is throughRetinato perform optoelectronic conversion, utilizing electrical stimulation to substitute for the retina in the optoelectronic conversion process, thereby restoring vision; another approach is toPrimary Visual CortexElectrical stimulation is applied directly to the brain, enabling patients to perceive the external world.

After years of development, commercialized cases of successful visual restoration through retinal stimulation have emerged overseas. In 2013, the FDA-approved Argus II bionic eye enabled visually impaired patients to “regain their sight” by converting light entering the eye into electronic signals that are directly transmitted to the optic nerve.

This product was once extremely popular, and its manufacturer, Second Sight, was hailed as a global leader in the field of neuromodulation devices for blindness.

However, as research has deepened and the needs of visually impaired patients have become increasingly clear, numerous shortcomings of photoelectric retinal stimulation have been exposed. First, due to the limited area of the retina, itsThe number of implantable electrode arrays will also be limited., the electrode array may fail to match the density of photoreceptors on the retina, significantly compromising visual quality. Furthermore, current technology is unable to fully replicate the complexity of natural vision, including color perception and dynamic vision.

Furthermore, this technology is only effective for certain patients with ocular conditions, such as phthisis bulbi and enucleation.Patients with retinal absence cannot restore vision through such technologies.

In light of this, visual reconstruction technologies that directly act on cranial nerves have garnered significant attention. Neuralink’s “Blindsight” adopts this technological approach. Neuralink implants interface devices into the brains of visually impaired patients, subsequently converting external light signals into electrical signals and wirelessly transmitting them directly into the brain, thereby bypassing the anterior portion of the human visual system.Generating images by directly stimulating the visual cortex with a chip.This may represent another new avenue for addressing severe eye diseases, following the advent of artificial retinas.

It is evident that overseas companies have already paved a way from theory to product, and potentially to commercialization, in the exploration of visual restoration technology. According to the market size estimation logic submitted by an overseas brain-computer interface company for its IPO, based on an addressable population of 82,000 totally blind patients in the United States, the U.S. market for visual restoration technology could reach $4 billion. By extrapolation, according to 2020 statistics from the International Agency for the Prevention of Blindness (IAPB) (the latest available data as of 2020),The number of totally blind patients in China is approximately 8.69 million., ranking second globally, with the estimated market demand reaching RMB 890 billion.

Due to the relatively weak foundational infrastructure in China during the early stages,Facing a multi-billion-dollar blue ocean, there are currently no precedents for domestic visual reconstruction solutions to follow.In 2011, Dr. Liu Bing fromInstitute of Biophysics, Chinese Academy of Sciences; State Key Laboratory of Brain and CognitionAfter graduation, I went to the United States for further study. InDepartment of Neuroscience, University of Chicago; EECS, University of California, Berkeley; Neurobiology, Duke UniversityThe department and many other world-renowned brain science laboratories possess extensive research experience. In 2023, Dr. Liu Bing decided to return to China to continue his R&D efforts, leveraging his expertise and practical experience in brain-computer interfaces and visual reconstruction technology, thereby filling the domestic gap in visual reconstruction technology.

In layman's terms, the process of visual reconstruction mainly consists of three major steps: acquiring external information, encoding it into optoelectronic signals, and stimulating the visual nerves. Each step presents numerous challenges, such as the completeness of external data acquisition, the effectiveness of data encoding, and the precision of neural stimulation. After decades of effort, Dr. Liu’s team has now developed a comprehensive solution.

First, the primary objective of visual reconstruction technology is to provide users with more accurate and realistic feedback imagery. Current visual reconstruction techniques mainly rely on the spatial topological information of the visual cortex, without fully considering the mesoscale functional organization of the visual cortex and the encoding characteristics of neuronal populations. Therefore, there is an urgent need in visual reconstruction research for a conversion model that maps the spatiotemporal patterns of population neuronal stimulation to visual scene perception, as well as a conversion model that translates visual images into stimulation codes.

To address this issue, the team designed aVisual Acquisition Peripherals, and developed a stimulation pattern system aligned with visual neural drive based on in-depth research into neural mechanisms. Through this system, the team can simulate the transmission patterns of the brain’s visual nerves, enabling users to “see” images.

Furthermore, precise neural stimulation is also a crucial component. To identify the most appropriate stimulation sites for different patients, the team determines cortical locations via MRI prior to electrode implantation. After electrode implantation, electrical stimulation is applied to the cerebral cortex, and the patient’s perceptions of light spots, lines, and other visual phenomena are recorded. The content composed of these simple visual elements is referred to by the team asphosphene map(Brain Phosphene Map). While the principles underlying visual neural stimulation responses remain unclear, phosphene maps can reflect the correspondence between electrode stimulation and the visual phenomena perceived by patients. Using this “map,” researchers can personalize the stimulation patterns, enabling users to perceive more accurate visual images.

In the field of visual reconstruction technology, robust software capabilities and advanced hardware infrastructure are both indispensable. However, the clinical trial and regulatory approval processes for implantable electrodes in China are not yet mature, posing challenges to the application and widespread adoption of this technology.

To overcome these challenges and accelerate the translation of technological achievements,Dr. Liu’s team simulated the functionality of virtual electrodes by combining multiple physical patch electrodes.This design not only enhances the precision of visual reconstruction but also increases the system’s flexibility and scalability. In this way, the team can achieve more refined visual stimulation and reconstruction within the existing technological framework, providing patients with a higher-quality visual experience.

Dr. Liu stated, "In the future, our team will continue to actively explore alignment with international standards, aiming to contribute Chinese expertise to the development of visual reconstruction technology."

Currently, Dr. Liu’s team is developing the first prototype of a domestically produced visual reconstruction device, laying the groundwork for its subsequent commercialization and clinical deployment. In the next phase, the team will conduct further in-depth research to optimize stimulation patterns, improve electrode design, and refine signal processing algorithms, with the aim of providing users with more precise, rich, and even color-capable visual stimulation.

Furthermore, in the view of Dr. Liu BingVisual reconstruction technology with only open-loop stimulation and no closed-loop feedback is incomplete.In fact, the brain possesses strong learning capabilities and plasticity, enabling it to adjust its responses based on experience. Following the implantation of visual stimulation electrodes, the brain is highly likely to generate new patterns of visual perception. Therefore, only by obtaining feedback information and analyzing the potential variability in each patient’s response to stimulation can we ensure that the stimulation consistently aligns with the patient’s perception and needs.

Globally, no visual reconstruction product has yet achieved widespread adoption. Therefore, in the view of Dr. Liu Bing,It is still the golden age for the development of domestically produced visual reconstruction technology.Through continuous technological innovation and investment in research and development, Chinese research teams are striving to narrow the gap with international advanced standards. Dr. Liu Bing stated, “As our team’s research deepens and the technology matures, domestically developed visual reconstruction technology will gradually reach maturity, bringing significant benefits to a broad patient population.”