Shanghai Jiao Tong University to Transfer AR-Guided Microscope System for Cataract Surgery at RMB 1.67 Million

Recently, the Advanced Industrial Technology Research Institute of Shanghai Jiao Tong University released an announcement on the transformation of scientific and technological achievements, proposing to“Microscope-Augmented Reality Guidance System and Method for Ophthalmic Cataract Surgery”Patent assignment to industry partners. The patent was jointly developed by Zheng Ce, Chen Xiaojun, Tu Puxun, Zhao Peiquan, and Huang Danqing. The rights holders are Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine and Shanghai Jiao Tong University (each holding a 50% ownership interest), with the proposed transaction amount reaching1.67 million yuan。

As an innovative technological achievement focused on ophthalmic cataract surgery, this patent establishes an augmented reality guidance system that integrates a surgical microscope, image acquisition card, workstation, and semi-transparent display. Leveraging deep learning algorithms, the system enables precise semantic segmentation of the surgical field and calculation of guidance parameters, directly overlaying augmented reality visual cues onto the microscope’s optical path. This design effectively addresses key pain points of traditional guidance systems, such as poor hand-eye coordination, insufficient real-time performance, and low guidance efficacy, thereby significantly enhancing surgical precision and safety. It holds substantial significance for advancing the intelligent and minimally invasive evolution of cataract surgery.

Cataracts are the leading cause of blindness worldwide,Phacoemulsification Cataract SurgeryIt is a standard clinical treatment protocol. The procedure must be performed under an operating microscope, which not only provides a narrow field of view but also requires the surgeon to overcome physiological hand tremors, thereby placing extremely high demands on professional skills and clinical experience.

To enhance intraoperative performance, existing research has focused on cataract surgery guidance systems. The prevailing technical approach involves acquiring and processing microscope video feeds to extract information on surgical steps or anatomical structure boundaries, which is then transmitted to an external display for guidance.However, from a technical perspective, this solution has obvious shortcomings:

First, augmented reality information must be displayed on an external monitor, while ophthalmic surgeons need to focus on the microscope’s field of view during surgery; this separated display disrupts intraoperative hand-eye coordination and increases the risk of complications.

Secondly, video processing mostly adopts traditional iterative optimization schemes, which are difficult to meet the real-time guidance needs and are prone to stuttering and delay.

Moreover, extracting only low-level information such as surgical steps and anatomical boundaries fails to align with the high-level guidance required by physicians, resulting in a significant reduction in the effectiveness of the guidance.

For existing systems“Hand-eye coordination imbalance, poor real-time performance, and low guidance effectiveness”Addressing these core pain points, the patent achieves breakthrough innovations in display architecture, image processing, information extraction, and interaction design, creating a precise, efficient augmented reality guidance solution that closely aligns with clinical needs.

1. Display Architecture Innovation: Optical Path-Integrated AR Solves the Hand-Eye Coordination Challenge

Breaking through the traditional technical path of “external display guidance,” innovativelyIntegrate a semi-transparent display into the optical path of the surgical microscope,Achieves direct overlay of augmented reality guidance information onto the original surgical field. Surgeons can simultaneously observe the surgical field and guidance information through the microscope eyepieces, eliminating the need to shift their gaze between the eyepieces and an external display. This design perfectly aligns with the operational habits of ophthalmic surgeons, fundamentally addressing the drawbacks of split-screen displays—such as disrupted hand-eye coordination and increased risk of complications—thereby enhancing surgical continuity and safety.

2. Image Processing Upgrade: Deep Learning Empowerment Achieves Breakthrough in Real-Time Performance

Abandoning traditional iterative optimization-based image processing schemes, pioneering the adoption ofSemantic segmentation of surgical video images using a U-Net-based convolutional neural network,Precisely extract information from the iris and surgical instrument regions. Compared with traditional methods, deep learning algorithms significantly improve processing efficiency, achieving a frame rate exceeding 30 frames per second. This fully matches the acquisition frequency of image capture cards, enabling smooth, lag-free, and low-latency dynamic intraoperative guidance. It breaks through the real-time performance bottlenecks of existing solutions, ensuring synchronization between guidance information and surgical maneuvers.

3. Information Extraction Optimization: Precise Transformation from Underlying Data to Upper-Level Parameters

Innovative Design"Semantic Segmentation — Post-Processing Optimization — Guidance Parameter Calculation"Full-process information extraction logic overcomes the limitations of existing solutions that only extract low-level information. The iris segmentation results are optimized through post-processing steps such as maximum connected component extraction, sparse contour sampling, and curvature filtering (threshold 0.65). Ellipse parameters are then fitted using the least squares method to precisely calculate the positional parameters of the main incision (85°–105° arc segment) and secondary incision (170°–180° arc segment), as well as the capsulorhexis range parameters (with the radius defined as half the sum of the ellipse’s major and minor axes). Simultaneously, centerline parameters of surgical instruments are extracted to achieve precise alignment between guidance information and surgeons’ operational needs, significantly enhancing the effectiveness of surgical guidance.

4. Interactive Design Innovation: Dynamic Step Adaptation to Enhance Clinical Utility

DesignDynamic Control Mechanism for Augmented Reality Information Based on Foot Pedals, enabling on-demand activation and deactivation of guidance information based on the different steps of cataract surgery (guidance is required for core steps such as incision, capsulorhexis, and intraocular lens implantation, while it is unnecessary for other steps). This design allows surgeons to switch guidance modes without interrupting the procedure, ensuring precise guidance during critical steps while avoiding visual interference from guidance information during non-critical steps, thereby further enhancing the clinical adaptability of the technical solution.

5. Extended Value: Multi-dimensional Expansion Aligning with the Trend of Precision Medicine

The core patented technology features excellent scalability, enabling multi-dimensional expansion based on the existing architecture: First, expansion of surgical steps, by optimizing guidance parameters and prompt designs for other procedures such as phacoemulsification and cortical aspiration; second, scenario expansion, by adapting the core guidance technology to other ophthalmic contexts such as glaucoma and retinal surgeries, thereby building a universal AR-guided platform for ophthalmic surgery; third, intelligent extension, by integrating AI-assisted decision-making technologies to achieve dynamic optimization of personalized guidance parameters based on patients’ ocular characteristics and surgeons’ operational habits, aligning with the global trend toward precision medicine in ophthalmology.

In the field of ophthalmic surgical guidance, numerous patents have focused on enhancing surgical precision; however, these differ significantly from the present patent in terms of technical approach, core functionalities, and application scenarios:

Leica Microsystems (Switzerland)“Surgical Microscope System for Ophthalmology and Its Detection Unit” focuses on illumination feedback and scattering detection within the surgical field of view. It does not involve AR guidance or deep learning technologies, provides only basic optical data, and is technically positioned toward optimizing microscope performance.

Jiaxing Zhitong Technology“A Method for Ophthalmic Surgical Navigation” covers iris segmentation and the calculation of incision and capsulorhexis parameters; however, it does not employ augmented reality (AR) technology to visualize guidance information, nor does it achieve integration with the microscope’s optical path. Consequently, the issue of dissociation between guidance information and the surgical field of view persists.

Shandong Cancer Hospital and InstituteThe “Endoscopic Surgical Navigation Method Based on Augmented Reality and Deep Learning” focuses on the fusion of preoperative 3D models with intraoperative real-time images, tailored for endoscopic surgical scenarios, but lacks customized optimization for the specific anatomical structures and procedural steps of ophthalmic cataract surgery.

Overall, the field is rapidly evolving toward intelligence, precision, and scenario-specific adaptation. The deep integration of augmented reality with surgical equipment, the practical implementation of AI technologies, and the synergistic use of multiple technologies to address clinical pain points have become core trends, driving the continuous development of cataract surgery toward minimally invasive, standardized, and universally accessible practices.