We care about your privacy. Read about your rights and how we protect your data. Get Details

Augmented Reality Hybrid Endoscopic Visualization System with Refined Tracking and Calibration


Researchers at Children's National Hospital have created an augmented reality hybrid visualization system that integrates real-time endoscopic video with live ultrasound image with superior electromagnetic and optical image tracking, refinement and calibration for flat or angled endoscopes.


Minimally invasive surgeries allow for reduced operative exposure and time, reduced patient discomfort and lower post-operative patient morbidity. Currently, surgeons use endoscopic cameras to view the region being operated for such surgeries. However, they have to separately view the ultrasound image that highlights nearby tissues not visible through the endoscopic camera. To overcome this problem, Children's National inventors developed an augmented reality visualization system that integrates both the real-time endoscopic video and the live ultrasound image so surgeons may simultaneously view not only the surgical region but also adjacent unseen blood vessels and tissues. This integrated visualization system offers the following advantages:

  1. Applicable to any type of endoscopic device including 2-D and 3-D endoscopic cameras, flat and angled endoscopes, and imaging platform systems currently used in hospitals.
  2. Accurate, clinical quality image tracking, refinement by limiting effects of occlusion.
  3. Real-time camera calibration for angled or oblique-viewing rigid endoscopes.


  • Real-time augmented reality visualization of surgical images using endoscopes

Stage of Development

  • Ongoing multi-institutional human clinical trials. Studies with animal models completed.

Intellectual Property Status

  • 4 pending applications including 1 U.S. and 1 European application for the Device, Method for Generating Composite Images for Endoscopic Surgery of Moving and Deformable Anatomy, and 2 International/PCT Patent Applications for System, Apparatus, & Method for Calibrating Oblique-viewing Rigid Endoscope, & Hybrid Hardware & Computer Vision-Based Tracking System and Method

This technology is available for exclusive or non-exclusive licensing.

Licensing Contact