Stereoscopic Augmented Reality Visualization for Laparoscopic Surgery

Image banner for stereoscopic augmented visualization project

Key Personnel

  • Raj Shekhar, PhD 
  • Timothy Kane, MD 
  • Craig Peters, MD 
  • Xinyang Liu, PhD 
  • Sukryool Kang, PhD 
  • James McConnaughey

The overall goal of this project is to develop and commercialize a technology that gives minimally invasive surgeons an enhanced view of the surgical anatomy for improved safety, precision, and efficiency. The two new visual cues we introduce are 1.) perception of true depth and improved understanding of 3D spatial relationship among anatomical structures, and 2.) visualization of critical internal structures along with a more comprehensive visualization of the operative field. This is accomplished by integrating two real-time surgical imaging modalities: newly emerged 3D laparoscopic camera technology that allows visualizing the surgical anatomy with the highest image quality currently available and perception of true depth, and a laparoscopic ultrasound capable of visualizing hidden structures. We call the resulting visualization capability stereoscopic augmented reality, in which stereoscopic laparoscopic video (the reality) is augmented with ultrasound findings, especially the blood vessels, ducts, and tumors. For accurate spatial registration between the two types of images, the 3D location and orientation of the imaging devices are continuously tracked. A fully functioning prototype that has been tested in the laboratory, and through animal studies, is currently being tested in humans under an IRB-approved protocol. The human testing is generating critical data for technology improvement and product design and assessment of clinical benefits.