SLAM inside the human body: camera tracking and 3D reconstruction for medical procedures

Overview

Minimally invasive surgery and endoscopic interventions rely heavily on the clinician’s ability to understand and navigate complex internal anatomy using only a narrow and often restrictive field of view. Having access to an accurate and dynamic 3D reconstruction of the endoscopic scene, together with reliable camera pose estimation can significantly improve spatial awareness and navigation during procedures. The generated map can be used alongside the device’s estimated location to help clinicians better orient themselves within the patient, and it also enables accurate measurement of anatomical structures from multiple viewpoints even without real-time localisation.

However, the complex and constrained nature of these scenes poses challenges for traditional 3D reconstruction techniques. Factors such as limited field of view, frequent occlusions, specular reflections, and dynamic tissue deformation make robust and accurate reconstruction particularly difficult. Recent advances in deep neural networks, and their application to 3D reconstruction from endoscopic video, have shown promising potential to address some of these challenges.

This project aims to integrate Simultaneous Localisation and Mapping (SLAM) with state-of-the-art deep learning algorithms to improve the accuracy, efficiency, and real-time performance of endoscopic 3D reconstruction, ultimately contributing to improved clinical outcomes and patient care.

The project offers an opportunity for international collaboration with researchers at the University of Oxford, UK, and providing access to world-leading expertise in robotics, computer vision, and medical imaging.

Skills and experience

• Strong background in programming (preferably Python).
• Machine learning experience.
• An appreciation of concepts in computer vision and deep learning.
• Familiarity with SLAM, monocular and multi-view 3D reconstruction, and scene representation methods (e.g. NeRF, Gaussian Splatting) is highly desirable.

Scholarships

You may be eligible to apply for a research scholarship.

Explore our research scholarships

Keywords

• computer vision
• deep learning
• medical imaging
• medical robotics
• 3D scene reconstruction

Contact

Contact the supervisor via email maryam.haghighat@qut.edu.au for more information.