Endo-4DGX: Robust Endoscopic Scene Reconstruction and Illumination Correction with Gaussian Splatting.


MICCAI 2025


Yiming Huang1,2*, Long Bai1,2*, Beilei Cui1,2*, Yanheng Li3, Tong Chen4,
Jie Wang2, Jinlin Wu5, Zhen Lei5, Hongbin Liu5, Hongliang Ren1,2 †

1The Chinese University of Hong Kong, Hong Kong SAR, China,
2Shenzhen Research Institute, CUHK, Shenzhen, China,
3City University of Hong Kong, Hong Kong, China,
4The University of Sydney, Sydney, NSW, Australia,
5Centre for Artificial Intelligence and Robotics, HKISI-CAS, Hong Kong, China,
* Equal Contribution, † Corresponding Authors
Paper arXiv Code



Endo-4DGX achieves state-of-the-art performance for robust reconstruction and illumination correction in the surgical scene with varying illumination.

Abstract

Accurate reconstruction of soft tissue is crucial for advancing automation in image-guided robotic surgery. The recent 3D Gaussian Splatting (3DGS) techniques and their variants, 4DGS, achieve high-quality renderings of dynamic surgical scenes in real-time. However, 3D-GS-based methods still struggle in scenarios with varying illumination, such as low light and over-exposure. Training 3D-GS in such extreme light conditions leads to severe optimization problems and devastating rendering quality. To address these challenges, we present Endo-4DGX, a novel reconstruction method with illumination-adaptive Gaussian Splatting designed specifically for endoscopic scenes with uneven lighting. By incorporating illumination embeddings, our method effectively models view-dependent brightness variations. We introduce a region-aware enhancement module to model the sub-area lightness at the Gaussian level and a spatial-aware adjustment module to learn the view-consistent brightness adjustment. With the illumination adaptive design, Endo-4DGX achieves superior rendering performance under both low-light and over-exposure conditions while maintaining geometric accuracy. Additionally, we employ an exposure control loss to restore the appearance from adverse exposure to the normal level for illumination-adaptive optimization. Experimental results demonstrate that Endo-4DGX significantly outperforms combinations of state-of-the-art reconstruction and restoration methods in challenging lighting environments, underscoring its potential to advance robot-assisted surgical applications.

Architecture

Illustration of the proposed Endo-4DGX. We first preprocess the inputs for initialization, and then we train the Gaussians with our illumination embedding, region- aware enhancement, and spatial-aware adjustment for sub-area and global restoration.

Visualization

Illumination Correction & Novel View Synthesis


Qualitative Result on EndoNeRF-EC Dataset. Our method provides the best reconstruction and illumination correction results for challenging illumination.

Illumination Correction results.


Uneven Illumination Reconstruction Results

BibTeX


      @misc{huang2025endo4dgxrobustendoscopicscene,
      title={Endo-4DGX: Robust Endoscopic Scene Reconstruction and Illumination Correction with Gaussian Splatting}, 
      author={Yiming Huang and Long Bai and Beilei Cui and Yanheng Li and Tong Chen and Jie Wang and Jinlin Wu and Zhen Lei and Hongbin Liu and Hongliang Ren},
      year={2025},
      eprint={2506.23308},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.23308}, 
      publisher={arXiv},
      }