Marrying Multi-view Geometry with Deep Priors for Image-based 3D Reconstruction

Abstract:

We live in a world where all interactions with the environment necessitate a 3D understanding of our surroundings. While humans excel at reasoning about 3D structures from both multi-view and single-view images, replicating this capability in computers remains challenging due to the need to combine mathematically proven geometric knowledge with end-to-end learned priors.

In this talk, the speaker introduces his work on narrowing this technical gap by devising new geometry-informed neural networks. These networks enhance the synergy of geometric principles and learned priors by: (1) combining single-view 3D reasoning with multi-view supervision; (2) embedding geometric aggregation into deep networks for multi-view stereo; and (3) fusing single and multi-view geometric cues for robust dynamic scene reconstruction. Finally, the speaker discusses new trends in geometry-informed networks that promise to advance this field.

Short bio:

Rui Li is a Ph.D. student at Northwestern Polytechnical University and a Visiting Researcher at King Abdullah University of Science and Technology (KAUST). He was a Visiting Student at the Computer Vision Lab of ETH Zürich under the supervision of Dr. Federico Tombari and Prof. Luc Van Gool. His research interests include 3D reconstruction, depth estimation, and novel view synthesis. He has published eight first-authored papers, including three at CVPR and others in proceedings such as ACM Multimedia, ICIP, Pattern Recognition, and IEEE Transactions on Multimedia. He led a team that won first and third places in the TRICKY Depth Estimation Challenge at ECCV 2024. He serves as a reviewer for international conferences and journals, including CVPR (2023, 2024), ICCV (2023), ECCV (2022, 2024), AAAI (2025), and 3DV (2023, 2025). He has broad collaborations in the field of 3D vision with researchers from ETH, Google, UNSW, and others.