Efficient Fine-Grained Segmentation and Histological Classification of Colorectal polyps in White-Light Colonoscopy

Yael Tudela will defended his PhD thesis on November 14, 2025.

What is the thesis about?

Colorectal cancer (CRC) prevention is heavily reliant on colonoscopy for the detection and removal of precancerous polyps. However, challenges persist in minimizing polyp miss-rates and in accurately characterizing polyp histology (e.g., adenoma, hyperplastic, sessile serrated lesion - SSL) in real-time using standard white-light imaging (WLI), impacting clinical decisions. Current AI systems are often found to fall short due to reliance on advanced imaging modalities, limited classification granularity, or inadequate validation.

The aim of this thesis is the development of novel deep learning systems for efficient, fine-grained segmentation and histological classification of colorectal lesions in real-time, using exclusively WLI. Key objectives included the creation of robust multi-class models, the ensurance of WLI compatibility for generalizability, and the achievement of real-time performance. A core contribution is the creation of a new dataset: PolypSegm-ASH, a public benchmark of WLI images with pixel-wise segmentation masks and pathologist-confirmed, fine-grained histological labels (Adenoma, SSL, Hyperplastic). A critical need for training and evaluating AI is addressed by this resource.

Building upon this, Swin-Expand, an efficient Swin Transformer-based architecture for simultaneous multi-class polyp segmentation and classification, is introduced. Superior fine-grained segmentation and classification on PolypSegm-ASH is demonstrated, highlighting that overall polyp characterization is improved by training with richer histological detail.

To further enhance classification by incorporating clinical context, VITAL (Visual Information and Tabular Adaptive Learning), a plug-and-play module, is presented. In VITAL, structured lesion descriptors (e.g., Paris classification, size, location) are integrated with visual features from segmentation models using gated FiLM blocks, significantly boosting classification accuracy across architectures and showing robustness to missing metadata.

In conclusion, AI in colonoscopy is advanced by this research through provision of a crucial dataset and development of novel deep learning models (Swin-Expand and VITAL), enabling more precise, clinically relevant polyp analysis using standard WLI. The way for more comprehensive and reliable AI-assisted diagnostic support is paved by these contributions, aiming to improve CRC screening efficacy.

Keywords

Colorectal cancer, computer-vision, histology classification, medical imaging, semantic segmentation