CVC Seminar

Abstract:

A calibrated machine learning model produces probabilistic predictions that are well-aligned with real probabilities: it tends to be more certain when it is correct. Unfortunately, the unique characteristics of modern neural networks, e.g. over-parametrization or iterative training dynamics, can often result in overfitting the training data and generating over-confident predictions. The goal of this first part of the seminar is to introduce machine learning practitioners to the main ideas and methods behind modern model calibration: its formal definition, how we can measure it, and how we can improve it. This talk was part of a tutorial delivered at MICCAI 2023 on uncertainty quantification for medical image analysis, and we will also have a short hands-on code session: we will train a skin lesion classifier in pytorch, measure its calibration, and learn a post-processing transform to improve it. Finally, we will go over a couple of recently published works on model calibration (again, MICCAI 2023) that make use of the concepts introduced in the first part of the seminar.

Short bio:

Adrian Galdran is a Marie Skłodowska-Curie Research Fellow at Universitat Pompeu Fabra in Barcelona jointly with the Australian Institute of Machine Learning in Adelaide, leading a project on Uncertainty Quantification for medical imaging. Prior to this, he worked as lecturer/post-doctoral researcher in biomedical image analysis at Bournemouth University in the UK, ETS Montréal in Canada and INESC-TEC Porto in Portugal.

• 

•