Meet PICASSO: personalised bronchoscopies for early detection of lung diseases

CVC researchers have developed a new model for the segmentation of distal airways in lung cancer biopsies named PICASSO and presented at PLOS, paving the way towards more accurate, improved and personalized bronchoscopies.

The World Health Organisation estimates that more than 1.76 million people perish due to lung cancer per year, being the most common cause of cancer and, thus of cancer death. Dr. Debora Gil, head of the medical imaging team at the Computer Vision Center and Computer Science Professor at the Autonomous University of Barcelona, has led novel research on the segmentation of airways in computed tomography for the accurate support of diagnosis and intervention of a set of pulmonary disorders.

Segmentation of airways in computed tomography is vital as it is the clue for accurate planning of pulmonary lesions biopsy and reliable support in intervention guidance. It helps us ensure that the sample is taken from the right tissue”, states Dr. Gil. What her team has presented is an original strategy, PICASSO (which stands for: PerIpheral bronChiAl Segmentation with Structural Optimization), which uses graph structural analysis in order to select the optimal thresholds of maps codifying bronchi local appearance. In other words, they have achieved a way of creating a personalized map of our bronchial tree with a more accurate method. “This is important for pulmonologists”, affirms Dr. Gil, “as each patient has a high variability in appearance of their bronchial tree, pulmonologists face new pathways in each bronchoschopy and the risk of getting lost is high”.

Early detection of lung cancer demonstrates a 20% reduction in mortality, studies show, and bronchoschopy examinations are the diagnostic cornerstone of lung cancer, allowing the biopsy of nodules with a minimum risk for patients. However, a main limitation of this technique is the difficulty to determine the best pathway to peripheral lesions.

“Accuracy of bronchoscopists at defining proper 3D routes is only around 40% for injuries that are located near airways at the fourth bronchial level at most. When we say fourth bronchial level, we mean that the bronchoschopist has had to choose a pathway within our bronchial  ‘crossroads’ at least 4 times”, states Dr. Carles Sánchez, TecnioSpring researcher at the CVC. Despite recent advances, new endoscopy techniques only increase diagnostic yield to 70%. “With PICASSO, we want to minimise errors and obtain reliable routes for bronchoscopists. Every error means more time in the operating room” claims Dr. Sanchez. An unsuccessful bronschoschopy means that the patient needs to return to the hospital at some point, with appointments being rescheduled twice or even three times until the physician gets it right. “This overwhelms our health systems and is terribly inefficient. Our aim is to overcome this”, adds Dr. Gil.

Endoscopy can be highly improved, by both reducing radiation and costs if imaging technology is improved in the guidance to the target lesion by personalising the treatment. “Radiation is an issue”, claims Dr. Gil, “current guidance systems are not only inaccurate but also rely on an XRay fluoroscopy that radiates both patients and clinicians”. Exposure to radiation is especially dangerous for physicians, who perform periodical interventions (2 to 3 per week) and whose main protection, lead vests and collars, are not only uncomfortable but also burdensome for clinical practice.

It is with this in mind that Dr. Debora Gil and her team are actively searching a simple and scalable way to segment bronchial airways in Computed tomography scans, delivering a first, reliable method: PICASSO. By combining descriptors of bronchi local appearance and graph global structural analysis, the aim is to fine-tune thresholds on the descriptors adapted for each bronchial level, paving the way towards automated personalised medicine in lung cancer treatments.

Our method’s main strength comes from being based on a computational codification of the airway anatomy”. This allows the segmentation of even the smallest bronchi, giving a highly detailed map of our lung system. The more detailed the map, the better the diagnosis. However, in order to build such a detailed map, the amount of information in traditional methods needs to be huge.

Unlike fashionable deep learning approaches”, she continues, “our method can perform well without an exhaustive training or access to big amounts of data”. This is PICASSO’s main advantage. As Dr. Carles Sánchez explains, in medical applications, the bottleneck for machine learning is the gathering of enough unambiguous annotated data. In the particular case of the segmentation of anatomical structures, this is a critical issue aggravated by the time-consuming delineating of manual contours and the high variability among experts. PICASSO overcomes this crucial limitation by adjusting its segmentation parameters depending on the personal structural bronchial anatomy of a patient.

In terms of societal impact, from the top 10 AI applications identified by Accenture in 2020, 6 of them are health related. The same study states that Growth in the AI health market is expected to reach $6.6 billion by 2021. Although PICASSO isn’t just ready, the journey towards actual deployment in clinical centres is closer each day. Dr. Gil’s team is currently working on a parallelization of the method in a set of cloud computing platforms. The aim: to deploy PICASSO in clinical centres in the form of a software as a service model in order to minimize costs and allow its use globally.


Reference: Gil D, Sánchez C, Borras A, Diez-Ferrer M, Rosell A.  Segmentation of distal airways using structural analysis. 2020 PLOS one

Related articles: 

Developing The Bronchoscope Of Tomorrow



A total of 7 CVPRs 2020 accepted at CVC

There’s no doubt that this year’s events will be quite different to past years, however, we are very pleased to announce that a total of 7 papers have been accepted from CVC people to this year’s CVPR. Congratulations to the authors!

The papers can be accessed here (list isn’t ready with all the papers, will be updated as soon as they are published in arxiv):

Y. Wang, A. González-Garcia, D. Berga, L. Herranz, F. Shahbaz Khan, J. van de Weijer (2020): MineGAN: effective knowledge transfer from GANs to target domains with few images

L. Porzi, M. Hofinger, I. Ruiz, J. Serrat, S. Rota Bulò, P. Kontschieder (2020): Learning Multi-Object Tracking and Segmentation from Automatic Annotations

V. Oguz Yazici, A. Gonzalez-Garcia, A. Ramisa, B. Twardowski, J. van de Weijer (2020): Orderless Recurrent Models for Multi-label Classification

L. Yu, B. Twardowski, X. Liu, L. Herranz, K. Wang, Y. Cheng, S. Jui, J. van de Weijer (2020): Semantic Drift Compensation for Class-Incremental Learning

Y. Wang, S. Khan, A. Gonzalez-Garcia, J. van de Weijer, F. Shahbaz Khan (2020): Semi-supervised Learning for Few-shot Image-to-Image Translation

S. Sudhakaran, S. Escalera, O. Lanz (2020): Gate-Shift Networks for Video Action Recognition


The CVC collaborates in two projects of the RIS3CAT Media Community

The Computer Vision Center is taking part of the newly launched RIS3CAT Media Community, which will promote eight R&D+i projects within the cultural and creative industry sector. The CVC is collaborating in two: ViVIM and Zinkat.

Coordinated by technology center Eurecat, the RIS3CAT Media community will develop eight collaborative projects in the next four years with an investment of 13.688.490 €. This action plan will be supported with 6.000.000 € granted by the Government of Catalonia through its Agency for Business Competitiveness, ACCIÓ.

This community is comprised of 34 members which will collaborate to turn knowledge and technology into economic, social and cultural value, and to develop new opportunities of cultural production, diffusion and consumption. To this end, RIS3CAT Media brings together science, technology and creativity to tackle topics such as Artificial Intelligence and automation; Big Data and data analysis; hyper-personalization; new formats development; human-machine interaction and haptics; cloud computing; next-generation networks; cybersecurity and cryptography; social media and collaborative economy; and immersive media, virtual reality and augmented reality.

The CVC is collaborating in the following projects:

ViVIM: Computer Vision for Immersive Multi-Platform Video

ViVIM is a collaborative project aimed at implementing an innovative media production and consumption system. This system will focus on a novel form of audiovisual narrative based on omnidirectional video. Furthermore, the project will develop production tools for immersive screens, support for omnidirectional cameras and coordinated access to content through head mounted displays, tablet devices and conventional TV. Finally, ViVIM will test the viability of this system by means of two demonstrators addressed to live and deferred broadcasting.

The ViVIM project is coordinated by i2CAT and its consortium includes renowned academic and industry actors with complementary expertise and skills, such as the Catalan Broadcasting Corporation (CCMA), Eurecat, Visyon and the CVC.

“The project ViVIM is funded by the Agency for Business Competitiveness of the Government of Catalonia (ACCIÓ) within the RIS3CAT Media Community with the reference number COMRDI18-1-008”


zinkCAT will develop a new protocol for designers and marketing agencies that will allow more connected designs to improve the online user expereince. The vision of the project is to link innovation and technology with the creative process specifically the design oriented to communication processes, which will allow the capture of new data relating to Artificial Intelligence algorithms to capture effective leads, increase sales and improve the audience experience.

The project is coordinated by the Creactivitat agency and its consortium is formed by BCD Barcelona Design Centre, Eurecat, Konodrac and the CVC.

The project zinkCAT is funded by the Agency for Business Competitiveness of the Government of Catalonia (ACCIÓ)