Image Sequence Evaluation

The ISE Lab research topics cover different cognitive vision skills for performing video-hermeneutics, i.e. the semantic understanding of human behaviors in image sequences, either captured from cameras or taken from internet. This video-hermeneutics process requires, on one hand, the computational analysis of human motion and, on the other hand, an epistemological reasoning on the the context in which human motion is being detected.

Consequently our research lines are driven to infer plausible semantic interpretations from pixel values, and to explain these conceptual results to end-users using natural-language texts. In particular, ISE Lab is focused on three main topics: object detection and tracking; human behavior modeling; and context recognition.

The first research line aims to detect and keep track of those agents detected in a scene. Detection errors can be handled by keeping the identification of the agents over time, while considering motion filters for avoiding target losses during occlusions.

The second goal is the generation of natural language texts which convey what is happening in the scene, and why. By combining the knowledge of the human motion and the context, we use reasoning tools and symbolic representations to infer and explain those observed behaviors.

The third challenge is focused on context recognition. Our aim is to extract the knowledge of the context in which human motion is observed. In order to achieve this goal, context recognition is hierarchically divided into three subtasks: (i) scene classification of images into indoor (such as offices, corridors, rooms, subway stations, cafeterias...) or outdoor (streets, squares, airports and highways) scenarios; (ii) semantic segmentation of regions in either indoor (wall, ground...) or outdoor (road, sidewalk, building, trees, sky...) scenarios; and (iii) object detection again in either indoor (window, chair, tables, book, telephone..) and outdoor (car, bicycles, dog, airplanes...) scenarios.

Summarizing, the video-hermeneutics paradigm provides a challenging domain of research on Cognitive Vision which encompasses topics on not only Computer Vision, but also Artificial Intelligence, Computational Linguistics and Computer Animation.