Word Spotting and Recognition with Embedded Attributes

Jon Almazán, Albert Gordo, Alicia Fornés, Ernest Valveny

Abstract

We deal with the problems of word spotting and word recognition on images. In word spotting, the goal is to find all instances of a query word in a dataset of images. In recognition, the goal is to recognize the content of the word image, usually aided by a dictionary or lexicon. We propose a formulation for word representation and matching based on embedded attributes that jointly addresses these two problems. Contrary to most other existing methods, our representation has a fixed length, is low dimensional, and is very fast to compute and, especially, to compare.

We propose to use character attributes to learn a semantic representation of the word images and then perform a calibration of the scores with CCA that puts images and text strings in a common subspace. After that, spotting and recognition become simple nearest neighbor problems in a very low dimensional space. We test our approach on four public datasets of both document and natural images showing results comparable or better than the state-of-the-art on spotting and recognition tasks.

Material

[New] Extended version accepted in TPAMI
[New] Code
Original ICCV2013 paper
ICCV2013 Poster

Citation

@inproceedings{almazan2014, 
author = {J. Almaz\'an and A. Gordo and A. Forn\'es and E. Valveny}, 
title = {Word Spotting and Recognition with Embedded Attributes}, 
booktitle = {TPAMI}, 
year = {2014}}

@inproceedings{almazan2013, 
author = {J. Almaz\'an and A. Gordo and A. Forn\'es and E. Valveny}, 
title = {Handwritten Word Spotting with Corrected Attributes}, 
booktitle = {ICCV}, 
year = {2013}}