Physical and digital content are usually disconnected, this project seeks to link those two sources of information.
Dr. Fernando Vilariño, researcher at the Computer Vision Center (CVC) and associate professor of the Universitat Autònoma de Barcelona has received a Google Faculty Research Award for the project “Linking physical documents with digital information”.
Physical contents, such as books or magazines, and digital content, like contextualized references, descriptions and multimedia contents, are usually disconnected. This project seeks to link those two sources of information by using a low cost eye-tracker and a Google glass device while reading a document.
In order to do it so, the person’s head and eyes position will be tracked so the system will be able to tell which region of the document, or word, the person is reading. From this data, the system can display useful information in the Google glass display.
The project has four main research challenges to address; first of all, natural light eye-trackers are very sensitive to light variations and the used tracking features very unstable, so it can lead to accuracy errors. Secondly, they need to develop and algorithm for on-the-fly mapping between the obtained data by the eye-tracker and the document which allows minimizing the accumulated errors that come from the calibration of both eye-tracker and Google Glass camera.
Then, the researchers will also have to develop another algorithm to assure an efficient on-line recognition of the digitized documents and automatic word-by-word identification.
Finally, they must assure that it will have a real time implementation, a fully functional open software application using low cost eye-tracker and Google Glasses for on-the-fly word by word identification, so at each moment, the information displayed is related to the area of the document that the user is looking at.
The project particularly seeks to offer an augmented-reading experience to English students in their natural environment, so the validation of this research will be performed in a living lab, with actual English learners, therefore the feedback received by the researchers will be from real users in a real learning context. Besides, the application can be easily adapted to different reading experiences such as map analysis, detailed observation of artwork or music reading among others.
Google Research Awards are one-year awards structured as unrestricted gifts to universities to support the work of world-class full-time faculty members at top universities around the world.
For more information you can contact with:
Dr. Fernando Vilariño: firstname.lastname@example.org
+34 93 581 38 41