Description of the result files format

Introduction

This page presents the file format that has to be used to describe results obtained by participants. This format is XML-based, as:

• XML becomes a standard of data representation, and a lot of tools are available to handle XML files (as XSLT style sheets). This is very suitable for the contest, as we plan to automatically process the results.
• Results may be very different from a test to another one, and as XML allows to express these differences.
• Parsing XML requires a XML parser, but generating remains it relatively simple.

DTD definition (download it!)

    
<!ELEMENT testresult (testname, participant, imageresult+)>
<!ELEMENT testname        (#PCDATA)>
<!ELEMENT participant     (#PCDATA)>
<!ELEMENT imageresult     (imagename, symbol*)>
<!ELEMENT imagename       (#PCDATA)>
<!ELEMENT symbol (symbolname, location?, orientation?, scalefactor?, time?,
                  confidencerate?, secondcandidate?, thirdcandidate?)>
<!ELEMENT symbolname      (#PCDATA)>
<!ELEMENT location         EMPTY>
<!ELEMENT orientation     (#PCDATA)>
<!ELEMENT scalefactor     (#PCDATA)>
<!ELEMENT time            (#PCDATA)>
<!ELEMENT confidencerate  (#PCDATA)>
<!ELEMENT secondcandidate (#PCDATA)>
<!ELEMENT thirdcandidate  (#PCDATA)>

<!ATTLIST location x1 CDATA #REQUIRED
                   y1 CDATA #REQUIRED
                   x2 CDATA #REQUIRED
                   y2 CDATA #REQUIRED>
  

Description

• testresult: root of the XML document, containing all information resulting from a test file.
• testname: name of the test file used for the current test, available in the [testname] section of the .test file (see test format description).
• participant: name of the participant, given by the contest organization committee (see naming conventions).
• imageresult: element describing results obtained from a single test image.
• symbol: a recognized symbol. Please note that several occurrences of the same symbol have to be described in several symbol elements.
• symbolname: name of the recognized symbol. This name must match the filename of a model symbol, and thus must be postfixed by .vec extension.
• location: location of the current occurence of the recognized symbol. This element must include 4 attributes : x1, y1, x2 and y2, corresponding to the coordinates of 2 diagonally opposite corners of the bounding box of the recognized symbol.
• orientation: angle of the recognized symbol, according to the model (it is assumed that all symbol models are defined with an orientation of 0). Angles are measured as positive in the clockwise direction and are expressed in degrees.
• scalefactor: scale factor of the recognized symbol, according to the scale of the model (which is assumed to be equal to 1).
• time: processing time (please note that this information will be only used to determine the global complexity of the processing). Time is expressed in milliseconds.
• confidencerate: confidence rate of the recognition for methods that delivery this result. The rate has to be defined between 0 (null confidence rate) and 1 (perfect confidence rate).
• secondcandidate and thirdcandidate: name of the second and third symbol candidates for this symbol. Each name must match the filename of a model symbol, and thus must be postfixed by the .vec extension.

Important notes: further explanations about the DTD

• In the XML result files, the required elements are testname, participant and one imageresult element per test image (as described in the .test file used).
• In the imageresult element, the only required element is the imagename element. Within the imageresult element, one symbol element is required to describe each recognized symbol. Thus, if none symbol is found, the imageresult does not contain any symbol element.
• The location element is required only if the image test contains non-segmented symbols.
• The elements orientation, scalefactor, time, confidencerate, secondcandidate and thirdcandidate are never required. Participants can define them, but the data defined inside these elements will not be processed for the contest (they can be used to generate further statistics about the contest).

Example

    
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE testresult SYSTEM "GRECParticipantResults.dtd">
<testresult>
  <testname>testgrec</testname>
  <participant>participant1</participant>
  <imageresult>
    <imagename>
      testgrec-image1.tif
    </imagename>
    <symbol>
      <symbolname>ArchitecturalB.vec</symbolname>
      <location x1="35" y1="65" x2="67" y2="108"/>
      <orientation>34</orientation>
      <confidencerate>0.9885</confidencerate>
    </symbol>
  </imageresult>
  <imageresult>
    <imagename>
      testgrec-image2.tif
    </imagename>
    <symbol>
      <symbolname>ElectricalC.vec</symbolname>
      <location x1="108" y1="2003" x2="54" y2="21"/>
    </symbol>
  </imageresult>
  <imageresult>
    <imagename>
      testgrec-image3.tif
    </imagename>
  </imageresult>
  <imageresult>
    <imagename>
      testgrec-image4.tif
    </imagename>
    <symbol>
      <symbolname>ArchitecturalF.vec</symbolname>
      <location x1="30" y1="77" x2="165" y2="245"/>
    </symbol>
    <symbol>
      <symbolname>ArchitecturalF.vec</symbolname>
      <location x1="450" y1="479" x2="35" y2="88"/>
    </symbol>
  </imageresult>
  <imageresult>
    <imagename>
      testgrec-image5.tif
    </imagename>
    <symbol>
      <symbolname>ElectricalB.vec</symbolname>
      <confidencerate>0.97</confidencerate>
      <secondcandidate>ElectricalF.vec</secondcandidate>
    </symbol>
  </imageresult>
</testresult>
    

Example annotations

• testgrec-image1.tif: one symbol (ArchitecturalB.vec) is recognized. The location, orientation and confidence rate of this occurrence are detailed.
• testgrec-image2.tif: one symbol (ArchitecturalC.vec) is recognized. The location of this occurrence is detailed.
• testgrec-image3.tif: none symbol is found.
• testgrec-image4.tif: two occurrences of the symbol ArchitecturalF.vec are recognized, each one described in a separate symbol element (with the location of the occurrences).
• testgrec-image5.tif: one symbol (ElectricalB.vec) is recognized with a confidence rate of 0.97. A second candidate is detailed, but it will not be used for the contest (it will maybe used in further statistics about the contest).