Visit our website
Pricing
Blog Playground API Docs

Supported Formats

Sources: Wikipedia and IANA

This page details all the supported Datasaur formats, provides examples for each format and clarifies expected file structure where appropriate. Note: Through file transformers we can customize the output format.

TXT

TXT file is a simple file format that contains unformatted text and can be easily opened and edited using a basic text editor. It is commonly used for storing and exchanging data, code, and other textual information.

2KB
Datasaur sample - TXT.txt

TSV

A TSV (tab-separated values) file is a simple text format for storing data in a tabular structure. A TSV file encodes a number of records that may contain multiple fields.

  • Each record is represented as a single line.

  • Each field

    value is represented as text.

  • Fields in a record are separated from

    one other by the tab character .

    • Note that because is a special character for this format, fields that contain tabs are not allowed in this encoding.

  • The header (first) line of this encoding contains the name of

    each field, separated by tabs.

Example

Book Title    Author    Genre
Sherlock Holmes: A Study in Scarlet    Sir Arthur Conan Doyle    Fiction
To Kill a Mockingbird    Harper Lee    Fiction
Alan Turing: The Enigma    Andrew Hodges    Non fiction
Humble Pie    Gordon Ramsay    Non fiction
The Little Prince    Antoine de Saint-Exupéry    Fiction

IOB (specialized .tsv)

IOB (inside, outside, beginning) is a common labeling format for labeling tokens in computational linguistics (ex: named-entity recognition). IOB is also a .tsv, but conforms to the following rules:

  • The B- prefix before a tag indicates that the tag is the beginning of a chunk.

  • The I- prefix before a tag indicates that the tag is inside a chunk.

  • The B- tag is used only when a tag is followed by a tag of the same type without O tokens between them.

  • The O tag indicates that a token does not belong to a chunk.

Example

Sherlock	B-PER
Holmes	I-PER
become	O
widely	O
popular	O
in	O
1891 YEAR	O
.	O

CSV

A CSV (comma-separated values) file is a delimited text file that uses a comma to separate values. Each line of the file is a data record. Each record consists of one or more fields, separated by commas. The use of the comma as a field separator is the source of the name for this file format.

A CSV file typically stores tabular data (numbers and text) in plain text, in which case each line will have the same number of fields.

Example

Book Title,Author,Genre
Sherlock Holmes: A Study in Scarlet,Sir Arthur Conan Doyle,Fiction
To Kill a Mockingbird,Harper Lee,Fiction
Alan Turing: The Enigma,Andrew Hodges,Non fiction
Humble Pie,Gordon Ramsay,Non fiction
The Little Prince,Antoine de Saint-Exupéry,Fiction

💡 As for now, a row-based project using CSV format does not support answers containing ;. We treat it as multiple answers. For example, the answer is She brings some flowers: rose; sunflower; and daisy. It will be interpreted as three answers that containShe brings some flowers: rose ,sunflower , and daisy.

XLS and XLSX

XLS and XLSX is a well-known format for Microsoft Excel documents that was introduced by Microsoft XLS is an older format that was used in older versions of Excel, while XLSX is a newer format that is the default in more recent versions of Excel. Both formats allow users to input, organize, and analyze data in rows and columns. They also support features such as formulas, charts, and graphs. XLSX is a more efficient format that offers better data recovery and larger file size limits.

Example

5KB
Datasaur sample - XLS and XLSV.xlsx

JSON

JSON (JavaScript Object Notation), or JSON Simplified, is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and array data types (or any other serializable value).

A JSON file may contain the following data structures:

  • An object is an unordered set of name/value pairs.

    • An object begins with {left brace and ends with }right brace. Each name is followed by :colon and the name/value pairs are separated by ,comma.

  • An array is an ordered collection of values.

    • An array begins with [left bracket and ends with ]right bracket. Values are separated by ,comma.

  • A value can be a string in double quotes, or a number, or true or false or null, or an object or an array. These structures can be nested.

  • A string is a sequence of zero or more Unicode characters, wrapped in double quotes, using backslash escapes. A character is represented as a single character string. A string is very much like a C or Java string.

  • A number is like a C or Java number, except that the octal and hexadecimal formats are not used.

  • Whitespace can be inserted between any pair of tokens. Excepting a few encoding details, that completely describes the language.

In the example below, here are the objects recognized at Datasaur.

  • text: the sentence.

  • entities: array of label applied

    • text: the token

    • type: the label applied.

    • start_idx: the character position in the labeled token.

      • The character position uses zero-based index.

    • end_idx: the last character position + 1 (because end_index does not include the last character).

      • The character position uses zero-based index.

JSON (Simplified) export format limitation:

  • Can't export arrow label

  • Can't export label multi sentence

Example

[
  {
    "text": "The new series Narcos created by Chris Brancato , Eric Newman and Carlo Bernard , represents a pretty ambitious step for Netflix .",
    "entities": [
      {
        "text": "Narcos",
        "type": "TITLE",
        "start_idx": 15,
        "end_idx": 21
      },
      {
        "text": "Chris Brancato",
        "type": "PER",
        "start_idx": 33,
        "end_idx": 47
      },
      {
        "text": "Carlo Bernard",
        "type": "PER",
        "start_idx": 66,
        "end_idx": 79
      },
      {
        "text": "Eric Newman",
        "type": "PER",
        "start_idx": 50,
        "end_idx": 61
      },
      {
        "text": "Netflix",
        "type": "ORG",
        "start_idx": 121,
        "end_idx": 128
      }
    ]
  }
]

JSON_TABULAR

JSON_TABULAR is a derivative of the JSON format that is used to represent table data format (in the form of an array of objects). You can choose this format if you are working on row-based labeling.

Example

[
  {
    "Book Title": "Sherlock Holmes: A Study in Scarlet",
    "Author": "Sir Arthur Conan Doyle",
    "Genre": "Fiction"
  },
  {
    "Book Title": "To Kill a Mockingbird",
    "Author": "Harper Lee",
    "Genre": "Fiction"
  },
  {
    "Book Title": "Alan Turing: The Enigma",
    "Author": "Andrew Hodges",
    "Genre": "Non fiction"
  },
  {
    "Book Title": "Humble Pie",
    "Author": "Gordon Ramsay",
    "Genre": "Non fiction"
  },
  {
    "Book Title": "The Little Prince",
    "Author": "Antoine de Saint-Exupéry",
    "Genre": "Fiction"
  }
]

JSONL (JSON Lines)

Per version 6.43.0, Datasaur now supports JSONL natively 🎉

JSONL (JSON Lines) - https://jsonlines.org/ - is a text file format suitable for storing data that can be processed one record at a time. Datasaur supports a subset of valid JSONL files, namely:

  1. the file must end in the .jsonl extension

  2. each record in a file must be in the same structure / format. If the first record / line is an array, all the following lines must also be an array. If the first record is an object, all the following lines must also be a JSON object.

The JSONL file format is supported for row-based project.

Here are some sample JSONL structure that Datasaur supports:

159B
json-lines-as-csv-replacement.jsonl
277B
json-lines-with-object.jsonl

For JSONL with objects, you can have nested values, for example:

{ "detail" : { "id": 1, "name": "product-name" }, "key": "value", "another-key": "another-value" }

Datasaur will render all values after stringify-ing them.

Note that Datasaur relies on the first record / line to check the header length. Any items not in the first line will not be parsed.

Here is an example of how it may affect your workflow:

Let’s take the sample data above, and alter it a bit such that if someone has not completed a session, there is no completed data stored

{"name": "Gilbert", "session": 2013, "score": 24}
{"name": "Alexa", "session": 2014, "score": 29, "completed": true}
{"name": "May", "session": "2012B", "score": 14}
{"name": "Deloise", "session": "2012A", "score": 19, "completed": true}

This file will be parsed just fine, but you will be missing the completed column - because there is no completed key in the first line.

As such, we highly recommend making your data consistent between each line, to ensure the best compatibility with our parser.

TSV_NON_IOB

TSV_NON_IOB is a derivative of the TSV format that represents data that does not follow the IOB format - for example, B-GEO is just GEO. If your project is token-based (with or without arrows), you can choose this format for export.

A TSV_NON_IOB file contains the following data structure (this explanation is based on our example below):

  1. #FORMAT: the file header.

  2. #Text: the sentence representation.

  3. 1-1: the sentence-token.

    1. The first 1 indicates the sentence number.

    2. The second 1 indicates the token number.

  4. 0-3: the character index.

  5. TITLE[1]: the label applied.

    1. [1] indicates uniquely identify annotation across lines.

  6. Column 5: indicates layer 2.

  7. author[2-1]: the label on the arrow.

    1. 2 indicates the arrow’s token origin.

    2. 1 indicates the arrow’s token destination.

  8. Column 7: indicates layer 4.

  9. Column 8: indicates layer 5.

Note: column 5, 7, and 8 will be filled if you label the token in the mentioned layers.

_\_💡_We built this format to be compatible with [_WebAnno](https://webanno.github.io/webanno/releases/3.4.5/docs/user-guide.html#sect_webannotsv)_._

Example (token-based)

#FORMAT=Datasaur TSV 3

#Text=The Little Prince is a novella by French aristocrat , writer , and aviator Antoine de Saint - Exupéry .
1-1    0-3    The    TITLE[1]    _    _    _    _
1-2    4-10    Little    TITLE[1]    _    _    _    _
1-3    11-17    Prince    TITLE[1]    _    _    _    _
1-4    18-20    is    _    _    _    _    _
1-5    21-22    a    _    _    _    _    _
1-6    23-30    novella    _    _    _    _    _
1-7    31-33    by    _    _    _    _    _
1-8    34-40    French    _    _    _    _    _
1-9    41-51    aristocrat    _    _    _    _    _
1-10    52-53    ,    _    _    _    _    _
1-11    54-60    writer    _    _    _    _    _
1-12    61-62    ,    _    _    _    _    _
1-13    63-66    and    _    _    _    _    _
1-14    67-74    aviator    _    _    _    _    _
1-15    75-82    Antoine    PER[2]    _    _    _    _
1-16    83-85    de    PER[2]    _    _    _    _
1-17    86-91    Saint    PER[2]    _    _    _    _
1-18    92-93    -    PER[2]    _    _    _    _
1-19    94-101    Exupéry    PER[2]    _    _    _    _
1-20    102-103    .    _    _    _    _    _

Example (token-based with arrows)

#FORMAT=Datasaur TSV 3

#Text=The Little Prince is a novella by French aristocrat , writer , and aviator Antoine de Saint - Exupéry .
1-1    0-3    The    TITLE[1]    _    *    author[2_1]    _
1-2    4-10    Little    TITLE[1]    _    _    _    _
1-3    11-17    Prince    TITLE[1]    _    _    _    _
1-4    18-20    is    _    _    _    _    _
1-5    21-22    a    _    _    _    _    _
1-6    23-30    novella    _    _    _    _    _
1-7    31-33    by    _    _    _    _    _
1-8    34-40    French    _    _    _    _    _
1-9    41-51    aristocrat    _    _    _    _    _
1-10    52-53    ,    _    _    _    _    _
1-11    54-60    writer    _    _    _    _    _
1-12    61-62    ,    _    _    _    _    _
1-13    63-66    and    _    _    _    _    _
1-14    67-74    aviator    _    _    _    _    _
1-15    75-82    Antoine    PER[2]    _    _    _    _
1-16    83-85    de    PER[2]    _    _    _    _
1-17    86-91    Saint    PER[2]    _    _    _    _
1-18    92-93    -    PER[2]    _    _    _    _
1-19    94-101    Exupéry    PER[2]    _    _    _    _
1-20    102-103    .    _    _    _    _    _

CoNLL-U

Universal Dependencies use a revised version of the CoNLL-X format called CoNLL-U. Sentences consist of one or more word lines, and word lines contain the following fields:

  1. sent_id: Sentence id.

  2. text: Sentence.

  3. ID: Word index, integer starting at 1 for each new sentence; may be a range for multiword tokens; may be a decimal number for empty nodes (decimal numbers can be lower than 1 but must be greater than 0).

  4. FORM: Word form or punctuation symbol.

  5. LEMMA: Lemma or stem of word form.

  6. UPOS: Universal part-of-speech tag.

  7. XPOS: Language-specific part-of-speech tag; underscore if not available.

  8. FEATS: List of morphological features from the universal feature inventory or from a defined language-specific extension; underscore if not available.

  9. HEAD: Head of the current word, which is either a value of ID or zero (0).

  10. DEPREL: Universal dependency relation to the HEAD (root iff HEAD = 0) or a defined language-specific subtype of one.

  11. DEPS: Enhanced dependency graph in the form of a list of head-deprel pairs.

  12. MISC: Any other annotation.

Example

# sent_id = 1
# text = Sherlock Holmes become widely popular in 1891 .
1	Sherlock	_	_	NNP	_	3	nsubj	_	_
2	Holmes	_	_	NNP	_	_	_	_	_
3	become	_	_	VBD	_	0	root	_	_
4	widely	_	_	RB	_	5	advmod	_	_
5	popular	_	_	JJ	_	3	xcomp	_	_
6	in	_	_	IN	_	_	_	_	_
7	1891	_	_	CD	_	_	_	_	_
8	.	_	_	.	_	_	_	_	_

CoNLL_2003

CoNLL_2003 is usually used for POS tagging and named entity recognition labeling. All data files contain one word per line with empty lines representing sentence boundaries. At the end of each line there is a tag which states whether the current word is inside a named entity or not. The tag also encodes the type of named entity. Each line contains four fields:

  1. The word

  2. Part of-speech tag

  3. Chunk tag

  4. Named entity tag

Note: Importing or exporting files with conll_2003 format can be done if you checked the following task settings.

  • Tokens and token spans should have at most one label.

  • Allow arrows to be drawn between labels. Checking this setting will activate layer feature.

You could do POS tagging on Layer 0 and NER tagging on Layer 1. If you export the file with conll_2003, the result will be as shown as sample file below.

803KB
Datasaur sample - CoNLL 2003.conll

JSON_ADVANCED

JSON_ADVANCED is a proprietary Datasaur format designed in collaboration with our users to capture all possible data. This format is commonly used for partial token labeling projects. You can also use it when exporting token-based with arrow projects, such as coreference and dependency.

A JSON_ADVANCED file may contain the following data structures:

  1. Sentences field

    1. id: the sentence position.

    2. content: the text of the sentence.

    3. tokens: the tokens form of the sentence.

    4. labels

      1. l: the label applied.

      2. layer: the layer position of the labels. This field is reserved for a project where a labeling of multiple tag set at once. For now you can disregard this field and this field is always set to 0.

      3. id: the unique identifier of a label.

        1. If the id has 9 segments, this indicates span label. For example, INNM0ViFwo8LluMTaTIK9:0:0:14:0:0:18:6:0 and here's the explanation <label set item id>:<layer>:<sidS>:<s>:<charS>:<sidE>:<e>:<charE>:<index>.

        2. If the id has 21 segments, this indicates arrow label. For example, tfc1FkbbEk9fOLx6haR1s:0:INNM0ViFwo8LluMTaTIK9:0:0:14:0:0:18:6:0:Oq_VuB0s_N7D8ZY0rgYsg:0:0:0:0:0:2:5:0:0 and here's the explanation <label set item id>:<arrow layer>:<….. origin id>:<….destination id>:<arrow index>.

      4. hashCode: Datasaur's code to represent label information __

        1. Span label. For example, SPAN:gpe:0:0:0:4:0:0:0:4:3:0:undefined:undefined. Below is the explanation:

          1. type:label set item id:layer or label set index:start cell line:cell index:start token index: start char index: end cell line: end cell index: end token index: end char index: counter.

        2. Arrow label. For example, ARROW:dyC-o1HBnn49dcqDSphmJ:1:0:0:0:0:0:0:10:6:0:SPAN:geo:0:0:0:0:0:0:0:0:4:0:undefined:undefined:SPAN:geo:0:0:0:10:0:0:0:10:6:0:undefined:undefined. Below is the explanation:

          1. type:label set item id:layer or label set index:start cell line:cell index:start token index: start char index: end cell line: end cell index: end token index: end char index: counter:<span label: origin>:<span label: destination>.

      5. documentId: the id of document.

      6. sidS, sidE: the sentence starting and ending position of a label in 0-based index. In Datasaur, it is possible that a label spans across sentences.

      7. s: the token starting position of a label in the starting sentence in 0-based index.

      8. e: the token ending position of a label in the ending sentence in 0-based index.

      9. charS: the character starting position of a label in the starting token in 0-based index.

      10. charE: the character ending position of a label in the ending token in 0-based index.

    5. metadata: additional information for a cell

  2. labelerInfo: the information about the labeler.

    1. id: the unique identifier of a labeler (each labeler has different id).

    2. email: email that labeler used when signing in.

    3. displayName: the display name of the email.

  3. labelSets: contains all the label items that you used for the project.

    1. index: the position of the label set in UI

    2. labelItems: an array of labelItems for a label set

      • id: id of the labelSetItem

      • labelName: the displayed name of the label set item

      • parentId: id of the parent label set item

      • color: the color of the label set item

  4. labels: an array of labels for the document

    1. labelText: label content for row-based project. It will be null for other project beside the row-based project.

    2. id: identifier from the applied label.

    3. documentId : identifier for document where the label is applied.

    4. startCellLine: starting line sentence position

    5. startCellIndex: starting line column position

    6. startTokenIndex: starting token index position

    7. startCharIndex: starting character index position (relative to tokenIndex, start from 0 again when tokenIndex incremented)

    8. endCellLine: ending line sentence position

    9. endCellIndex: ending line column position

    10. endTokenIndex: ending token index position

    11. endCharIndex: ending character index position

    12. layer: the layer where the token is positioned

    13. counter: labels with the same name to be placed multiple times in the same position, start from 0

    14. type: the type of labels -> SPAN, ARROW, BOUNDING_BOX

    15. createdAt:

      1. Labeler: the time labels applied

      2. Reviewer: the time labels got accepted

    16. updatedAt: last update timestamp on the label

    17. Review related fields

      1. acceptedByUserId: the user id of a reviewer who accepts the label. It will be null if there's no user who accept it manually.

      2. rejectedByUserId: the user id of a reviewer who rejects the label. It will be null if there's no user who rejects it manually

      3. labeledByUserId: the user id of a reviewer

      4. labeledBy:

        • CONFLICT if it has not been resolved

        • REVIEWER if it has been resolved

        • AUTO if it has been resolved by meeting the consensus

    18. Arrow label type specific fields

      1. originId: origin id of an arrow label

      2. originNumber: auto increment ID for origin

      3. destinationId: origin id of an arrow label

      4. destinationNumber: auto increment ID for destination

    19. Bounding box label type specific fields

      1. pageIndex: index of the page if the document contain multiple pages

      2. nodeCount: total number of the bounding box points

      3. x0: x coordinate of top left position of the bounding box

      4. y0: y coordinate of top left position of the bounding box

      5. x1: x coordinate of top right position of the bounding box

      6. y1: y coordinate of top right position of the bounding box

      7. x2: x coordinate of bottom right position of the bounding box

      8. y2: y coordinate of bottom right position of the bounding box

      9. x3: x coordinate of bottom left position of the bounding box

      10. y3: y coordinate of bottom left position of the bounding box

    20. pages: an array of page information for OCR project type

      1. pageIndex: index of the page if the document contain multiple pages

      2. pageHeight: original page height in pixel

      3. pageWidth: original page width in pixel

  5. comments

    1. id: the id of the comment

    2. parentId: the id of the parent comment - this will be filed if the comment thread has replies.

    3. hashCode: Datasaur's code to represent comment's information, including the value being commented

    4. message: the content of the comment

    5. type: the type of comment, can be SPAN_LABEL,SPAN_TEXT, ARROW_LABEL, and CELL_LABEL

    6. userId: the id of user who create the comment

    7. createdAt: the time when the user create the comment

Example (token-based with arrow)

{
  "sentences": [
    {
      "id": 0,
      "content": "The Little Prince is a novella by French aristocrat , writer , and aviator Antoine de Saint - Exupéry .",
      "tokens": [
        "The",
        "Little",
        "Prince",
        "is",
        "a",
        "novella",
        "by",
        "French",
        "aristocrat",
        ",",
        "writer",
        ",",
        "and",
        "aviator",
        "Antoine",
        "de",
        "Saint",
        "-",
        "Exupéry",
        "."
      ],
      "labels": [
        {
          "id": "vGOy0ZKA-2rqK7netKz9I:0:0:0:0:0:2:5:0",
          "l": "vGOy0ZKA-2rqK7netKz9I",
          "layer": 0,
          "deleted": false,
          "labeledBy": "PRELABELED",
          "createdAt": "2024-08-05T20:23:47.284Z",
          "updatedAt": "2024-08-05T20:23:47.606Z",
          "hashCode": "SPAN:vGOy0ZKA-2rqK7netKz9I:0:0:0:0:0:0:0:2:5:0:undefined:undefined",
          "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
          "confidenceScore": null,
          "sidS": 0,
          "s": 0,
          "charS": 0,
          "sidE": 0,
          "e": 2,
          "charE": 5
        },
        {
          "id": ":0:vGOy0ZKA-2rqK7netKz9I:0:0:0:0:0:2:5:0:9I-5oYKvnzJRWHgsZrDe_:0:0:14:0:0:18:6:0:0",
          "l": "",
          "layer": 0,
          "deleted": false,
          "labeledBy": "LABELER",
          "labeledByUserId": 4500,
          "createdAt": "2024-08-05T20:26:12.466Z",
          "updatedAt": "2024-08-05T20:26:12.611Z",
          "hashCode": "ARROW::0:0:0:0:0:0:0:18:6:0:SPAN:vGOy0ZKA-2rqK7netKz9I:0:0:0:0:0:0:0:2:5:0:undefined:undefined:SPAN:9I-5oYKvnzJRWHgsZrDe_:0:0:0:14:0:0:0:18:6:0:undefined:undefined",
          "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
          "confidenceScore": null,
          "sidS": 0,
          "s": 0,
          "charS": 0,
          "sidE": 0,
          "e": 18,
          "charE": 6
        },
        {
          "id": "A-92mMT_WppaawOOBJbjt:1:9I-5oYKvnzJRWHgsZrDe_:0:0:14:0:0:18:6:0:vGOy0ZKA-2rqK7netKz9I:0:0:0:0:0:2:5:0:0",
          "l": "A-92mMT_WppaawOOBJbjt",
          "layer": 1,
          "deleted": false,
          "labeledBy": "PRELABELED",
          "createdAt": "2024-08-05T20:23:47.291Z",
          "updatedAt": "2024-08-05T20:23:47.622Z",
          "hashCode": "ARROW:A-92mMT_WppaawOOBJbjt:1:0:0:0:0:0:0:18:6:0:SPAN:9I-5oYKvnzJRWHgsZrDe_:0:0:0:14:0:0:0:18:6:0:undefined:undefined:SPAN:vGOy0ZKA-2rqK7netKz9I:0:0:0:0:0:0:0:2:5:0:undefined:undefined",
          "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
          "confidenceScore": null,
          "sidS": 0,
          "s": 0,
          "charS": 0,
          "sidE": 0,
          "e": 18,
          "charE": 6
        },
        {
          "id": "9I-5oYKvnzJRWHgsZrDe_:0:0:14:0:0:18:6:0",
          "l": "9I-5oYKvnzJRWHgsZrDe_",
          "layer": 0,
          "deleted": false,
          "labeledBy": "PRELABELED",
          "createdAt": "2024-08-05T20:23:47.284Z",
          "updatedAt": "2024-08-05T20:23:47.606Z",
          "hashCode": "SPAN:9I-5oYKvnzJRWHgsZrDe_:0:0:0:14:0:0:0:18:6:0:undefined:undefined",
          "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
          "confidenceScore": null,
          "sidS": 0,
          "s": 14,
          "charS": 0,
          "sidE": 0,
          "e": 18,
          "charE": 6
        }
      ],
      "metadata": []
    }
  ],
  "labelSets": [
    {
      "index": 0,
      "labelItems": [
        {
          "id": "vGOy0ZKA-2rqK7netKz9I",
          "labelName": "Novel"
        },
        {
          "id": "9I-5oYKvnzJRWHgsZrDe_",
          "labelName": "Male"
        },
        {
          "id": "A-92mMT_WppaawOOBJbjt",
          "labelName": "Author"
        }
      ]
    }
  ],
  "labels": [
    {
      "labelText": null,
      "id": "1685710595",
      "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
      "labeledByUserId": null,
      "startCellIndex": 0,
      "startCellLine": 0,
      "startTokenIndex": 0,
      "startCharIndex": 0,
      "endCellIndex": 0,
      "endCellLine": 0,
      "endTokenIndex": 2,
      "endCharIndex": 5,
      "layer": 0,
      "counter": 0,
      "labeledBy": "PRELABELED",
      "acceptedByUserId": null,
      "rejectedByUserId": null,
      "originNumber": "0",
      "destinationNumber": "0",
      "type": "SPAN",
      "labelSetItemId": "vGOy0ZKA-2rqK7netKz9I",
      "status": "ACCEPTED",
      "confidenceScore": null,
      "createdAt": "2024-08-05T20:23:47.284Z",
      "updatedAt": "2024-08-05T20:23:47.606Z",
      "hashCode": "SPAN:vGOy0ZKA-2rqK7netKz9I:0:0:0:0:0:0:0:2:5:0:undefined:undefined"
    },
    {
      "labelText": null,
      "id": "1685710604",
      "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
      "labeledByUserId": 4500,
      "startCellIndex": 0,
      "startCellLine": 0,
      "startTokenIndex": 0,
      "startCharIndex": 0,
      "endCellIndex": 0,
      "endCellLine": 0,
      "endTokenIndex": 18,
      "endCharIndex": 6,
      "layer": 0,
      "counter": 0,
      "labeledBy": "LABELER",
      "acceptedByUserId": null,
      "rejectedByUserId": null,
      "originId": "1685710595",
      "originNumber": "1685710595",
      "destinationId": "1685710596",
      "destinationNumber": "1685710596",
      "type": "ARROW",
      "labelSetItemId": "",
      "status": "ACCEPTED",
      "confidenceScore": null,
      "createdAt": "2024-08-05T20:26:12.466Z",
      "updatedAt": "2024-08-05T20:26:12.611Z",
      "origin": {
        "labelText": null,
        "id": "1685710595",
        "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
        "labeledByUserId": null,
        "startCellIndex": 0,
        "startCellLine": 0,
        "startTokenIndex": 0,
        "startCharIndex": 0,
        "endCellIndex": 0,
        "endCellLine": 0,
        "endTokenIndex": 2,
        "endCharIndex": 5,
        "layer": 0,
        "counter": 0,
        "labeledBy": "PRELABELED",
        "acceptedByUserId": null,
        "rejectedByUserId": null,
        "originNumber": "0",
        "destinationNumber": "0",
        "type": "SPAN",
        "labelSetItemId": "vGOy0ZKA-2rqK7netKz9I",
        "status": "ACCEPTED",
        "confidenceScore": null,
        "createdAt": "2024-08-05T20:23:47.284Z",
        "updatedAt": "2024-08-05T20:23:47.606Z"
      },
      "destination": {
        "labelText": null,
        "id": "1685710596",
        "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
        "labeledByUserId": null,
        "startCellIndex": 0,
        "startCellLine": 0,
        "startTokenIndex": 14,
        "startCharIndex": 0,
        "endCellIndex": 0,
        "endCellLine": 0,
        "endTokenIndex": 18,
        "endCharIndex": 6,
        "layer": 0,
        "counter": 0,
        "labeledBy": "PRELABELED",
        "acceptedByUserId": null,
        "rejectedByUserId": null,
        "originNumber": "0",
        "destinationNumber": "0",
        "type": "SPAN",
        "labelSetItemId": "9I-5oYKvnzJRWHgsZrDe_",
        "status": "ACCEPTED",
        "confidenceScore": null,
        "createdAt": "2024-08-05T20:23:47.284Z",
        "updatedAt": "2024-08-05T20:23:47.606Z"
      },
      "hashCode": "ARROW::0:0:0:0:0:0:0:18:6:0:SPAN:vGOy0ZKA-2rqK7netKz9I:0:0:0:0:0:0:0:2:5:0:undefined:undefined:SPAN:9I-5oYKvnzJRWHgsZrDe_:0:0:0:14:0:0:0:18:6:0:undefined:undefined"
    },
    {
      "labelText": null,
      "id": "1685710597",
      "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
      "labeledByUserId": null,
      "startCellIndex": 0,
      "startCellLine": 0,
      "startTokenIndex": 0,
      "startCharIndex": 0,
      "endCellIndex": 0,
      "endCellLine": 0,
      "endTokenIndex": 18,
      "endCharIndex": 6,
      "layer": 1,
      "counter": 0,
      "labeledBy": "PRELABELED",
      "acceptedByUserId": null,
      "rejectedByUserId": null,
      "originId": "1685710596",
      "originNumber": "1685710596",
      "destinationId": "1685710595",
      "destinationNumber": "1685710595",
      "type": "ARROW",
      "labelSetItemId": "A-92mMT_WppaawOOBJbjt",
      "status": "ACCEPTED",
      "confidenceScore": null,
      "createdAt": "2024-08-05T20:23:47.291Z",
      "updatedAt": "2024-08-05T20:23:47.622Z",
      "origin": {
        "labelText": null,
        "id": "1685710596",
        "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
        "labeledByUserId": null,
        "startCellIndex": 0,
        "startCellLine": 0,
        "startTokenIndex": 14,
        "startCharIndex": 0,
        "endCellIndex": 0,
        "endCellLine": 0,
        "endTokenIndex": 18,
        "endCharIndex": 6,
        "layer": 0,
        "counter": 0,
        "labeledBy": "PRELABELED",
        "acceptedByUserId": null,
        "rejectedByUserId": null,
        "originNumber": "0",
        "destinationNumber": "0",
        "type": "SPAN",
        "labelSetItemId": "9I-5oYKvnzJRWHgsZrDe_",
        "status": "ACCEPTED",
        "confidenceScore": null,
        "createdAt": "2024-08-05T20:23:47.284Z",
        "updatedAt": "2024-08-05T20:23:47.606Z"
      },
      "destination": {
        "labelText": null,
        "id": "1685710595",
        "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
        "labeledByUserId": null,
        "startCellIndex": 0,
        "startCellLine": 0,
        "startTokenIndex": 0,
        "startCharIndex": 0,
        "endCellIndex": 0,
        "endCellLine": 0,
        "endTokenIndex": 2,
        "endCharIndex": 5,
        "layer": 0,
        "counter": 0,
        "labeledBy": "PRELABELED",
        "acceptedByUserId": null,
        "rejectedByUserId": null,
        "originNumber": "0",
        "destinationNumber": "0",
        "type": "SPAN",
        "labelSetItemId": "vGOy0ZKA-2rqK7netKz9I",
        "status": "ACCEPTED",
        "confidenceScore": null,
        "createdAt": "2024-08-05T20:23:47.284Z",
        "updatedAt": "2024-08-05T20:23:47.606Z"
      },
      "hashCode": "ARROW:A-92mMT_WppaawOOBJbjt:1:0:0:0:0:0:0:18:6:0:SPAN:9I-5oYKvnzJRWHgsZrDe_:0:0:0:14:0:0:0:18:6:0:undefined:undefined:SPAN:vGOy0ZKA-2rqK7netKz9I:0:0:0:0:0:0:0:2:5:0:undefined:undefined"
    },
    {
      "labelText": null,
      "id": "1685710596",
      "documentId": "e19c1143-a6ea-490b-ab9f-2a0d7bdafa3e",
      "labeledByUserId": null,
      "startCellIndex": 0,
      "startCellLine": 0,
      "startTokenIndex": 14,
      "startCharIndex": 0,
      "endCellIndex": 0,
      "endCellLine": 0,
      "endTokenIndex": 18,
      "endCharIndex": 6,
      "layer": 0,
      "counter": 0,
      "labeledBy": "PRELABELED",
      "acceptedByUserId": null,
      "rejectedByUserId": null,
      "originNumber": "0",
      "destinationNumber": "0",
      "type": "SPAN",
      "labelSetItemId": "9I-5oYKvnzJRWHgsZrDe_",
      "status": "ACCEPTED",
      "confidenceScore": null,
      "createdAt": "2024-08-05T20:23:47.284Z",
      "updatedAt": "2024-08-05T20:23:47.606Z",
      "hashCode": "SPAN:9I-5oYKvnzJRWHgsZrDe_:0:0:0:14:0:0:0:18:6:0:undefined:undefined"
    }
  ]
}

Example (token-based with character-based labeling)

{
    "sentences": [
        {
            "id": 0,
            "content": "The Little Prince is a novella by French aristocrat , writer , and aviator Antoine de Saint - Exupéry .",
            "tokens": ["The","Little","Prince","is","a","novella","by","French","aristocrat",",","writer",",","and","aviator","Antoine","de","Saint","-","Exupéry","."
            ],
            "labels": [
                {
                    "layer": 2,
                    "sidS": 0,
                    "s": 5,
                    "charS": 0,
                    "sidE": 0,
                    "e": 5,
                    "charE": 4,
                    "l": "dKXDeLxSHz1wZdXvA5yQz",
                    "id": "dKXDeLxSHz1wZdXvA5yQz:2:0:5:0:0:5:4:0",
                    "deleted": false,
                    "labeledBy": "LABELER",
                    "labeledByUserId": 752,
                    "hashCode": "dKXDeLxSHz1wZdXvA5yQz:2:0:0:5:0:0:0:5:4:0:SPAN:undefined:undefined",
                    "documentId": "fc324eb5-3cf4-4a16-baa0-954d1d2e13c8",
                    "comments": []
                }
            ]
        }
    ],
    "labelSets": [
        {
            "labelItems": [
                {
                    "id": "O1T-l9CGbonHyxj0GOtAo",
                    "labelName": "Noun phrase",
                    "parentId": null,
                    "color": "#ff8000"
                },
                {
                    "id": "dKXDeLxSHz1wZdXvA5yQz",
                    "labelName": "NN",
                    "parentId": "O1T-l9CGbonHyxj0GOtAo",
                    "color": "#ff8000"
                },
                {
                    "id": "pWUd_Sa1bAiFe38MzU8OL",
                    "labelName": "NNP",
                    "parentId": "O1T-l9CGbonHyxj0GOtAo",
                    "color": "#ff8000"
                },
                {
                    "id": "6zUYagMuBuYmal8zITzqZ",
                    "labelName": "Verb phrase",
                    "parentId": null,
                    "color": "#df3920"
                },
                {
                    "id": "-8zD8jA9XRKQJBNL5snmp",
                    "labelName": "VBT",
                    "parentId": "6zUYagMuBuYmal8zITzqZ",
                    "color": "#df3920"
                },
                {
                    "id": "dZ9UplPt07D97EmrV5Dpn",
                    "labelName": "VBD",
                    "parentId": "6zUYagMuBuYmal8zITzqZ",
                    "color": "#df3920"
                },
                {
                    "id": "m59PdwZSqx50OY4K58vCw",
                    "labelName": "VBN",
                    "parentId": "6zUYagMuBuYmal8zITzqZ",
                    "color": "#df3920"
                },
                {
                    "id": "BCn1q1clVI9oAje2boyLX",
                    "labelName": "VBI",
                    "parentId": "6zUYagMuBuYmal8zITzqZ",
                    "color": "#df3920"
                },
                {
                    "id": "BwjMomxD2E_UJoCNuy_IL",
                    "labelName": "VB",
                    "parentId": "6zUYagMuBuYmal8zITzqZ",
                    "color": "#df3920"
                }
            ]
        }
    ],
    "labels": [
        {
            "labelText": null,
            "id": "509007771",
            "documentId": "fc324eb5-3cf4-4a16-baa0-954d1d2e13c8",
            "labeledByUserId": 752,
            "startCellIndex": 0,
            "startCellLine": 0,
            "startTokenIndex": 5,
            "startCharIndex": 0,
            "endCellIndex": 0,
            "endCellLine": 0,
            "endTokenIndex": 5,
            "endCharIndex": 4,
            "layer": 2,
            "counter": 0,
            "labeledBy": "LABELER",
            "acceptedByUserId": null,
            "rejectedByUserId": null,
            "originId": null,
            "originNumber": "0",
            "destinationId": null,
            "destinationNumber": "0",
            "type": "SPAN",
            "labelSetItemId": "dKXDeLxSHz1wZdXvA5yQz",
            "status": "ACCEPTED",
            "createdAt": "2021-09-03T09:22:58.456Z",
            "updatedAt": "2021-09-03T09:22:58.512Z"
        }
    ]
}

Example (token-based with bounding-box labeling)

{
  "sentences": [
    {
      "id": 0,
      "content": "73",
      "tokens": ["73"],
      "labels": [
        {
          "layer": 0,
          "sidS": 0,
          "s": 0,
          "charS": 0,
          "sidE": 0,
          "e": 0,
          "charE": 1,
          "l": "fDsCQJFyWy5LnMPtHK4DC",
          "id": "fDsCQJFyWy5LnMPtHK4DC:0:0:0:0:0:0:1:0",
          "deleted": false,
          "labeledBy": "LABELER",
          "labeledByUserId": 1,
          "hashCode": "fDsCQJFyWy5LnMPtHK4DC:0:0:0:0:0:0:0:0:1:0:SPAN:undefined:undefined",
          "documentId": "dac38af2-cfb3-4007-b2ca-302dc8c450fe",
          "comments": []
        }
      ]
    }
  ],
  "labelSets": [
    {
      "labelItems": [
        { "id": "fDsCQJFyWy5LnMPtHK4DC", "labelName": "Queue number" }
      ]
    }
  ],
  "labels": [
    {
      "labeledBy": "PRELABELED",
      "labeledByUserId": null,
      "acceptedByUserId": null,
      "rejectedByUserId": null,
      "type": "BOUNDING_BOX",
      "status": "LABELED",
      "startCellIndex": 0,
      "startCellLine": 0,
      "startTokenIndex": 0,
      "startCharIndex": 0,
      "endCellIndex": 0,
      "endCellLine": 0,
      "endTokenIndex": 0,
      "endCharIndex": 1,
      "layer": 0,
      "counter": 0,
      "pageIndex": 0,
      "x0": 228,
      "y0": 114,
      "x1": 286,
      "y1": 114,
      "x2": 286,
      "y2": 158,
      "x3": 228,
      "y3": 158,
      "nodeCount": 4
    },
    {
      "labelText": null,
      "labeledByUserId": 1,
      "startCellIndex": 0,
      "startCellLine": 0,
      "startTokenIndex": 0,
      "startCharIndex": 0,
      "endCellIndex": 0,
      "endCellLine": 0,
      "endTokenIndex": 0,
      "endCharIndex": 1,
      "layer": 0,
      "counter": 0,
      "labeledBy": "LABELER",
      "acceptedByUserId": null,
      "rejectedByUserId": null,
      "originId": null,
      "originNumber": "0",
      "destinationId": null,
      "destinationNumber": "0",
      "type": "SPAN",
      "labelSetItemId": "fDsCQJFyWy5LnMPtHK4DC",
      "status": "ACCEPTED"
    }
  ],
  "pages": [
    {
      "pageIndex": 0,
      "pageHeight": 619,
      "pageWidth": 551
    }
  ]
}

Datasaur Schema Format

Datasaur Schema is a customized JSON format that is designed to fit all available project types in Datasaur app. This format can be used for mixed project