Common Terminology
A glossary of words being used in the app, ensuring the same level of understanding for all users.
Last updated
A glossary of words being used in the app, ensuring the same level of understanding for all users.
Last updated
Labels or answers that are applied in:
Labeler Mode: This includes manually applied labels, assisted labeling features, and pre-labeled data.
Reviewer Mode: This includes all labels or answers reviewed manually, applied through assisted labeling features, and pre-labeled data, but excludes those automatically accepted through consensus.
Pre-labeled Data: Refers to the labels or answers that are already present in the input file by default when a project is first created.
Assisted Labeling Features: Search (bulk label application towards the search result), ML-assisted Labeling, Data Programming, and Predictive Labeling.
Span Labeling: The total number of labels assigned to spans.
Row or Document Labeling: The total number of answers.
Bounding Box Labeling: The total number of bounding boxes created.
Conversational Labeling: The total number of labels assigned to spans, including those assigned to each utterance.
Mixed Labeling: The sum of all labels or answers based on the above calculations.
A Cell is a box that is used to display data in the Editor. For example, in the above picture, the box that contains the text "Sherlock Holmes become widely popular in 1891" is a Cell. Cells are structured in a matrix-like manner.
Cell's line is its position relative to the vertical axis, it's numbered from 0 starting from top to bottom.
Cell's index is its position relative to the horizontal axis, it's numbered from 0 starting from left to right. We refer to a Cell by using its Line and Index. For example, Cell with Line equals 3 and Index equals 0, is the Cell that contains the text "All but one are set in the Victorian or Edwardian eras, between about 1880 and 1914."
Typically, the atomic unit in a document. This can be a single word but can also refer to punctuation such as '.'.
Spans are indexed starting from 0 within each cell, and its characters are indexed starting from 0 as well. For example, the Span "popularity" has index 1 in the current Cell and the character "u" has index 3 in "popularity".
The level of agreement required among labelers before a label is automatically accepted. This mechanism ensures that multiple labelers reach a mutual agreement on the label/answer.
For example, if the consensus is set at 2, and Labeler 1 (L1) labels a span as a PERSON, it will not be automatically accepted since it hasn't met the minimum requirement of two agreements. However, if Labeler 2 (L2) labels the same span as a PERSON, thereby meeting the consensus requirement, the label will be automatically accepted.
Label Set: A collection of related label classes. In the example above, "NER" is the name of the Label Set.
Label Class: The specific labels used for annotation. In the example above, "GEO," "ORG," "PER," and "GPE" are label classes.
As the name suggests, the main goal of a labeler is to label all types of data across assigned projects. A labeler can be defined in two contexts: as a role in a project or as a role in a Workspace.
Project Level: Anyone can be assigned as a labeler for a project. See the Labeler Mode explanation below.
Workspace Level: A user assigned as a labeler in a Workspace has limited access to various features within the Workspace and is primarily granted access to the assigned projects only.
It's an isolated version of a project where individual team members can work independently. This approach allows labelers to work on their own set of documents, which are later consolidated in Reviewer mode if any conflicts arise.
Each labeler has their own mode, which is basically having their own version of each assigned document.
The reviewers, on the other hand, have a single shared cabinet.
For example, in a project with ten labelers and three reviewers, there will be ten Labeler Modes and only one Reviewer Mode.
A project represents the highly customized configuration of labeling tasks which can consist of multiple documents to be labeled.
As the name suggests, the main goal of a reviewer is to review all labelers' work and manage conflicts. The reviewer's decisions will be used as the final result of the labeling process. Same as a labeler, reviewer can be defined in two contexts: as a role in a project or as a role in a Workspace.
Project Level: Anyone can be assigned as a reviewer for a project. See the Reviewer Mode explanation above.
Workspace Level: A user assigned as a reviewer in a Workspace can have access to various things within the Workspace. For a more detailed scope, see this page.
