Label Sets / Question Sets

Span labeling

For span labeling, a label set is a single-column .csv file that follows the structure below.

Column 1

Label 1

Label 2

Label 3

etc...

Color-coded labels

We provide twelve default colors that you can configure during project creation or in the Labels extension.

You can also create a .csv label set file with your desired label colors using HTML color codes. In the sample below, label,color is the header, indicating two columns. This will always be the first row in the .csv file.

label,color
Annabeth Chase,#df3920
Harry Potter,#ff8000
Hermione Granger,#4db34d
John Watson,#3399cc
Percy Jackson,#cc3399
Sherlock Holmes,#9933cc

Colored label sets are only supported in the .csv format.

Limit selection to bottom-level labels only

In hierarchical label sets, some labels act as broad categories while others are more specific. This setting restricts selection to bottom-level labels—those without child labels—to improve precision and consistency.

When to use

Use this setting when your project requires detailed and specific classification. Some examples:

Fruit
- Apple
- Banana
Vegetable
- Carrot
- Spinach

When enabled, labelers can select Apple, Banana, Carrot, and Spinach, but not Fruit or Vegetable.

How to configure

You can enable this setting during project creation, in an existing project, or in the Label Management page.

During project creation

Create new project.
Go to step 3 and select Span labeling.
In the Label set section, click the triple-dot menu on the label set and enable the setting.

Within a project

Open the Labels extension and select a label set.
Click the triple-dot menu and choose one of the following:
1. Add new label set
2. Replace existing label set
3. Edit label set
Expand the Label set settings accordion and enable the setting.

Label management page

Go to the Label management page.
Select Add label set or click Edit icon on an existing label set.
Expand the Label set settings accordion and enable the setting.

Labeling behavior

When this setting is enabled:

Only bottom-level labels can be selected.
Parent labels with child labels are not selectable.
Keyboard shortcuts (numbers, arrow keys, and Enter) apply only to bottom-level labels.

Bounding box labeling

Label sets

You can use .csv, .tsv, or .json formats for bounding box label sets.

For .csv and .tsv files
- We support color names (example: red), hex values (example: #00FF00), or RGB (example: rgb(0,0,255)).
- You can also create a label set with only label names, as shown in the Datasaur sample - Bbox only name.csv file below.
- Other properties, such as captionAllowed and captionRequired, will use default values if not specified.
For .json files
- We support hex values and RGB formats only.

564B

Datasaur sample - Bbox.json

Open

24B

Datasaur sample - Bbox only name.csv

Open

201B

Datasaur sample - Bbox with header.csv

Open

158B

Datasaur sample - Bbox (without header).csv

Open

2KB

Datasaur sample - Bbox (with custom attributes).json

Open

Text caption

The Allow text caption setting allows labelers to add a caption to a bounding box. If disabled, captions cannot be added.

You can require text for specific labels by enabling the Require caption option.

Row/Document labeling

For row labeling and document labeling projects, a question set is a .csv file with questions in the first column and answers in the subsequent columns.

Column 1

Column 2

Column 3

Column 4.

Question 1

Answer 1

Answer 2

Answer 3

Question 2

Answer 1

Answer 2

Question 3

Answer 1

Answer 2

Answer 3

Answer 4

Answer 5

You can also use a .json file to create a question set with multiple question types.

Question hint

You can optionally add a hint to each question. Hints can include instructions or explanations to help labelers provide more accurate answers. Each hint can contain up to 65,000 characters.

You can configure question hints during project creation or in the Label management page.

Supported format

The Document labeling and Row labeling extensions support Markdown syntax in question hints. his allows you to format text, create lists, add links, or emphasize content using standard Markdown.

Formatting

Syntax

Bold

your text

Italic

your text

Underline

• Bullet

dashes (-) or asterisks (*)

1. Numbering

1., 2., 3., etc.

Link

[your text] (https://example.com)

Markdown syntax counts toward the character limit (65,000 characters).

Best practices

Keep hints brief and focused on relevant information. Long hints may appear as large text blocks and clutter the UI. For more complex content, consider including links that labelers can open when needed.

Question types

Here are the available question types for question sets.

1. Text field

Text field allows labelers to answer a question by entering a single line of free-form text.

You can add validation in the More settings accordion.

2. Text area

Text area allows labelers to answer a question by entering multi-line free-form text.

Dropdown requires labelers to answer a question by selecting one answer from a list.

You can upload a .csv file as an answer set for the dropdown options.

178B

Datasaur sample - Question set (book genre).csv

Open

You can also allow multiple selections by enabling Allow multiple answers.

Hierarchical dropdown allows labelers to answer a question by selecting from structures, multi-level options.

You can upload a hierarchical answer set after creating the question with this format.

5. True/False

Previously called Yes/No.

True/False allows labelers to answer a question by checking the checkbox. You can also add a hint.

6. Single choice

Previously called Radio button.

Single choice allows labelers to answer a question by selecting one answer from up to 25 options.

You can also add a hint.

7. Multiple choice

Multiple choice allows labelers to answer a question by selecting several answers from up to 25 options.

8. Date

Date allows labelers to answer a question by selecting a date from a calendar, ensuring valid date input. Labelers can still type it manually if needed.

You can set the current date as the default value by enabling the Use current date as default value setting in step 3 of project creation.

9. Time

Time allows labelers to answer a question by selecting a time from a clock, ensuring valid time input. Labelers can still type it manually if needed.

You can set the current time as the default value by enabling the Use current time as default value setting in step 3 of project creation.

10. Slider

Slider allows labelers to answer a question by selecting a numeric value within a defined range (example: from 1 to 10).

You can hide the value from labelers if needed (it remains visible in reviewer mode).

You can customize the slider color using predefined options or custom values (hex, color name, or RGB).

You can also use the interactive preview to test the slider.

When you change the minimum or maximum value of a slider question, existing answers outside the new range will trigger an error message in the Row labeling or Document labeling extension when users select the affected row or document.

861B

Datasaur sample - Question set (custom slider color).json

Open

11. URL

URL allows labelers to input links with validation.

12. Grouped attributes

Grouped attributes allows multiple related questions to be grouped together.

13. Script-generated questions

Available only for Row labeling projects.

Script-generated questions dynamically generates questions for each row based on its data, allowing more flexible and dynamic workflows. For more details, see this page here.

Advanced settings

In row labeling projects, you can use the Refer answer to table column option in Advanced settings to pre-fill answers based on your dataset. See the full guide for more details.

Answer validation script

This feature is available only in row labeling projects and is disabled by default. Please reach out to [email protected] if your team needs this feature.

The answer validation script is a TypeScript-based feature that validates answers in row labeling tasks. It allows you to define custom validation logic for complex scenarios, such as validating answers using other questions, comparing data across questions, or calling external APIs. If a submission fails validation, an error message is shown.

This feature improves control, accuracy, and consistency in the labeling process.

Key capabilities

Row-specific validation: Validates data based on the current row.
Cross-question validation: Compares answers across multiple questions.
API-based validation: Use external APIs or external business logic for validation.

Configure the validation script

Validation can only be configured after questions are set up. Only admins can access this feature in reviewer mode.

Go to the Row labeling extension in a project.
Click on the three-dot menu.
Select Configure answer validation script.

When you open the Configure answer validation script dialog for the first time, a template is provided.

View Template

/**
 * This function should return a ValidationResult object.
 *
 * @param {ValidationArgs} args - The arguments for the validation.
 * @returns {ValidationResult} The result of the validation.
 * @example
 * To return an error:
 *   return { errorMessage: "This is a sample error message." };
 * To return no error:
 * return {}
 */
async ({ columns, row, questions, answers }: ValidationArgs): Promise<ValidationResult> => {
  **// TODO: Implement your validation logic here.**

  return {};

  /** Helper functions with access to the validation args */

  /**
   * Get the answer for a question by its label.
   * For grouped attributes, pass in the questions array from the grouped attribute.
   * @param label - The label of the question.
   * @param searchQuestions - Optional. The questions to search through. Defaults to the questions in the validation args.
   * @returns The answer for the question.
   */
  function getAnswerByQuestionLabel(label: string, searchQuestions: Question[] = questions) {
    const question = searchQuestions.find((q) => q.label === label);
    if (!question) throw new Error(`Could not find question with label: "${label}".`);
    return answers[`Q${question.id}`];
  }

  /**
   * Get the value of a cell by its column label.
   * @param label - The label of the column.
   * @returns The value of the cell.
   */
  function getCellValueByColumnLabel(label: string) {
    const column = columns.find((column) => column.name === label);
    if (!column) {
      throw new Error(`Couldn't find column with label: "${label}".`);
    }

    const cell = row.find((row) => row.index === column.id);
    if (!cell) {
      throw new Error(`Couldn't find cell with index: ${column.id}. Column: "${column.name}"`);
    }

    return cell.content;
  }
};

To provide feedback when a submission fails, return an object with an errorMessage property (optional). When the submission passes, return an empty object.

async ({ columns, row, questions, answers }: ValidationArgs): Promise<ValidationResult> => {
  // this script will always prevent the labeler to submit the answer.
  return { errorMessage: 'Please double-check your answers.' };
}

When validating, you may need to access certain information to determine whether the answer is valid or needs adjustment before submission. You can access all required data from the function argument, as shown in the example function.

columns: TableColumn[] contains information about the column structure of the data being labeled.

View structure

interface TableColumn {
  id: number;
  name: string;
  displayed: boolean;
  labelerRestricted: boolean;
  rowQuestionId?: number;
}

row: Cell[] is an array of cells containing the data that is being labeled.

View structure

interface Cell {
  line: number;
  index: number;
  content: string;
  tokens: string[];
  metadata?: CellMetadata[];
}

interface CellMetadata {
  key: string;
  value: string;
  type?: string;
  pinned?: boolean;
  config?: TextMetadataConfig;
}

interface TextMetadataConfig {
  backgroundColor: string;
  color: string;
  borderColor: string;
}

questions: Question[] is an array of questions in the project.

View structure

enum QuestionType {
  DROPDOWN = 'DROPDOWN',
  HIERARCHICAL_DROPDOWN = 'HIERARCHICAL_DROPDOWN',
  NESTED = 'NESTED',
  TEXT = 'TEXT',
  SLIDER = 'SLIDER',
  DATA = 'DATA',
  DATE = 'DATE',
  TIME = 'TIME',
  CHECKBOX = 'CHECKBOX',
  URL = 'URL',
}

enum SliderTheme {
  PLAIN = 'PLAIN',
  GRADIENT = 'GRADIENT',
}

interface Question {
  id: number;
  label: string;
  type: QuestionType;
  required: boolean;

  activationConditionLogic?: string;
  bindToColumn?: string;

  config: QuestionConfig;
}

interface QuestionConfig {
  multiple?: boolean;

  // TEXT
  maxLength?: number;
  minLength?: number;
  pattern?: string;
  multiline?: boolean;

  // SLIDER
  theme?: SliderTheme;
  min?: number;
  max?: number;
  step?: number;

  // DATE TIME
  format: string;
  defaultValue?: string;

  // DROPDOWN HIERARCHICAL
  options?: Array<{ id: string; label: string; parentId?: string | null }>;

  // GROUPED
  questions?: Array<Question>;

  // CHECKBOX
  hint?: string;
}

answers is an object where each key is a question ID and each value is its corresponding answer. Depending on the question type, the value can take one of these four formats:
multiple: false
multiple: true
normal question
string
string[]
grouped attributes
Answer
Answer[]

View structure

type Answer = string | string[] | Answers[];

interface Answers {
  [questionId: string]: Answer;
}

We provide helper functions in the template to simplify common data access tasks, such as:

function getCellValueByColumnLabel(label: string): string;
This function returns the value of a cell based on the column label.
function getAnswerByQuestionLabel(label: string, searchQuestions: Question[] = questions): Answers;
This function returns the answer based on the question label.

Validating answer through an API call

You can include API requests in your validation logic using the Fetch API, enabling dynamic or third-party validations.

Disclaimer

We are not responsible for API calls that are misrouted, misconfigured, or sent to unintended parties, which may result in data exposure or compromise.
You are responsible for ensuring the accuracy, security, and integrity of your API configurations and transmissions. By using this feature, you acknowledge and accept these responsibilities.

Examples

Validating answers between two questions

async ({ questions, answers }: ValidationArgs): Promise<ValidationResult> => {
  const questionAAnswer = getAnswerByQuestionLabel("Question A");
  const questionBAnswer = getAnswerByQuestionLabel("Question B");

  if (questionAAnswer !== questionBAnswer) {
    return { errorMessage: "The answer to Question A must match Question B." };
  }

  return {}; // Pass the validation
  
  // ...existing helpers provided by template
};

Validating an answer based on a cell value

async ({ columns, row, questions, answers }: ValidationArgs): Promise<ValidationResult> => {
  const statusColumnValue = getCellValueByColumnLabel("Status");
  const approvalStatusAnswer = getAnswerByQuestionLabel("Approval Status");

  if (statusColumnValue === "Complete" && approvalStatusAnswer !== "Approved") {
    return { errorMessage: "If 'Status' is 'Complete', 'Approval Status' must be 'Approved'." };
  }

  return {};
  
  // ...existing helpers provided by template
};

Validating an answer through an API request using the Fetch API

async ({ columns, row, questions, answers }: ValidationArgs): Promise<ValidationResult> => {
	const response = await fetch('<https://some.api.net/validate>', ...);
	const data = await response.json(); // assume the API returns a JSON object: { result: 'valid|invalid', message: 'the error message' };
	
	if (data.result === 'invalid') {
	  return { errorMessage: data.message };
	}

  return {};
  
  // ...existing helpers provided by template
}

FAQs

Can I validate across multiple rows?
No, the validation script is row-specific and operates on each row individually.
What happens if there is an error in the script?
Unhandled errors or exceptions will trigger a validation error and prevent submission. You can handle errors within the script to allow submission if needed.

You can upload multi-level hierarchical label sets for span labeling projects, and hierarchical dropdown options for row labeling or document labeling projects.

The following example shows a supported .csv format:

148B

Datasaur sample - Hierarchical label set (dropdown options).csv

Open

id,label
1,Novel
1.1,Author
1.1.1,Name
1.1.2,Works
1.2,Title
1.2.1,Main Title
1.2.2,Subtitle
2,Characters
2.1,Antagonist
2.2,Protagonist

File structure

1. Header

The header id,label will always be the first row in the .csv file. The first label or option should have 1 as the ID, just like in the example above.

2. ID format

IDs follow a hierarchical numbering format similar to Microsoft Word:

Novel is the root level with ID 1.
Author is a second-level category under Novel with ID 1.1.
Name is a third-level item under Author with ID 1.1.1

Important notes

When importing data, the CSV format uses dots (.) to represent hierarchical relationships. However, these dots are automatically converted into a different ID structure in the JSON format because dots are reserved for path traversal operations in the system. This means dots must not be used in JSON IDs. Here's how it works:

CSV Input:
```
id,label
1,Novel
1.1,Author
```

Will be converted to JSON as:

[  
  {    
    "label": "Novel",    
    "id": "1"  
  },  
  {    
    "label": "Author",    
    "id": "2",    
    "parentId": "1"  
  }
]

In JSON format:

✅ Correct: "id": "2"
❌ Incorrect: "id": "1.1"

Using dots for JSON IDs will cause incorrect path resolution when selecting items in the hierarchy.

3. Hierarchical label set in span labeling projects

The hierarchy will be visible in the Labels extension and the label box. You can also use the same label name under different parent labels.

id,label
1,Software
1.1,Java
2,Geography
2.1,Java

In the example above, although Java appears twice, each instance belongs to a different parent, making it contextually unique.

Using the same label name more than once under the same parent is not allowed. In the example below, the system flags an error because both Apple entries are under Fruit.

id,label
1,Fruit
1.1,Apple
1.2,Apple

Choose Hierarchical dropdown as the question type when creating the project. The hierarchy will be displayed in the Row labeling and Document labeling extension and the answer column in the table.

In the dropdown menu:

Clicking the Home icon navigates to the top-level label.
You can search for bottom-level options globally.

Last updated 14 days ago

hashtagSpan labeling

hashtagColor-coded labels

hashtagLimit selection to bottom-level labels only

hashtagWhen to use

hashtagHow to configure

hashtagLabeling behavior

hashtagBounding box labeling

hashtagLabel sets

hashtagText caption

hashtagRow/Document labeling

hashtagQuestion hint

hashtagSupported format

hashtagBest practices

hashtagQuestion types

hashtag1. Text field

hashtag2. Text area

hashtag3. Dropdown

hashtag4. Hierarchical dropdown

hashtag5. True/False

hashtag6. Single choice

hashtag7. Multiple choice

hashtag8. Date

hashtag9. Time

hashtag10. Slider

hashtag11. URL

hashtag12. Grouped attributes

hashtag13. Script-generated questions

hashtagAdvanced settings

hashtagAnswer validation script

hashtagKey capabilities

hashtagConfigure the validation script

hashtagValidating answer through an API call

hashtagExamples

hashtagFAQs

hashtagHierarchical label sets or dropdown options

hashtagFile structure

hashtag1. Header

hashtag2. ID format

hashtag3. Hierarchical label set in span labeling projects

hashtag4. Hierarchical dropdown options in row or document labeling projects

Span labeling

Color-coded labels

Limit selection to bottom-level labels only

When to use

How to configure

Labeling behavior

Bounding box labeling

Label sets

Text caption

Row/Document labeling

Question hint

Supported format

Best practices

Question types

1. Text field

2. Text area

3. Dropdown

4. Hierarchical dropdown

5. True/False

6. Single choice

7. Multiple choice

8. Date

9. Time

10. Slider

11. URL

12. Grouped attributes

13. Script-generated questions

Advanced settings

Answer validation script

Key capabilities

Configure the validation script

Validating answer through an API call

Examples

FAQs

Hierarchical label sets or dropdown options

File structure

1. Header

2. ID format

3. Hierarchical label set in span labeling projects

4. Hierarchical dropdown options in row or document labeling projects