# Bounding Box Labeling

## Bounding box projects

This guide shows how to create a bounding box project and how to draw and label bounding boxes on images, PDFs, or documents. You can also [watch this 2:00 minute Youtube video](https://youtu.be/5j4gV3tu1Ps) for a walkthrough.

## Interface legend

If you already have a project, use this legend to understand the basics of the labeling interface. Also, see [this :43 second Youtube video](https://youtu.be/H1B_G_t5bXc) to learn how to draw and label bounding boxes.

<figure><img src="/files/iOE6Wae3lg1gTuI29dUQ" alt=""><figcaption></figcaption></figure>

## Create a project

To create a bounding box labeling project:

1. Go to the **Projects** page and click **Create project.**

   <figure><img src="/files/Ua46AdtlhYXzqkwRI7wP" alt=""><figcaption></figcaption></figure>
2. Upload your image or document file. Supported file types:
   1. **Images:** `.bmp`, `.gif`, `.jpeg`, `.jpg`, `.png`, `.svg`, `.tiff`, `.tif`, `.webp`
   2. **Document**: `.pdf`

Here is an example:

{% file src="/files/AVeUflb9SqHgUTpk3e7G" %}

<figure><img src="/files/Qj8F0zrpGhMbsMfllV9Q" alt=""><figcaption></figcaption></figure>

3. In step 2, set **Apply OCR Method** to **None**.
4. In step 3, select **Bounding box labeling** and add a label set.
   1. Supported formats: `.csv`, `.tsv`, or `.json`
      1. For `.csv` and `.tsv`, we support **color names** (example: `red`), **hex values** (example: `#00FF00`), and **RGB** (example: `rgb(0,0,255)`). You can also provide label names only. Other fields such as `captionAllowed` and `captionRequired` will use default values.
      2. For `.json`, we support **hex** and **RGB** only.

{% file src="/files/88e4WXaWyWBV7b9u2NRd" %}

{% file src="/files/TXH3IJwCMJuq4vhBKCiH" %}

{% file src="/files/IDatnggwXcViBmOVnXxs" %}

{% file src="/files/iG5wYdtJYSKSIkIjYr1E" %}

{% file src="/files/idpJJxOEejnkoMFE4BO3" %}

5. Configure the **Allow text caption** setting. It allows labelers to add text to a bounding box.

   <figure><img src="/files/EIxXJSBAuurvw4ucFfGJ" alt=""><figcaption></figcaption></figure>

   1. You can type them manually or use an OCR provider.
   2. You can also require text for specific labels by enabling the **Require caption** option.

      <figure><img src="/files/zd9BTvrwW3IyGQeECt5X" alt=""><figcaption></figcaption></figure>

Finish the remaining steps as you would for other project types: **Assignment** and **Project settings**.

## Upload pre-labeled files

You can upload pre-labeled files in a bounding box labeling project by uploading a JSON file together with the document you want to label. he JSON file must follow the [Datasaur Schema](/compatibility-and-updates/supported-formats.md#datasaur-schema-format) format.

## Label sets capabilities

### Label set modification

You can update the label set at any time:

* **Add a label set**: Drag and drop a file, or use the three-dot menu.

  <figure><img src="/files/lpErsy021nLoNBjpQc4x" alt=""><figcaption></figcaption></figure>
* **Edit a label set**: Open the three-dot menu and select **Edit label set.**

  <figure><img src="/files/lBxi0XfQoUbZuhggF9mJ" alt=""><figcaption></figcaption></figure>

  <figure><img src="/files/u9LzghlOMqTfzyi90Zik" alt=""><figcaption></figcaption></figure>
* **Replace a label set**: Open the three-dot menu and select **Replace existing label set**, then create a new one or upload a new file.

  <figure><img src="/files/ek6kl4bTQSawhwLomOy8" alt=""><figcaption></figcaption></figure>

  <figure><img src="/files/0Df2Ej6twutHjPDaGwC3" alt=""><figcaption></figcaption></figure>

By default, both reviewers and labelers can perform these actions. To restrict access, go to **File** > **Settings** > **Administrator** and disable this option.

<figure><img src="/files/blRXR8jvbxDn4DWH9wZB" alt=""><figcaption></figcaption></figure>

### Custom attributes for label classes

Each label class in the label set can have multiple custom attributes (questions). The supported question types are [Text](https://docs.datasaur.ai/data-studio-projects/lets-get-labeling/pages/sjMjHjQsHtR5BrPf3PUF#id-1.-text-field) and [Dropdown](https://docs.datasaur.ai/data-studio-projects/lets-get-labeling/pages/sjMjHjQsHtR5BrPf3PUF#id-3.-dropdown). Each question can also have a default answer.

To modify the custom attributes:

1. Open the label set editor via **Add new label set**, **Edit label set**, or **Replace existing label set**.

   ![](/files/msIU413fzvgu4B4kZIqB)
2. Go to the **JSON** tab.

   ![](/files/nNzfxyHjIcP39ByYbeWZ)
3. Add or modify the <mark style="color:red;">`questions`</mark> field as needed for each label class. You can't modify the <mark style="color:red;">`type`</mark> and <mark style="color:red;">`internalId`</mark> of existing questions, as it may affect existing labels. Question structure:

   1. **id**: Unique question ID.
   2. **label**: Question name.
   3. **type**: <mark style="color:red;">`TEXT`</mark> or <mark style="color:red;">`DROPDOWN`</mark>.
   4. **config**: Question settings
      1. **required**: Determines whether the question must be answered. It can be <mark style="color:red;">`true`</mark> or <mark style="color:red;">`false`</mark>.
      2. **defaultValue**: The default answer for the question
         1. For <mark style="color:red;">`TEXT`</mark> questions, it can be any text.
         2. For <mark style="color:red;">`DROPDOWN`</mark> questions, it must match an <mark style="color:red;">`options.id`</mark>.
      3. **multiline**: Specifies if the <mark style="color:red;">`TEXT`</mark> question allows multiple lines. It can be <mark style="color:red;">`true`</mark> or <mark style="color:red;">`false`</mark>.
      4. **options**: An array of options for the <mark style="color:red;">`DROPDOWN`</mark> question, each containing:
         1. **id**: ID of the dropdown option. It can be used in the <mark style="color:red;">`defaultValue`</mark>.
         2. **label**: The text label for the dropdown option.
   5. **internalId**: Read-only unique ID used by the system.

   For a full example, refer to the [Label Sets](#label-sets) section to see a sample bounding box label set with custom attributes.

## Create bounding boxes

In bounding box labeling projects, the document viewer displays the uploaded documents or images and allows you to create bounding boxes.

There are two cursor types for creating bounding boxes: **Bounding box** and **Text selection**.

### Bounding box

Use **Bounding box** cursor to draw a bounding box directly on the document. This method gives you precise control over placement.

<figure><img src="/files/VxtzWUDYnPeNIV76MhsL" alt=""><figcaption></figcaption></figure>

### Text selection

![](/files/mwPY7Gem2lAnpOjQcVng)

Use **Text selection** to create bounding boxes by selecting text instead of drawing manually. This is faster when working with text-based documents.

{% hint style="info" %}

* Available only for PDF documents.
* Works only with PDFs that contain selectable text (text layers).
* PDFs created from tools like Microsoft Word usually contain text layers. Scanned or image-based PDFs typically do not.
  {% endhint %}

To use **Text selection** mode:

1. Enable **Text selection** from the toolbar.
2. Select text by clicking and dragging, or double-click a word.

![](/files/FtSXS9k8eWa3JTO7bTbo)

3. When you release the mouse, a bounding box is created automatically. A label dialog appears so you can assign a label. The selected text is added to the caption field by default.

![](/files/xLOnzaJdD8meUsg0ZKyb)

## Apply labels, captions, and custom attributes

After drawing a bounding box, a modal appears with the following fields:

* Label,
* Caption,
* Custom attributes (if configured).

Select a label, enter a caption, and answer any attributes as needed. To submit, click anywhere in the document viewer.

<figure><img src="/files/tb2AOzXgcp8yWQRY8YHf" alt=""><figcaption></figcaption></figure>

To review a bounding box, hover over it to see a tooltip with its values.

<figure><img src="/files/zQpGFPxtzJ29PTAup5AI" alt=""><figcaption></figcaption></figure>

### Automatically fill the caption using OCR

You can automatically fill captions using OCR (Tesseract). When you draw a bounding box around text, the detected text is added to the **Caption** field.

{% hint style="info" %}
Processing may take longer for larger text selections.
{% endhint %}

To enable this setting:

1. In step 3 of project creation, enable **Allow text caption**.
2. Select **Tesseract** from the **OCR provider** dropdown.

<figure><img src="/files/wY23jlLFVrbz730L0UNh" alt=""><figcaption></figcaption></figure>

After creating the project, you can enable or disable auto-caption from the three-dot menu in the **Bounding box labels** extension. Changes apply to all documents.

<figure><img src="/files/PXJ4RSZ6BZ5Dsgpn0QZg" alt=""><figcaption></figcaption></figure>

**Notes on auto-caption:**

1. It runs only when drawing a bounding box (not after it is created).
2. It runs again if the box is resized while still in the drawing state.
3. It can be canceled using the **(x)** in the **Caption** field.

   ![](/files/akxY7OS7zD8ev8KAvt6A)
4. It stops if you click outside the modal before the process is finished. In this case:
   1. It returns an empty value if the caption is optional.
   2. It shows an error if the caption is required.
5. It supports multiple bounding boxes. Captions are separated by line breaks.

## Bounding box arrows / relationships

Bounding box relationships allow you to define connections (arrows) between bounding boxes within the same document. This is useful for capturing structural or semantic links such as key–value pairs, question–answer pairs, or grouped elements.

In many cases, related information is captured as separate bounding boxes but needs to be understood together. For example:

* A field label like “Invoice Number” is associated with its corresponding value.
* A “Total Amount” label is linked to the numeric value shown elsewhere on the page.
* Multiple cells in a row belong to the same table record.

Bounding box relationships enable you to explicitly define these connections, making the data more structured and easier to use for downstream extraction or model training.

<figure><img src="/files/9l6QJlTW0cC0IMJA7d73" alt=""><figcaption></figcaption></figure>

**Creating a relationship** — To create a relationship between bounding boxes:

1. Select the **arrow** cursor type at the top of the document viewer.
2. Click the origin bounding box, then click the destination bounding box.
3. (Optional) To add a label, click the arrow and select a label.

**Deleting relationships** — To remove a relationship:

1. Right-click on an arrow to open the context menu.
2. Click **Delete.**

{% hint style="info" %}

* Relationships can only be created between bounding boxes within the same document.
* Deleting a bounding box will also remove its associated relationships.
  {% endhint %}

## Single vertex edit

A bounding box is initially drawn as a rectangle. To improve accuracy for document parts that are not perfectly square, you can adjust individual cornet points by holding the `Alt` key (or the `Option` key on Mac) and dragging a corner of the box.

## Move and edit bounding boxes on hover

The **Show resize handles on hover** setting allows you to edit bounding boxes by hovering over a box instead of clicking it first. Only admins can configure this setting.

![](/files/ViFFaPxIikGvanaHAZf7)

### Enable the setting

1. During project creation, go to **Step 5** and enable the **Show resize handles on hover** setting.

   ![](/files/lNWdmkZ9srLHEvkZHumD)
2. To enable it for an existing project, go to **File** → **Settings** → **Administrator** and enable the **Show resize handles on hover** setting.

   ![](/files/blRXR8jvbxDn4DWH9wZB)

## Manage bounding boxes

### Copy and paste

{% hint style="info" %}
Bounding boxes can only be copied and pasted within the same document.
{% endhint %}

You can copy and paste bounding boxes in two ways:

* Right-click a bounding box, and select **Copy** or **Paste**.
* Select a bounding box, then press `Ctrl`+`C` / `Ctrl`+`V` (or `Command`+`C` and `Command`+`V` on Mac).

### Delete

You can delete bounding boxes in two ways:

* Right-click a bounding box and select **Delete.**
* Select a bounding box, then press `Del`.

### Multiple selection

Multiple selection is helpful for copying, pasting, or deleting multiple bounding boxes. To select multiple bounding boxes, hold `Ctrl` while selecting each one.

### Merge multiple bounding boxes

You can merge multiple bounding boxes into a single label with multiple connected boxes. The result depends on the label classes:

* If all selected bounding boxes have the same label class, they are merged under that label class.
* If they have different label classes, the label box appears so you can choose a label class and update the caption before saving.

The merged caption is determined by the selection order. For example, if the first box is `Merchant` and the second is `name`, the result will be `Merchant name`.

To merge bounding boxes:

1. Select multiple bounding box.
2. Right-click one of the selected boxes and select **Merge.**

### Manage visibility: Show and hide <a href="#show-and-hide" id="show-and-hide"></a>

When working with datasets containing many overlapping bounding boxes, the view can become cluttered. You can manage visibility by hiding or showing bounding boxes by label class or individually, helping you focus on specific areas.

#### **Hide and show bounding boxes**

You can hide and show bounding boxes in three ways:

1. **By label class**
   1. Go to the **Bounding box labels** extension.
   2. For each label class, there's an eye icon located next to the label instance counter.

      <div data-full-width="true"><img src="/files/SHCqMTdq8yBJIADfk5Gp" alt=""></div>
   3. Click the eye icon to hide all bounding boxes associated with that label class. Click it again to show them.
   4. In the extension, you can also see the number of bounding boxes that are currently shown and hidden.
2. **From document viewer, individually**
   1. Right-click on a bounding box.
   2. Select **Hide**.

      ![](/files/uJVV37odhMLclbH3ElLI)
3. **From extension, individually**

   1. Navigate to the **Bounding box labels** extension.
   2. Click the instance counter for a label class. It will show all boxes associated with that label class.
   3. Click the eye icon to hide the bounding box. Click it again to show them.

   <div data-full-width="true"><img src="/files/34LpilXWhM9ItVZAyZSH" alt=""></div>

   <div data-full-width="true"><img src="/files/12IPhK6RzqHGzIc3O3SQ" alt=""></div>

{% hint style="info" %}

* If a bounding box is added or updated after you hide it, it will still be visible. This ensures that new or updated data is always shown for review.
* Refreshing the page resets all visibility filters and shows all bounding boxes.
  {% endhint %}

## Reviewer capabilities overview

#### Capabilities

* View all labelers' work.
  * Note: Bounding box labels applied in similar positions may overlap.
* Apply labels.
* Update labels applied by labelers.

#### Limitations

* Cannot accept or reject labels.
* Cannot view all labelers' work with information on who applied each label.
* When updating a label applied by a labeler, there is no information or indicator showing that the label has been changed.

## Upload pre-labeled files

In bounding box labeling projects, you can upload pre-labeled files. Learn more about creating pre-labeled bounding box labeling projects in [this documentation](https://docs.datasaur.ai/data-studio-projects/pre-labeled-project#bounding-box-labeling).


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.datasaur.ai/data-studio-projects/lets-get-labeling/bounding-box-labeling.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.