Datasaur
Visit our websitePricingBlogPlaygroundAPI Docs
  • Welcome to Datasaur
    • Getting started with Datasaur
  • Data Studio Projects
    • Labeling Task Types
      • Span Based
        • OCR Labeling
        • Audio Project
      • Row Based
      • Document Based
      • Bounding Box
      • Conversational
      • Mixed Labeling
      • Project Templates
        • Test Project
    • Creating a Project
      • Data Formats
      • Data Samples
      • Split Files
      • Consensus
      • Dynamic Review Capabilities
    • Pre-Labeled Project
    • Let's Get Labeling!
      • Span Based
        • Span + Line Labeling
      • Row & Document Based
      • Bounding Box Labeling
      • Conversational Labeling
      • Label Sets / Question Sets
        • Dynamic Question Set
      • Multiple Label Sets
    • Reviewing Projects
      • Review Sampling
    • Adding Documents to an Ongoing Project
    • Export Project
  • LLM Projects
    • LLM Labs Introduction
    • Sandbox
      • Direct Access LLMs
      • File Attachment
      • Conversational Prompt
    • Deployment
      • Deployment API
    • Knowledge base
      • External Object Storage
      • File Properties
    • Models
      • Amazon SageMaker JumpStart
      • Amazon Bedrock
      • Open AI
      • Azure OpenAI
      • Vertex AI
      • Custom model
      • Fine-tuning
      • LLM Comparison Table
    • Evaluation
      • Automated Evaluation
        • Multi-application evaluation
        • Custom metrics
      • Ranking (RLHF)
      • Rating
      • Performance Monitoring
    • Dataset
    • Pricing Plan
  • Workspace Management
    • Workspace
    • Role & Permission
    • Analytics
      • Inter-Annotator Agreement (IAA)
        • Cohen's Kappa Calculation
        • Krippendorff's Alpha Calculation
      • Custom Report Builder
      • Project Report
      • Evaluation Metrics
    • Activity
    • File Transformer
      • Import Transformer
      • Export Transformer
      • Upload File Transformer
      • Running File Transformer
    • Label Management
      • Label Set Management
      • Question Set Management
    • Project Management
      • Self-Assignment
        • Self-Unassign
      • Transfer Assignment Ownership
      • Reset Labeling Work
      • Mark Document as Complete
      • Project Status Workflow
        • Read-only Mode
      • Comment Feature
      • Archive Project
    • Automation
      • Action: Create Projects
  • Assisted Labeling
    • ML Assisted Labeling
      • Amazon Comprehend
      • Amazon SageMaker
      • Azure ML
      • CoreNLP NER
      • CoreNLP POS
      • Custom API
      • FewNERD
      • Google Vertex AI
      • Hugging Face
      • LLM Assisted Labeling
        • Prompt Examples
        • Custom Provider
      • LLM Labs (beta)
      • NLTK
      • Sentiment Analysis
      • spaCy
      • SparkNLP NER
      • SparkNLP POS
    • Data Programming
      • Example of Labeling Functions
      • Labeling Function Analysis
      • Inter-Annotator Agreement for Data Programming
    • Predictive Labeling
  • Assisted Review
    • Label Error Detection
  • Building Your Own Model
    • Datasaur Dinamic
      • Datasaur Dinamic with Hugging Face
      • Datasaur Dinamic with Amazon SageMaker Autopilot
  • Advanced
    • Script-Generated Question
    • Shortcuts
    • Extensions
      • Labels
      • Review
      • Document and Row Labeling
      • Bounding Box Labels
      • List of Files
      • Comments
      • Analytics
      • Dictionary
      • Search
      • Labeling Guidelines
      • Metadata
      • Grammar Checker
      • ML Assisted Labeling
      • Data Programming
      • Datasaur Dinamic
      • Predictive Labeling
      • Label Error Detection
      • LLM Sandbox
    • Tokenizers
  • Integrations
    • External Object Storage
      • AWS S3
        • With IRSA
      • Google Cloud Storage
      • Azure Blob Storage
      • Dropbox
    • SAML
      • Okta
      • Microsoft Entra ID
    • SCIM
      • Okta
      • Microsoft Entra ID
    • Webhook Notifications
      • Webhook Signature
      • Events
      • Custom Headers
    • Robosaur
      • Commands
        • Create Projects
        • Apply Project Tags
        • Export Projects
        • Generate Time Per Task Report
        • Split Document
      • Storage Options
  • API
    • Datasaur APIs
    • Credentials
    • Create Project
      • New mutation (createProject)
      • Python Script Example
    • Adding Documents
    • Labeling
      • Create Label Set
      • Add Label Sets into Existing Project
      • Get List of Label Sets in a Project
      • Add Label Set Item into Project's Label Set
      • Programmatic API Labeling
      • Inserting Span and Arrow Label into Document
    • Export Project
      • Custom Webhook
    • Get Data
      • Get List of Projects
      • Get Document Information
      • Get List of Tags
      • Get Cabinet
      • Export Team Overview
      • Check Job
    • Custom OCR
      • Importable Format
    • Custom ASR
    • Run ML-Assisted Labeling
  • Security and Compliance
    • Security and Compliance
      • 2FA
  • Compatibility & Updates
    • Common Terminology
    • Recommended Machine Specifications
    • Supported Formats
    • Supported Languages
    • Release Notes
      • Version 6
        • 6.112.0
        • 6.111.0
        • 6.110.0
        • 6.109.0
        • 6.108.0
        • 6.107.0
        • 6.106.0
        • 6.105.0
        • 6.104.0
        • 6.103.0
        • 6.102.0
        • 6.101.0
        • 6.100.0
        • 6.99.0
        • 6.98.0
        • 6.97.0
        • 6.96.0
        • 6.95.0
        • 6.94.0
        • 6.93.0
        • 6.92.0
        • 6.91.0
        • 6.90.0
        • 6.89.0
        • 6.88.0
        • 6.87.0
        • 6.86.0
        • 6.85.0
        • 6.84.0
        • 6.83.0
        • 6.82.0
        • 6.81.0
        • 6.80.0
        • 6.79.0
        • 6.78.0
        • 6.77.0
        • 6.76.0
        • 6.75.0
        • 6.74.0
        • 6.73.0
        • 6.72.0
        • 6.71.0
        • 6.70.0
        • 6.69.0
        • 6.68.0
        • 6.67.0
        • 6.66.0
        • 6.65.0
        • 6.64.0
        • 6.63.0
        • 6.62.0
        • 6.61.0
        • 6.60.0
        • 6.59.0
        • 6.58.0
        • 6.57.0
        • 6.56.0
        • 6.55.0
        • 6.54.0
        • 6.53.0
        • 6.52.0
        • 6.51.0
        • 6.50.0
        • 6.49.0
        • 6.48.0
        • 6.47.0
        • 6.46.0
        • 6.45.0
        • 6.44.0
        • 6.43.0
        • 6.42.0
        • 6.41.0
        • 6.40.0
        • 6.39.0
        • 6.38.0
        • 6.37.0
        • 6.36.0
        • 6.35.0
        • 6.34.0
        • 6.33.0
        • 6.32.0
        • 6.31.0
        • 6.30.0
        • 6.29.0
        • 6.28.0
        • 6.27.0
        • 6.26.0
        • 6.25.0
        • 6.24.0
        • 6.23.0
        • 6.22.0
        • 6.21.0
        • 6.20.0
        • 6.19.0
        • 6.18.0
        • 6.17.0
        • 6.16.0
        • 6.15.0
        • 6.14.0
        • 6.13.0
        • 6.12.0
        • 6.11.0
        • 6.10.0
        • 6.9.0
        • 6.8.0
        • 6.7.0
        • 6.6.0
        • 6.5.0
        • 6.4.0
        • 6.3.0
        • 6.2.0
        • 6.1.0
        • 6.0.0
      • Version 5
        • 5.63.0
        • 5.62.0
        • 5.61.0
        • 5.60.0
  • Deployment
    • Self-Hosted
      • AWS Marketplace
        • Data Studio
        • LLM Labs
Powered by GitBook
On this page
  • Bounding Box Project
  • Interface Legend
  • Project Creation
  • Upload Pre-labeled Files
  • Label Sets Capabilities
  • Label Set Modification
  • Custom Attributes for Label Classes
  • Creating Bounding Box(es)
  • Bounding box
  • Text selection
  • Applying Label, Caption and Custom Attributes
  • Automatically fill the caption using OCR
  • Single Vertex Editing
  • Directly Able to Move and Edit a Bounding Box Upon Hover
  • How and Where to Enable
  • Managing Bounding Boxes
  • Copy and Paste
  • Delete
  • Multiple Selection
  • Merging Multiple Bounding Boxes
  • Managing Visibility: Show and Hide
  • Reviewer Capabilities Overview
  • Upload Pre-labeled Files
  1. Data Studio Projects
  2. Let's Get Labeling!

Bounding Box Labeling

Last updated 3 months ago

Bounding Box Project

The following guide will show you how to create a bounding box project and how to draw and label bounding boxes on your PDF/image/document. As a complimentary guide, on how to create Bounding Box project.

Interface Legend

If you have already your project, use this legend for understand the basics of the Bounding Box UI. Also, see on the basics of drawing and labeling a bounding box!

Project Creation

You can create a Bounding Box Labeling project in Datasaur. It is pretty simple, you need to prepare the image or document file for this project. From your project home page, open "Create a Custom Project." In order to begin, you should upload the supported image or document file.

The type of Image files we accept: .bmp, .gif, .jpeg, .jpg, .png, .svg, .tiff, .tif, .webp

The type of document files we accept: .pdf

Here is an example:

So now our window should look like this,

For the Step 2. Preview please make sure you select None for the Apply OCR Method under the preview settings. Afterward, we can move on to the next steps, under Step 3. Labeler’s tasks, you can add a label set by ticking the Bounding Box labeling checkbox. Here’s what the settings will do for your project:

Label Sets

You can utilize .csv, .tsv, or .json formats for the bounding box label set.

  • For .csv/.tsv, we support color names (e.g., red), hex values (e.g., #00FF00), and RGB (e.g., rgb(0,0,255)). You can also utilize the label set with just names, as shown in the Datasaur sample - Bbox only name.csv below. Other values such as captionAllowed and captionRequired will use default settings.

  • For .json, we support hex and RGB only.

Text Transcription

The Text Transcription setting allows the labeler to add corresponding text to a Bounding box. Disabling this setting means the labeler could not add the text.

By turning on the Text Transcription setting, the labeler can add text to a bounding box. You can choose whether a specific label must have a text by disabling or enabling the Require caption checkbox.

Require caption

For the rest of the Project Creation Process, you will then be able to take all the steps one takes in other project types: Preview, Labeler's Tasks, Assignment, and Project Settings.

Upload Pre-labeled Files

Label Sets Capabilities

Label Set Modification

If you've created a project but forgot to add a label set, no worries. You can either drag and drop or click the three-dot button to include a new label set.

If you've initially set up a label set while creating a bounding box project but need to make edits, simply click the three-dot button and select 'Edit label set'. This will open a pop-up section where you can make your edits.

If you need to replace the current label set, you can do so by clicking the three dots and choosing 'Replace existing label set'. This will provide a drag-and-drop area where you can upload your desired label set.

By default, both Reviewers and Labelers have access to these capabilities. However, if you prefer to restrict this capability for your Labelers, you can adjust it in File > Settings > Administrator and toggle off the option.

Custom Attributes for Label Classes

To modify the custom attributes within the label set, follow these steps:

  1. Switch to the JSON tab.

  2. Add or modify the questions field as needed for each label class.

    💡 We do not support the modification of the type and internalId value of existing questions, as it will affect labels that are already applied in unexpected ways.

    To add questions, include a questions field in the desired label class within the JSON structure. The questions field can contains multiple data with the following structures:

    1. id: ID of the question.

    2. label: The question name.

    3. type: The type of question. It can be TEXT or DROPDOWN.

    4. config: Configuration settings for the question

      1. required: Determines whether the question must be answered. It can be true or false.

      2. defaultValue: The default answer for the question

        1. For TEXT questions, it can be any text.

        2. For DROPDOWN questions, it must be an options.id value.

      3. multiline: Specifies if the TEXT question allows multiple lines. It can be true or false.

      4. options: An array of options for the DROPDOWN question, each containing:

        1. id: ID of the dropdown option. It can be used in the defaultValue.

        2. label: The text label for the dropdown option.

    5. internalId: readonly, unique id Datasaur uses to keep track of each individual question

Creating Bounding Box(es)

The first thing you will see in the Bounding Box labeling project is the Document Viewer. This viewer displays the documents or images uploaded during project creation and allows you to create bounding boxes for labeling purposes.

There are two cursor types available for creating bounding boxes: Bounding box and Text selection.

Bounding box

The main method for creating a bounding box is by drawing it directly on the document. To do this, click and drag your cursor to define the desired area. This method allows for precise placement of bounding boxes over images or documents.

Text selection

The Text Selection mode is another method to create bounding boxes by selecting text directly within documents. This approach eliminates the need to manually draw rectangles, making the labeling process quicker — especially when working with text-based documents.

  • This cursor type will only be shown in the Document Viewer toolbar for PDF documents.

  • This selection works only with native PDFs that contain selectable text. Some PDFs have text layers, while others do not.

  • Most PDFs generated from word processors (such as Microsoft Word) typically have text layers, while PDFs created from scanning documents or converting images do not.

To use this text selection mode, follow these steps:

  1. Enable Text selection mode: Select Text selection icon to enable the Text selection mode.

  2. Select text: Click and drag across text or double-click individual words to select them.

  1. Automatic Box Creation: When you release the left mouse button, a bounding box will be automatically created around the selected text. A label box dialog will then appear, allowing you to assign the appropriate label class and the selected text will automatically populate the caption input field.

Applying Label, Caption and Custom Attributes

Every time we finished drawing Bounding Box(es) on a certain area, a modal will be shown. This modal will have 2 fields by default consisting of Label and Caption, plus any attributes you have previously configured. You can select the label from the list, fill the caption and answer the attributes as needed. To submit the label, you can simply click anywhere inside the document viewer.

You can try to quickly review the value by hovering on the created Bounding Box, and it will show you a tooltip containing the values. Here’s what it will be looks like for a complete Bounding Box and the tooltip.

Automatically fill the caption using OCR

In addition to manual typing, we are now supporting an option to automatically fill captions using Tesseract! When you draw a bounding box around the desired text in the document, Tesseract will return it as the caption.

Please keep in mind that the processing time may be longer for longer text selections within the bounding box.

Enabling OCR for caption can be done in the Step 3. To do so, click on the Auto-caption provider dropdown and select Tesseract. After that, you can continue setting up your project configuration and launch it.\

Auto-caption provider dropdown is available when “Allow text caption” is enabled.

After your project has been successfully created, you are able to modify the auto-caption setting by clicking the triple-dots in the Bounding Box Labels extension. Disabling or enabling the setting will affect all documents within the project.

Auto-caption setting will not appear if no auto-caption provider has been selected in the Step 3.

There are a couple of things that you need to know about auto-caption.

  1. Auto-captioning process only happens on drawing bounding box process, not resizing or repositioning the created bounding box

  2. Auto-captioning process will be rerun when you resize or reposition the bounding box in a drawing state

  3. Auto-captioning process can be canceled by clicking the (x) inside the Caption text field

  4. Auto-captioning process will be canceled if you click outside the label box, and the bounding box will be created

    1. It will return an empty value if caption for label is optional

    2. It will throw an error if caption for label is required

  5. Auto-caption can be used for multiple bounding boxes

    1. It will read all the bounding boxes

    2. It will be separated by a line break between the bounding box content

Single Vertex Editing

Initially, a bounding box is drawn as a rectangle. But to enhance accuracy for images and documents that are not perfectly straight, users can edit individual points of the bounding box by holding the Alt key (or the Option key on Mac), and then clicking and dragging a corner of the bounding box.

Directly Able to Move and Edit a Bounding Box Upon Hover

Previously, users had to click on a bounding box before moving or resizing it. The new 'Show resize handles on hover' setting allows users to edit bounding boxes simply by hovering, providing a setting that can be customized to suit your preferences.

How and Where to Enable

Only Admins can configure this setting.

  1. During project creation, go to Step 5 and enable the "Show resize handles on hover" setting.

  2. To enable it for an existing project, open the project settings from the Projects page by clicking the three-dot icon, then select "View project settings." Navigate to the Administrator tab and enable the "Show resize handles on hover" setting.

  3. Alternatively, within the project, click the File menu, select "Settings," navigate to the Administrator tab, and enable the "Show resize handles on hover" setting.

Managing Bounding Boxes

Copy and Paste

Copying and pasting a bounding box can only be done within the same document.

Copying and pasting a bounding box can be done in two ways:

  1. The context menu: Right-click the bounding box, then select the option to copy or paste from the menu.

  2. Using the keyboard shortcut: Select the bounding box, then press Ctrl-C and Ctrl-V (or Command-C and Command-V on Mac).

Delete

Deleting a bounding box can be done in two ways:

  1. The context menu: Right-click the bounding box, then select the option to delete from the menu.

  2. Using the keyboard shortcut: Select the bounding box, then press Del.

Multiple Selection

The multiple selection capability will be helpful when bulk copying, pasting, or deleting is needed.

Selecting multiple bounding boxes can be done by holding the Ctrl key while selecting each bounding box.

Merging Multiple Bounding Boxes

Multiple bounding boxes can be merged into a single label containing multiple bounding boxes, which will be connected to each other. The merging process depends on the label classes of the selected bounding boxes:

  • If all selected bounding boxes share the same label class, they are automatically merged under that label class.

  • If the selected bounding boxes have different label classes, the label box will open, allowing the user to choose the label class to be applied. They will also be able to update the caption before saving the merged label.

The values of the merged label include the label class and caption. The caption for the merged label is determined by the selection order of the bounding boxes. For example, if the first selected bounding box has the caption "Merchant" and the second selected bounding box has the caption "name," the resulting caption will be "Merchant name”.

Merging bounding boxes can be accomplished by following these steps:

  1. Select multiple bounding box (refer to the previous section for instructions).

  2. Right-click on one of the selected labels to open the context menu.

  3. In the context menu, select "Merge" to initiate the merging process.

Managing Visibility: Show and Hide

When working with datasets containing numerous overlapping bounding boxes, the view can become cluttered, making it challenging to focus on specific areas. To address this, you can manage bounding box visibility by hiding or showing them based on their label class or individually, providing a cleaner and more organized view.

How to Hide Bounding Boxes

You can hide bounding boxes in three ways: by label class, individual bounding box from the context menu, and individual bounding box from the extension.

  1. Hiding by label class

    1. Navigate to the Bounding Box Labels extension.

    2. For each label class, you will see an eye icon located next to the label instance counter.

    3. Click the eye icon to hide all bounding boxes associated with that label class.

    4. The interface will also display the count of bounding boxes currently visible and hidden.

  2. Hiding individual bounding box from the context menu

    1. Right-click on the bounding box you want to hide in the viewer.

    2. Select Hide from the context menu.

  3. Hiding individual bounding box from the extension

    1. Navigate to the Bounding Box Labels extension.

    2. Click on the instance counter of the bounding box label class you wish to hide.

    3. Click the eye icon to hide the bounding box.

Bounding boxes added or updated after hiding will not be automatically hidden, ensuring newly created data is immediately visible for review.

How to Show Bounding Boxes

You can show bounding boxes in two ways: by label class or by individual bounding box.

  1. Showing by label class

    1. Navigate to the Bounding Box Labels extension.

    2. Simply click the eye icon. This will restore visibility for all the bounding boxes under that label class.

  2. Showing individual bounding box

    1. Navigate to the Bounding Box Labels extension.

    2. Click on the instance counter of the bounding box label class you wish to show.

    3. Click the eye icon to display the bounding box.

Tips & Tricks: Refreshing the page will reset the filter, displaying all bounding boxes. This will help when you need to quickly show all bounding boxes without manually unhiding each label class.

Reviewer Capabilities Overview

Capabilities

  • View all labelers' work.

    • Note: Bounding box labels applied in similar positions may overlap.

  • Apply labels.

  • Update labels applied by labelers.

Limitations

  • Unable to accept or reject labels.

  • Unable to view all labelers' work with information on who applied each label.

  • When updating a label applied by a labeler, there is no information or indicator showing that the label has been changed

Upload Pre-labeled Files

In Bounding Box Labeling Project, you also have the option to upload pre-labeled files.

Bounding Box Labeling supports several types of pre-labeled files. You can simply upload the appropriate file along with the document you want to label. Ensure that your pre-labeled file follows the correct format and that both files have the same name.

The supported formats for the pre-labeled files are the following.

  1. Datasaur Schema (.json)

  2. YOLO (.txt)

  3. LabelMe (.xml)

  4. Pascal VOC (.xml)

There are things to keep in mind when using YOLO, LabelMe, and Pascal VOC pre-labeled files in Datasaur:

  1. When creating a pre-labeled project, make sure to also provide your label set.

  2. During project creation, label classes from the file will be automatically mapped to the label classes in your provided label set.

  3. If the file contains more label classes than your label set, the project creation process will create additional label classes based on the number found in the file.

In Bounding Box Labeling Project, you also have the option to upload pre-labeled files. To do this, you can simply upload a JSON file along with the document that you want to label. Ensure that the JSON file follows the Datasaur Schema format. For further details about the Datasaur Schema, refer to .

Each label class in the label set can have multiple custom attributes (questions). The question types are limited to and only. Each question can also have a default answer.

Open the label set editor via Add new label set, Edit label set, or Replace existing label set. Please check the section above.

For a detailed example, refer to the section for a sample of the BBox label set with custom attributes.

Further details about the file format can be found .

Further details about the file format can be found .

Further details about the file format can be found .

Further details about the file format can be found .

here
Label Set Modification
Label Sets
watch this 2:00 minute Youtube video
this :43 second Youtube video
Text
Dropdown
this link
here
here
here
527KB
Datasaur sample - Bbox.zip
archive
24B
Datasaur sample - Bbox (only name).csv
201B
Datasaur sample - Bbox (with header).csv
158B
Datasaur sample - Bbox (without header).csv
564B
Datasaur sample - Bbox.json
2KB
Datasaur sample - Bbox (with custom attributes).json
Direct Moving and Editing.gif