ML-Assisted Labeling

ML-assisted labeling extension lets you use open-source models, LLMs, or your own models to generate labels automatically.

Introduction

ML-assisted labeling helps automate data labeling for NLP projects. It supports span, row, bounding box, and document labeling by using open-source models, large language models (LLMs), and custom models. This reduces manual effort and improves labeling speed and consistency.

Key features

1. Batch labeling: Label multiple items at once, eliminating the need to label each item individually.

2. Model integrations: Works with models for tasks such as named entity recognition (NER), part-of-speech (POS) tagging, and sentiment analysis, as well as LLMs and external providers.

3. Automation: Generate labels automatically and review them to ensure quality.

Quick start guide

To enable the ML-assisted labeling extension:

1. Go to the Manage extensions dialog and enable the ML-assisted labeling extension.

   

   ML-assisted labeling extension with spaCy
2. Select a service provider.

3. Click Predict labels to generate labels.

For row labeling projects, there are some additional steps:

1. Select rows: Choose which rows to include in prediction.

2. Target text: Select input column used as context.

3. Target question: Select output field to predict.

4. Faster prediction speed: Run predictions via the backend.

Supported model providers

Row labeling	Span labeling	Bounding box labeling	Document labeling
Sentiment Analysis	SpaCy	Custom Model	Datasaur LLM Labs
LLM Assisted Labeling	LLM Assisted Labeling
Amazon Comprehend	NLTK
Google Vertex AI	CoreNLP and SparkNLP NER
Amazon SageMaker	CoreNLP and SparkNLP POS
Datasaur LLM Labs	Datasaur LLM Labs
Azure	FewNERD
Hugging Face	Hugging Face
Custom Model	Custom Model

Model providers are grouped into the following categories:

• Datasaur hosted

  • Prebuilt models hosted by Datasaur for common NLP tasks such as NER, sentiment analysis, POS tagging, and dependency parsing.

  • Examples: spaCy, CoreNLP, and FewNERD.

• Cloud providers

  • Models hosted on external platforms. You can use pre-trained or fine-tuned models via API.

  • Examples: Hugging Face Inference API, Azure ML, Google Vertex AI, and Amazon SageMaker.

• LLM Assisted Labeling

  • Models from LLM providers that require an API key.

  • Examples: OpenAI (GPT models), Azure OpenAI, Anthropic (Claude models), Gemini (Google AI), Cohere, and other custom models that can be connected via API.

• LLM Labs

  • Models deployed through Datasaur LLM Labs, providing access to multiple providers through a single endpoint.

  • This allows users to switch between different models without manually configuring each provider separately.

• Custom models

  • Connect your own model using a custom REST API. The API must follow the required request format.

  • It provides flexibility for organizations with internally trained models or self-hosted LLMs.

ML-assisted labeling providers in Datasaur

Type	Examples
Datasaur hosted	spaCy, CoreNLP, SparkNLP, NLTK, Sentiment Analysis, FewNERD
Cloud provider	Hugging Face, Azure ML, Google Vertex AI, Amazon SageMaker
LLM Assisted Labeling	OpenAI, Azure OpenAI, Anthropic, Gemini, Cohere, custom
LLM Labs	100+ LLM providers
Custom	Depends on your internal API

Restrict ML-assisted labeling settings

Admins or reviewers can restrict ML-assisted labeling settings to ensure consistent configuration across labelers. When enabled, labelers use the configuration set by the admin or reviewer, ensuring consistent results.

Steps

1. Click the three-dot menu next to the ML-assisted labeling header.

2. In Modify service provider setting, choose one of the following options:

   1. All assignees: Allows all labelers to modify their own settings.

   2. Admin or reviewer only: Restricts changes to admins or reviewers.

Behaviors

• When Admin or reviewer only is selected, labelers cannot change the service provider or settings. Admins and reviewers can still update the configuration.

• In ongoing projects, labelers must refresh the page to apply updated settings.