> For the complete documentation index, see [llms.txt](https://docs.datasaur.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.datasaur.ai/advanced/extensions/review.md).

# Review

The **Review** extension helps streamline the review process by making it easier to identify conflicts, track progress, and maintain labeling quality and consistency.

## Enable the extension

Click the gear icon in the extension panel on the right to open the **Manage extensions** dialog. Then, enable the **Review** extension.

<figure><img src="/files/KdE4sr2zT1NGxWS5vXNd" alt=""><figcaption></figcaption></figure>

## How it works?

The extension helps you review labeling work by showing:

* Labeler information.
* Conflicting labels or answers.
* Review progress.

### Conflicts

Find and resolve disagreements between labelers.

* The **Document** field allows you to view conflicts in a specific document.
* The **Show work from** field allows you view conflicts made by a specific labeler.

  <figure><img src="/files/KbZ8LdHjhjtm8o0SEoDW" alt=""><figcaption></figcaption></figure>
* **Conflicts** section allows you to identify all conflicting answers provided by different labelers. The displayed information varies depending on the project type.

### Progress

This tab is only visible when the **Set review sampling rate** setting is enabled. Reviewers can use it to track their review progress and see whether they have met the configured sampling rate.

Read more information about review sampling [here](/data-studio-projects/reviewing-nlp-projects/review-sampling.md).

<figure><img src="/files/V9twNr8XVYogoEZrsSA0" alt=""><figcaption></figcaption></figure>

## Span labeling

<figure><img src="/files/wjZ2DNBHiroGLcTUIK4o" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
For the full review workflow, see [Reviewer mode](/data-studio-projects/reviewing-nlp-projects.md).
{% endhint %}

### Conflicts

In span labeling, there are three types of conflicts. Click **View all** in each section to see the full list of conflicts.

<figure><img src="/files/iyvi4fZOy4LM9oRqx746" alt=""><figcaption></figcaption></figure>

* **Contents:** Shows text conflicts caused by edits made by labelers. For example:
  * Original text: `It is Shakespeare's longest...`
  * Edited text: `It is his longest...`

    <figure><img src="/files/XxO1K1CQ3QvtxUTZs6Ns" alt=""><figcaption></figcaption></figure>
* **Spans:** Shows conflicting span labels created by different labelers.

  <figure><img src="/files/bOR1qbDxl7tYvi309te4" alt=""><figcaption></figcaption></figure>
* **Arrows:** Shows conflicting arrow relationships created by different labelers.

  <figure><img src="/files/tumXMjCMkci5JSHp2uXx" alt=""><figcaption></figcaption></figure>

### Accept or reject all conflicted labels

Reviewers can resolve multiple remaining conflicts at once instead of reviewing each label individually. This action is available for both span and arrow conflicts.

#### Quick guide

1. Open the **Review** extension and click **View all** in **Span labels** or **Arrow labels** section.

   <figure><img src="/files/iyvi4fZOy4LM9oRqx746" alt=""><figcaption></figcaption></figure>
2. Click **Accept all** or **Reject all**.

   <figure><img src="/files/5ZmFaJjdQKrUq1KQz2qw" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
**Important notes**

* These actions cannot be undone.
* Only currently visible labels are affected.
* Labels connected to a sentence conflict will not be affected until the sentence conflict is resolved first.
  {% endhint %}

#### Accept all conflicted labels

* If the **Spans should have at most one label** setting is **unchecked**, all conflicted labels will be accepted.
* If the **Spans should have at most one label** setting is **checked**:
  * Labels remain conflicted if another different label exists on the same span or arrow. This requires manual review.
  * Labels are accepted only if no other different labels exist on the same span or arrow.

<figure><img src="/files/SQ65YnkK5FDQQAkTQ952" alt=""><figcaption></figcaption></figure>

#### Reject all conflicted labels

All conflicted labels will be rejected.

## Row labeling

<figure><img src="/files/rUKSGKJuD3nzHBxgKEzW" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
For the full review workflow, see [Reviewer Mode](/data-studio-projects/reviewing-nlp-projects.md).
{% endhint %}

### Conflicts

In row labeling, there is only 1 type of conflicts.

<figure><img src="/files/A7eTj4oGj8DFqqJnCovF" alt=""><figcaption></figcaption></figure>

Click **View all** in the **Conflicted answers** section to see the full list of conflicts.

<figure><img src="/files/UbFLNs6ictvszUtt8OZt" alt=""><figcaption></figcaption></figure>

You can also filter conflicts by labeler by selecting a labeler from the **Show work from** dropdown. This displays only the conflicts associated with that labeler.

<figure><img src="/files/h0fFq96eFTonVxlleQz9" alt=""><figcaption></figcaption></figure>

## Document labeling

<figure><img src="/files/0dZu3Tt31f0LG4bKyY3C" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
For the full review workflow, see [Reviewer Mode](/data-studio-projects/reviewing-nlp-projects.md).
{% endhint %}

### Conflicts

To resolve conflicts in document labeling projects, use the **Document labeling** extension.

<figure><img src="/files/u587QNznIeboW62yTpRC" alt=""><figcaption></figcaption></figure>

You can view answers from all labelers and resolve conflicts by selecting and submitting the best answer.

<figure><img src="/files/icsDqJfDdNqSQQCRFxhM" alt=""><figcaption></figcaption></figure>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.datasaur.ai/advanced/extensions/review.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.