> For the complete documentation index, see [llms.txt](https://docs.datasaur.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.datasaur.ai/data-studio-projects/reviewing-nlp-projects.md).

# Reviewing Projects

## Overview

Reviewer mode helps you review labeling work and ensure data quality. As a reviewer, you check accuracy, resolve conflicts, and keep the workflow consistent.

## How it works?

You must have the **Reviewer** role to use the reviewer mode in a project. [Learn more about roles here](/workspace-management/role-and-permission.md).

### Reviewer mode for span labeling

In span labeling projects, there are three types of conflicts:

* Content conflict
* Span conflict
* Arrow conflict

See the [documentation](/advanced/extensions/review.md#conflicts) for details on the differences between these conflict types.

#### Resolve conflicts

To resolve a conflict, right-click the conflicted label to accept or reject it. You can also replace it by left-clicking the label and selecting a new one.

<figure><img src="/files/A4Eq359PcbPcevtVchbC" alt=""><figcaption></figcaption></figure>

#### Navigate between conflicts

You can navigate between conflicts from the **Go** menu or by pressing `Alt` + `Shift` + `Right` to go to the next conflict and `Alt` + `Shift` + `Left` to go to the previous one.

<figure><img src="/files/9uWxmyyujXlNTx4jfvuR" alt=""><figcaption></figcaption></figure>

#### Label status colors

In reviewer mode, label colors indicate the status of each label. You can view the color legend in the status bar at the bottom.

<figure><img src="/files/KiHwKbNqFTMV02ijjYp8" alt=""><figcaption></figcaption></figure>

* **Gray** labels mean they have reached consensus among labelers.

  <figure><img src="/files/IKyI1b5vgWU55laHbDYE" alt=""><figcaption></figcaption></figure>
* **Yellow** labels indicate that they were created using the [asisted labeling extensions](/assisted-labeling/ml-assisted-labeling.md).

  <figure><img src="/files/S9cm5m7yk4mzO35aLy8p" alt=""><figcaption></figcaption></figure>
* **Blue** labels mean the label has been rejected by the reviewer. By default, rejected labels are hidden. You can make them visible in **File** → **Settings** → **Administration** by enabling **Show rejected labels in reviewer mode**.

  <figure><img src="/files/rtazGbDvdQ5i0yno1Kda" alt=""><figcaption></figcaption></figure>
* **Purple** labels indicate that they were applied by the reviewer.

  <figure><img src="/files/jfPMzpKveXIIm8VDeyaM" alt=""><figcaption></figcaption></figure>
* **Red** labels indicate that they are conflicted (have not reached the consensus threshold) and require review to be accepted or rejected.

  <figure><img src="/files/WBNhjaa2IIj4mjxkOqTh" alt=""><figcaption></figcaption></figure>

In span labeling, you can see the number of applied labels, the last labeled row, and the total resolved rows. This information is shown in the bottom-right corner.

<figure><img src="/files/Aj8s30epxwO7954yrX3X" alt=""><figcaption></figcaption></figure>

### Reviewer mode for row labeling

Reviewing in row labeling focuses on accepting, rejecting, or updating answers in each row using the **Row labeling** extension.

<figure><img src="/files/VSj1fMdQ6zMaLJ7cDdxg" alt=""><figcaption></figcaption></figure>

#### Row colors

Each row has a colored line on the left side of the number column to indicate its status.

<figure><img src="/files/QmVDKqSQwfsXHdRoB1V4" alt=""><figcaption></figcaption></figure>

* **No colored line** means the answers for that row have reached consensus among labelers or have already been resolved by the reviewer.
* **A red line** indicates that the row has no consensus. All answers are conflicted.
* **A pink line** indicates that the row contains both a consensus answer and a conflicting answer. This occurs when a question allows multiple answers.

  <figure><img src="/files/ckMPnJtsIjX23VhHO0wQ" alt=""><figcaption></figcaption></figure>

#### Answers in the table

Whether answers are shown in the table depends on their status.

* **Meets consensus:** Answers are displayed.
* **Conflicted (No consensus):** No answers are displayed.
* **Mixed consensus and conflict:** Only consensus answers are shown; conflicted answers appear after review.
* **Resolved by reviewer:** Final answers are displayed after review.

Submitting answers in the **Row labeling** extension updates the table accordingly.

### Reviewer mode for document labeling

The behavior is similar to row labeling, with the main interactions happening in the **Document labeling** extension.

<figure><img src="/files/hBmxHRS56ZjMGWE9iPFo" alt=""><figcaption></figcaption></figure>

### Reviewer mode for bounding box labeling

Review for bounding box labeling is not yet available. This feature is coming soon!

<figure><img src="/files/84TqDNzRieaQg4PduMoA" alt=""><figcaption></figcaption></figure>