> For the complete documentation index, see [llms.txt](https://docs.datasaur.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.datasaur.ai/data-studio-projects/reviewing-nlp-projects/review-sampling.md).

# Review Sampling

Review sampling helps reviewers track progress and focus on a subset of data. This allows for efficient quality checks while reducing review time.

{% hint style="info" %}
This feature is only available for span labeling, row labeling, document labeling, and span + document labeling projects.
{% endhint %}

### Configuration

You can enable this feature during project creation or in project settings.

#### During project creation

1. In step 4, set the conflict resolution to **No consensus**.
2. Enable **Set review sampling rate** in the bottom right corner.
3. The default sampling rate is 10%, but you can adjust it as needed.

![](/files/fncgMrmMEKuhPOZRbPRx)

#### **In project settings**

* In project settings, go to the **Assignment** tab.
* Set the conflict resolution to **No consensus**.
* Enable **Set review sampling rate** in the bottom right corner.

<figure><img src="/files/rzJoG5cGs8y6I0PC5ohi" alt=""><figcaption></figcaption></figure>

### View sampling progress

Once sampling rate is enabled and labeling is complete, reviewers can track progress in the **Progress** tab of the **Review** extension. This shows how many samples need review and how many have been completed.

<figure><img src="/files/rK2EwK10DyxtcruEucH5" alt=""><figcaption></figcaption></figure>

### How the progress is calculated

#### Span labeling projects

Progress is based on the number of unique labeled spans in each document.

**Example**: If Document A has 8 labeled spans and Document B has 11 labeled spans, with a 50% sampling rate, the reviewer needs to review 4 spans from Document A and 6 spans from Document B.

#### Row labeling projects

Progress is based on the number of rows in each document.

**Example**: If Document A has 10 rows and Document B has 5 rows, with a 50% sampling rate, the reviewer needs to review 5 rows from Document A and 3 rows from Document B.

#### Document labeling projects

Progress is based on the total number of documents.

**Example**: If a project has 12 documents and a 50% sampling rate, the reviewer needs to review 6 documents.

#### Span + document labeling projects

Progress combines both span and document calculations.

**Example**: If a project has 12 documents, each with 8 labeled spans, and a 50% sampling rate, the reviewer needs to review 6 documents and at least 4 spans within each document.

### Marking the project as complete

When the feature is enabled, the system checks whether the required sampling rate is met before allowing the project to be marked as complete.

* If the requirement is met, a confirmation dialog appears and you can mark the project as complete.

  ![](/files/gA44JNl4lXkIaHgsTZRx)
* If the requirement is not met, a warning shows how many items remain. You must complete the required reviews before marking the project as complete.

  ![](/files/ol2BmWslyqdKmn9vlSK1)

This ensures the review process meets the required quality threshold before completion.
