> For the complete documentation index, see [llms.txt](https://docs.datasaur.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.datasaur.ai/advanced/extensions/analytics.md).

# Analytics

Analytics to track labeler performance and progress in a project.

## Analytics

Analytics extension provides real-time performance metrics for each labeler in a project. You can use it to:

* View labeler performance and status, including accepted and rejected labels or answers.
* View how labels and answers are distributed across your label sets and questions.
* Filter by document to analyze performance for specific files.
* Filter by member to compare individual labelers and identify top performers or those who may need support.

Reviewers can view performance data for all assigned labelers even before the project is marked as complete. This helps reviewers track progress throughout the labeling process.

Labelers can only view their own performance after the project has been marked as complete by a reviewer.

The extension has two tabs: **Overview** and **Distribution**.

### Overview tab

The Overview tab provides a summary of labeling performance and progress in the project. It shows key metrics for labels, answers, and pre-labeled data, helping reviewers understand how labelers are performing and how much of the work has been accepted, rejected, or remains unresolved.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-2df108da51bb84e7e7e5519f8eb90129c5a6b152%2Fimage%20(1).png?alt=media" alt=""><figcaption></figcaption></figure>

#### General

* **Effective time spent**: The actual time labelers spent during the labeling process.
* **Accepted pre-labeled**: Pre-labeled labels or answers that were accepted by labelers.
* **Rejected pre-labeled**: Pre-labeled labels or answers that were not applied by labelers, whether it's rejected, ignored, or changed.

#### Labels

These metrics are available for **span labeling, conversational labeling,** and **bounding box labeling** projects.

* **Accepted labels**: Labels from labelers that were accepted by reviewers, either manually or through consensus.
* **Rejected labels**: Labels from labelers that were rejected by reviewers.
* **Conflicted labels**: Labels that have not yet been resolved through review or consensus.
* **Missed labels**: Labels added by reviewers in Reviewer mode that were not provided by labelers.

#### Answers

These metrics are available for **row labeling** and **document labeling** projects. If a project consists of multiple questions, each answer will be counted as one.

* **Accepted answers**: Answers from labelers that were accepted by reviewers, either manually or through consensus.
* **Rejected answers**: Answers from labelers that were rejected by reviewers.
* **Answered rows**: Number of rows that have been labeled. This metric is available for row labeling projects only.

{% hint style="info" %}
For mixed labeling projects (projects with more than one labeling type), all relevant metrics are shown, including both labels and answers.
{% endhint %}

### Distribution tab

The Distribution tab shows the distribution of labels and answers based on data from Labeler mode. Use it to identify class imbalance in your dataset and see which label classes or answer options are most frequently rejected during review.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-391a2c6f278472f8c4ff22f875fb8e60863c2806%2Fimage%20(3).png?alt=media" alt=""><figcaption></figcaption></figure>

#### Reading the panel

The panel shows one section for each labeling type in your project, such as **Span labeling** or **Line labeling**.

Within each section, results are grouped by label set (for example, *Default NER*) or question (for example, *Yes or no?*). Only the top three entries in each group are shown, ranked by count.

Each entry shows:

* **Label class or answer:** The label class or answer, with its total count shown in a badge.
* **Accepted:** The number of labels or answers accepted by reviewers.
* **Rejected:** The number of labels or answers rejected by reviewers.

Select **View all** to see every entry.

#### Full distribution

The **Label and answer distribution** dialog shows one tab for each labeling type in the project. Select a tab to view the complete distribution table for that labeling type.

For labeling types that use labels, **span labeling, conversational labeling,** and **bounding box labeling**:

<table><thead><tr><th width="160.09765625">Column</th><th>Description</th></tr></thead><tbody><tr><td><strong>Label set</strong></td><td>The label set the class belongs to.</td></tr><tr><td><strong>Label class</strong></td><td>The label class, shown with its assigned color.</td></tr><tr><td><strong>Count</strong></td><td>The total number of times the class was applied.</td></tr><tr><td><strong>Accepted</strong></td><td>The number and percentage of labels of that class accepted by reviewers.</td></tr><tr><td><strong>Rejected</strong></td><td>The number and percentage of labels of that class rejected by reviewers.</td></tr></tbody></table>

For labeling types that use answers, **row labeling, line labeling,** and **document labeling**:

<table><thead><tr><th width="160.06640625">Column</th><th>Description</th></tr></thead><tbody><tr><td><strong>Question</strong></td><td>The question the answer belongs to.</td></tr><tr><td><strong>Answer</strong></td><td>The selected answer option.</td></tr><tr><td><strong>Count</strong></td><td>The total number of times the answer was selected.</td></tr><tr><td><strong>Accepted</strong></td><td>The number and percentage of answers accepted by reviewers.</td></tr><tr><td><strong>Rejected</strong></td><td>The number and percentage of answers rejected by reviewers.</td></tr></tbody></table>

Every column can be sorted, and long tables are paginated.

{% hint style="info" %}
For projects with a single labeling type, the dialog is titled according to its content. For example, **Label distribution** is used for a span labeling project and no tabs are shown.
{% endhint %}

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-5e7f5711cf2c2717736a71dd2d949ba84147d52a%2Fimage%20(4).png?alt=media" alt=""><figcaption></figcaption></figure>

#### Filtering and refreshing

* Use the document dropdown to switch between **All documents** and a specific document. This helps determine whether a distribution skew is dataset-wide or limited to certain files.
* Distribution data is cached. The panel and dialog show when the data was last updated.
  * Select **Update now** to recalculate the distribution data.