Consensus
Allows you to review your project automatically.
Consensus determines how many labelers must agree before a label is automatically accepted in a project. If a label does not meet the consensus threshold, it is marked as conflict. Currently, two types of consensus are available:
No consensus – Every label is marked as a conflict and must be resolved by a reviewer.
Peer review consensus – Labels are automatically accepted once the required number of labelers agree. For example, if the consensus is set to 1, every label will be automatically accepted.
No consensus
To enable No consensus, follow these steps:
During project creation, go to Step 4.
In the Conflict resolution section, select No consensus.
Use case
The No consensus setting can be used if you want your reviewers to review every label applied by the labelers. All applied labels will be considered as conflicts, and reviewers must manually accept or reject them in reviewer mode.
Peer review consensus
Peer review consensus is explained in the QA/Review section.
Last updated