# Rating

## Overview

The **Rating** evaluation project helps you assess the quality of your LLM outputs using human judgment, by rating and correcting the generated completions.

## Prerequisites

In **Rating** projects, you can evaluate two types of completions:

1. Pre-generated completions
2. Completions generated by models from Sandbox

### **Evaluate pre-generated completions**

There are two CSV formats for pre-generated completions:

1. Two column CSV format: `prompt` and `completion`.
2. Four column CSV format: `prompt_template`, `prompt`, `sources`, `completion`.

{% file src="/files/5LBrx35IRg3uRUTNqQb8" %}

{% file src="/files/b1MYRvWGithOfwS7UVjZ" %}

### **Evaluate models from Sandbox**

1. Ensure the model is deployed or saved to library.
2. Prepare a dataset in a CSV file with one column: `prompt`.

{% file src="/files/kV4uhIM4zyuJmW3JWnav" %}

## Create project

To create Rating evaluation projects:

1. Navigate to the **Evaluation** page under LLM Labs menu.
2. Click **Create evaluation project,** select **Rating,** then **Continue**.

<figure><img src="/files/BgS0npo3ENidePqarYDB" alt=""><figcaption></figcaption></figure>

3. Set up your project. Choose what you want to evaluate with:
   1. **Evaluate pre-generated completions**

      1. Upload the dataset in a CSV file with two columns: `prompt` and `completion`.

      <figure><img src="/files/gk4Y10RZVvRnxlV9aqyn" alt=""><figcaption></figcaption></figure>
   2. **Evaluate models from Sandbox**

      1. Upload the dataset in a CSV file with one column: `prompt`.
      2. Select the model that you want to use to generate completions. If you can’t find your model in the list, go to the [Sandbox](https://docs.datasaur.ai/llm-projects/sandbox) where your model is created, and [deploy](https://docs.datasaur.ai/llm-projects/sandbox#deploying-the-llm) or save to library. You can only evaluate deployed or saved models.

      <figure><img src="/files/vmeaQgzVIV7E9HkpYCxQ" alt=""><figcaption></figcaption></figure>
4. Click **Create evaluation project**.

## Evaluate completions

Open the project to evaluate the generated completions. You should rate each completion of a prompt from 1 to 5 stars. A 5-star rating usually means the completion is already perfect, so there is no need to provide feedback or edit the completion.

<figure><img src="/files/Z23uPNCkEekXQFYvQxDU" alt=""><figcaption><p>Labeler mode</p></figcaption></figure>

When the rating is below 5 stars, you have to refine the completion by providing your expected completion. After that, submit the answer to move to the next prompt.

<figure><img src="/files/wer71jjYkrK28E0Zv1Lr" alt=""><figcaption></figcaption></figure>

## View evaluation results

After evaluating all completions, mark the evaluation as complete from the app bar. Click the current status **Evaluation in progress** and change it to **Evaluation completed**.

<figure><img src="/files/UpaoPOF6EwcWq5q8VXzH" alt=""><figcaption></figcaption></figure>

After the evaluation is marked as complete, you can view the summary of the evaluation. For evaluating models from Sandbox, you can see:

* Average cost and processing time for generating completions
* Average evaluation score
* Evaluation results in a table view

<figure><img src="/files/8KjalsDAvRFQahs9cyzu" alt=""><figcaption></figcaption></figure>

For evaluating pre-generated completions, you can see:

* Average evaluation score
* Evaluation results in a table view

<figure><img src="/files/KzEA6n5LHGrbF3xhfxEQ" alt=""><figcaption></figcaption></figure>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.datasaur.ai/llm-projects/evaluation/rating.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
