> For the complete documentation index, see [llms.txt](https://docs.datasaur.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.datasaur.ai/llm-projects/evaluation/rating.md).

# Rating

## Overview

The **Rating** evaluation project helps you assess the quality of your LLM outputs using human judgment, by rating and correcting the generated completions.

## Prerequisites

In **Rating** projects, you can evaluate two types of completions:

* Pre-generated completions
* Completions generated by models from Sandbox

### Evaluate pre-generated completions

There are two CSV formats for pre-generated completions:

* Two column CSV format: `prompt` and `completion`.
* Four column CSV format: `prompt_template`, `prompt`, `sources`, `completion`.

{% file src="/files/5LBrx35IRg3uRUTNqQb8" %}

{% file src="/files/b1MYRvWGithOfwS7UVjZ" %}

### Evaluate models from Sandbox

1. Ensure the model is deployed or saved to library.
2. Prepare a dataset in a `.csv` file with one column: `prompt`.

{% file src="/files/kV4uhIM4zyuJmW3JWnav" %}

## Create a project

To create **Rating** evaluation projects:

1. Navigate to the **Evaluation** page from the sidebar.
2. Click **Create evaluation project,** select **Rating,** then **Continue**.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-a354bdcd751a1c663fb3e3cda34eb29322e00652%2FEvaluation%20-%20Create%20evaluation%20project%20dialog%20-%20Rating.png?alt=media" alt=""><figcaption></figcaption></figure>

3. Set up your project. Choose what you want to evaluate with:
   1. **Evaluate pre-generated completions**

      1. Upload the dataset in a `.csv` file with at least two columns: `prompt` and `completion`.

      <figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-5f475d7ad0e2ea221c7c19f8c3c0626bd9efbdcd%2FRating%20-%20PCW%20-%20Step%201%20-%20Pregenerated.png?alt=media" alt=""><figcaption></figcaption></figure>
   2. **Evaluate models from Sandbox**

      1. Upload the dataset in a `.csv` file with one column: `prompt`.
      2. Select the model that you want to use to generate completions. If you can’t find your model in the list, go to the [Sandbox](/llm-projects/sandbox.md) where your model is created, and [deploy](/llm-projects/sandbox.md#deploy-models) or save to library. You can only evaluate deployed or saved models.

      <figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-c2fa3becff5b5135c0a0ae7461d496e4df557de9%2FRating%20-%20PCW%20-%20Step%201%20-%20LLM%20Applications.png?alt=media" alt=""><figcaption></figcaption></figure>
4. Click **Create evaluation project**.

## Evaluate completions

Open the project to evaluate the generated completions. You should rate each completion of a prompt from 1 to 5 stars. A 5-star rating means the completion is already perfect, so there is no need to provide feedback or edit the completion.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-83398f012de3722e35292a41bffa408e5428dbf1%2FRating%20-%20Project%20-%20Rate%205%20stars.png?alt=media" alt=""><figcaption><p>Labeler mode</p></figcaption></figure>

When the rating is below 5 stars, you have to refine the completion by providing your expected completion. After that, submit the answer to move to the next prompt.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-1a8eaffd25cc569cd60c9f201116d9bed420c466%2FRating%20-%20Project%20-%20Rate%204%20stars.png?alt=media" alt=""><figcaption></figcaption></figure>

## View evaluation results

After evaluating all completions, mark the evaluation as complete from the app bar. Click the current status **Evaluation in progress** and change it to **Evaluation completed**.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-7babdc6fb184e9223703eca6abeffeb6d8092878%2FRating%20-%20Project%20-%20Evaluation%20status%20dropdown.png?alt=media" alt=""><figcaption></figcaption></figure>

After the evaluation is marked as complete, you can view the summary of the evaluation. For evaluating models from Sandbox, you can see:

* Average cost and processing time for generating completions.
* Average evaluation score.
* Evaluation results in a table view.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-5e3a800354887fd5688cc05f74644c2b8538e542%2FRating%20-%20Project%20-%20evaluation%20completed.png?alt=media" alt=""><figcaption></figcaption></figure>

For evaluating pre-generated completions, you can see:

* Average evaluation score.
* Evaluation results in a table view.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-ed7ad7cbc446a7f17070a9870b399a1b59f358d1%2FRating%20-%20Project%20-%20evaluation%20completed%20-%20pregenerated.png?alt=media" alt=""><figcaption></figcaption></figure>
