# Rating

## Overview

The **Rating** evaluation project helps you assess the quality of your LLM outputs using human judgment, by rating and correcting the generated completions.

## Prerequisites

In **Rating** projects, you can evaluate two types of completions:

1. Pre-generated completions
2. Completions generated by models from Sandbox

### **Evaluate pre-generated completions**

There are two CSV formats for pre-generated completions:

1. Two column CSV format: `prompt` and `completion`.
2. Four column CSV format: `prompt_template`, `prompt`, `sources`, `completion`.

{% file src="<https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-d75ab5c362d174c42910903f882f712c3deba6ca%2FRating%20-%20with%20source.csv?alt=media&token=89ea277c-6eb8-4452-b2b6-1fffa9f3f09f>" %}

{% file src="<https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-da5b3f87e2c78ce0ffa6d0beeffdcc7cd52a080b%2FLLM%20Evaluation%20-%20Credit%20card.csv?alt=media&token=33b564a8-abcc-4c6d-9a70-d69bf6dc0242>" %}

### **Evaluate models from Sandbox**

1. Ensure the model is deployed or saved to library.
2. Prepare a dataset in a CSV file with one column: `prompt`.

{% file src="<https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-842d0213365f7250ad4f89b700e5568e9a7186b7%2FEvaluation%20with%20LLM%20Application.csv?alt=media&token=0fd2a325-6fcc-4cb5-be22-29dfb0abca1d>" %}

## Create project

To create Rating evaluation projects:

1. Navigate to the **Evaluation** page under LLM Labs menu.
2. Click **Create evaluation project,** select **Rating,** then **Continue**.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-a354bdcd751a1c663fb3e3cda34eb29322e00652%2FEvaluation%20-%20Create%20evaluation%20project%20dialog%20-%20Rating.png?alt=media" alt=""><figcaption></figcaption></figure>

3. Set up your project. Choose what you want to evaluate with:
   1. **Evaluate pre-generated completions**

      1. Upload the dataset in a CSV file with two columns: `prompt` and `completion`.

      <figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-5f475d7ad0e2ea221c7c19f8c3c0626bd9efbdcd%2FRating%20-%20PCW%20-%20Step%201%20-%20Pregenerated.png?alt=media" alt=""><figcaption></figcaption></figure>
   2. **Evaluate models from Sandbox**

      1. Upload the dataset in a CSV file with one column: `prompt`.
      2. Select the model that you want to use to generate completions. If you can’t find your model in the list, go to the [Sandbox](https://docs.datasaur.ai/llm-projects/sandbox) where your model is created, and [deploy](https://docs.datasaur.ai/llm-projects/sandbox#deploying-the-llm) or save to library. You can only evaluate deployed or saved models.

      <figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-c2fa3becff5b5135c0a0ae7461d496e4df557de9%2FRating%20-%20PCW%20-%20Step%201%20-%20LLM%20Applications.png?alt=media" alt=""><figcaption></figcaption></figure>
4. Click **Create evaluation project**.

## Evaluate completions

Open the project to evaluate the generated completions. You should rate each completion of a prompt from 1 to 5 stars. A 5-star rating usually means the completion is already perfect, so there is no need to provide feedback or edit the completion.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-83398f012de3722e35292a41bffa408e5428dbf1%2FRating%20-%20Project%20-%20Rate%205%20stars.png?alt=media" alt=""><figcaption><p>Labeler mode</p></figcaption></figure>

When the rating is below 5 stars, you have to refine the completion by providing your expected completion. After that, submit the answer to move to the next prompt.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-1a8eaffd25cc569cd60c9f201116d9bed420c466%2FRating%20-%20Project%20-%20Rate%204%20stars.png?alt=media" alt=""><figcaption></figcaption></figure>

## View evaluation results

After evaluating all completions, mark the evaluation as complete from the app bar. Click the current status **Evaluation in progress** and change it to **Evaluation completed**.

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-7babdc6fb184e9223703eca6abeffeb6d8092878%2FRating%20-%20Project%20-%20Evaluation%20status%20dropdown.png?alt=media" alt=""><figcaption></figcaption></figure>

After the evaluation is marked as complete, you can view the summary of the evaluation. For evaluating models from Sandbox, you can see:

* Average cost and processing time for generating completions
* Average evaluation score
* Evaluation results in a table view

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-5e3a800354887fd5688cc05f74644c2b8538e542%2FRating%20-%20Project%20-%20evaluation%20completed.png?alt=media" alt=""><figcaption></figcaption></figure>

For evaluating pre-generated completions, you can see:

* Average evaluation score
* Evaluation results in a table view

<figure><img src="https://448889121-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MbjY0HseEqu7LtYAt4d%2Fuploads%2Fgit-blob-ed7ad7cbc446a7f17070a9870b399a1b59f358d1%2FRating%20-%20Project%20-%20evaluation%20completed%20-%20pregenerated.png?alt=media" alt=""><figcaption></figcaption></figure>
