Multi-model evaluation

This feature allows you to compare and evaluate the performance of multiple models using various metrics from evaluators like Ragas, Langchain, and Deepeval.

Get started

To evaluate multiple models:

1. Navigate to the Evaluation page from the sidebar.

2. Click the Create evaluation project button and choose Automated evaluation project type, then Continue.

   
3. Configure your evaluation by selecting the models to evaluate and choosing a dataset from the library. If you don’t have one, you can also upload a dataset in a CSV format containing two columns: prompt and expected completion.

   

If you can’t find your model in the list, go to the Sandbox where your model is created, and deploy or save to library. You can only evaluate deployed or saved models.

1. Select the metric, provider, and the evaluator model you want to use for evaluation. Learn more about the evaluators and metrics.

   
2. Click Create evaluation project and wait for the evaluation process to finish.

Analyze the evaluation results

After the evaluation process is completed, you can analyze the results.

Summary of the evaluation

Here you can view the total cost, time taken for generating completions, and the overall performance score given by the evaluator.

Result and score from each model

Here you can view the quality, the score, and the processing time of the generated completions from each model.

Evaluation details

To view the evaluation details of a completion, click the three-dot menu at the far right of the row, then select View details.