Custom metrics
Last updated
Last updated
Automated Evaluation: Custom metrics feature enables users to define their own evaluation configurations to suit specific project needs. With Custom Metrics, users have full control over evaluation criteria, scoring ranges, and evaluator models to ensure accurate and meaningful assessments.
To begin using the Automated Evaluation custom metrics:
Navigate to the Evaluation page under LLM Labs menu.
Click the Create evaluation project button and choose Automated evaluation project type.
Configure your evaluation by selecting the applications that you want to evaluate, and uploading the ground truth dataset in a CSV format containing two columns: prompt and expected completion.
Set Up the Configuration, the configuration consists:
Evaluator Model: Choose your desired models from the list of available models.
Custom Evaluator Name: Enter a unique name to identify your custom evaluation.
Minimum and Maximum Score: Define the scoring range (e.g., 0 to 100).
Prompt: Write a clear, detailed prompt that explains the evaluation process. Include specific criteria and instructions for assessing responses.
Click the Create evaluation project button and wait for your evaluation process to be done.
After the evaluation process is completed, you can analyze the results. Learn more on how to analyze the result.
