Re-running scorers without re-running tasks

Summary

You can re-run scorers on existing outputs without re-executing the original task prompts using three different approaches: manual rescoring on the Logs page, independent scorer testing in the Scorer UI, or full row re-runs in the Playground.

Applicable To

Plans: Any

Deployments: Any

Available Options

Option 1: Manual rescoring on Logs page

Rescore specific rows in your Logs without re-running the original tasks.

1. Navigate to your project's Logs page

2. Select the checkbox for the desired row(s)

3. Click the "Score" button

4. Choose the scorer you want to run on the selected rows

5. The scorer runs on existing outputs without re-invoking tasks

Option 2: Independent scorer testing

Use the Scorer UI to test and iterate on scoring logic before applying to evaluations.

1. Navigate to the Scorer UI

2. Input sample data and test your scorer logic

3. Iterate on scorer criteria without running full evaluations

Option 3: Full row re-runs in Playground

The Playground UI does not support scorer-only re-runs but allows re-executing both tasks and scorers together on a single selected row. Note: This requires rerunning the full task for that particular row.

1. Open your Playground with the desired prompt and dataset

2. Click into the row to re-evaluate

3. Click "Run row" to re-execute the task prompt and all scorers