Blank comparison rows due to mismatched inputs

Summary

Issue: Comparison rows appear blank when comparing two experiments, and the comparison table shows "–" instead of score values for the second experiment.

Cause: Experiments are compared based on a comparison key (defaults to input), and blank rows indicate the input values don't match exactly between the two experiments.

Resolution: Ensure input values match exactly across experiments, or configure a custom comparison key in project settings.

Applicable To

Plans: Free, Pro, Enterprise

Deployments: Braintrust Hosted, Hybrid

Use case: Experiment comparison with mismatched inputs or custom data structures

Resolution Steps

If input values differ between experiments

Step 1: Verify input matching

Check that the input field values are identical across both experiments being compared.

Step 2: Standardize inputs

Re-run experiments with consistent input values to enable row-by-row comparison.

Otherwise, if you need to compare on a different field

Step 1: Configure comparison key

Navigate to Project Settings and set a custom comparison key under the comparison settings section.

Step 2: Select appropriate field

Choose a field that consistently exists and matches across both experiments (e.g., id, metadata.key, or custom field).

Additional Information

Understanding comparison behavior

The comparison table matches rows based on the comparison key. When no match exists, the cell displays "–" in the main table view.

Checking for matches

Drill into individual traces to see the message "There are no input matches in the comparison experiment" when a row has no corresponding match.

Best practices

• Use consistent input formatting across experiment runs

• Configure comparison keys before running large experiment sets

• Test comparison configuration with small experiments first