Online scorer deduplication with concurrent feedback

Last updated: January 30, 2026

Summary

Issue: Online scorers don't re-trigger for all feedback calls when multiple feedback API calls update the same span's expected field in quick succession, and only the last feedback value is scored.

Cause: Scoring requests are deduplicated by row_id within write-ahead log (WAL) processing batches, so only one scoring invocation executes per span per batch.

Resolution: Space out feedback API calls to ensure they process in separate WAL batches, or restructure your application to send only one feedback update per span.

Applicable To

Plans: All plans

Deployments: Braintrust Hosted and Hybrid

Use case: Online scorers with multiple concurrent feedback API calls updating the same span

How Deduplication Works

When feedback calls arrive in quick succession:

  1. First feedback call updates expected, generates scoring request with token A

  2. Second feedback call updates expected again, generates scoring request with token B

  3. Both requests enter the same WAL batch

  4. Second request overwrites the first (same row_id key)

  5. Only one scoring invocation triggers (with token B's data)

This is intentional behavior to prevent redundant scoring invocations.

Resolution Steps

Option 1: Space out feedback calls

Introduce a delay between feedback API calls for the same span.

import time
from braintrust import update_span

# First feedback update
update_span(span_id, expected={"patient": None})

# Wait for first update to process
time.sleep(1)

# Second feedback update
update_span(span_id, expected={"customer": "9d50173e18664h91ab"})

Option 2: Consolidate feedback updates

Queue feedback updates and send only the final state per span.

from braintrust import update_span

# Instead of multiple calls, determine final state first
final_expected = {"customer": "9d50173e18664h91ab"}

# Single feedback update
update_span(span_id, expected=final_expected)