Debug hanging eval processes with external dependencies

Summary

Issue: Runtime process hangs indefinitely after braintrust eval completes successfully, preventing the process from exiting. The eval finishes but the CLI appears frozen with no output or error messages. (Discovered originally in the JS SDK but transpose instructions for whatever language runtime you use)

Cause: External dependencies (database connection pools, persistent connections, WebSocket clients) keep the Node.js event loop active after eval completion.

Resolution: Identify blocking dependencies using Node.js debugger and explicitly close all external connections after eval completion, or use in-memory alternatives during evals.

Resolution Steps

Step 1: Identify the blocking dependency

Run your eval with Node.js inspector to identify which module prevents process exit.

node --inspect-brk ./node_modules/.bin/braintrust eval your-eval.ts

Open chrome://inspect, pause execution when the process hangs, and check the call stack for active connections.

Step 2: Close external connections after eval completion

Explicitly close all database connections, clients, and persistent resources in your eval code.

import { Eval } from "braintrust";
import { MongoClient } from "mongodb";

const client = new MongoClient(process.env.MONGODB_URI);

Eval("my-eval", {
  data: async () => {
    await client.connect();
    // ... eval logic
    return data;
  },
  task: async (input) => {
    // ... task logic
  },
  scores: [/* ... */],
});

// Close connections after eval
process.on('beforeExit', async () => {
  await client.close();
});

Step 3: Common culprits to check

Verify these dependencies are properly closed:

• MongoDB connection pools: client.close()

• Redis clients: redis.disconnect()

• PostgreSQL/MySQL pools: pool.end()

• WebSocket connections: ws.close()

• HTTP keep-alive agents: agent.destroy()