Skip to main content

Copy this into your coding agent

Paste into Cursor, Codex, Claude Code, or another coding agent in your project.
Beta release. All four languages support tracing, real app tests and scripted simulations.

Install

Package and API reference · Source and examples

Record one interaction

Choose Go on the SDK setup page. Go 1.22 or later is required. This client uses the standard library.
Open the system’s Production tab to confirm receipt.

Wrap your application

Replace the callback with your request handler. Pass its ctx into nested bench.Trace calls for model and tool operations, including goroutines. Use Kind: "TOOL" for tools. The original result and error are returned unchanged. For streams, use StartSpan, keep the span open, then call SetOutput or SetError and End when the stream finishes. Explicit wrappers work with Google ADK and custom Go applications. Framework versions may require a newer Go version than the Bench client. Content capture defaults to off. To enable it, set CaptureContent: true and provide SpanInput.Input. Built-in filters run before delivery; add a Redact callback for extra fields. See privacy and redaction. Call Flush(ctx) at request boundaries and Shutdown(ctx) after active requests finish. If the request was canceled, use a fresh bounded context for flushing. Stats() reports dropped events; delivery errors do not replace application errors. Set ComponentID to a real Bench prompt component to link events to its criteria. Automatic framework adapters are coming soon. Production checks work with accepted, linked events.

Test your application

client.EvaluateSystem(ctx, options) calls your real application with pinned JSON cases and returns a redacted report without uploading it.
Use bench.Expect(value) to assert an output/state, including explicit JSON null with bench.Expect(nil). Omitted pointers mean no assertion. For state checks, set ExpectedState and Observe func(context.Context, string) (any, error). It must read the authoritative test state independently of the final reply. JSON input numbers arrive as json.Number to preserve large identifiers. Pass the provided context into every nested tool/model trace and await all goroutines. client.SimulateSystem(ctx, bench.SimulationOptions{...}) takes the same cases and revisions plus CreateSession func(context.Context, any) (bench.SimulationSession, error). Use bench.SimulationInput{InitialState: ..., Turns: []any{...}} as case input and require ExpectedState. The returned session implements Turn, Observe and Close, each accepting context.Context. A fresh session handles 1 to 20 scripted turns; observed state is snapshotted before Close resets it. bench.EvaluationCaseID(ctx) identifies the current case. Timeout defaults to 30 seconds per case; set Timeout up to five minutes. Cancel the supplied context to stop. Missing assertions/observations, unfinished spans and timeouts are incomplete and fail Passed(). Go cannot forcibly stop an uncooperative goroutine. Use isolated test dependencies and cooperative callbacks. Cleanup gets a fresh five-second context, which it must honor. Real app tests record redacted content locally, even when production tracing is metadata-only. They do not upload reports or spend evaluation credits by implicitly running a judge. Use synthetic test data. Automatic framework adapters are coming soon. Run the complete refund simulation example. It sends two refund requests, checks the number of refunds written, and demonstrates the failing implementation and its fix.

Run in CI

Use Bench with your existing pull-request checks. See Run tests in CI for setup, report handling and the features available in each language.