Rust

Record Rust application requests and tools in Bench.

Copy this into your coding agent

Paste into Cursor, Codex, Claude Code, or another coding agent in your project.

Add Bench using https://docs.usebench.ai/sdk/rust and my Bench SDK setup prompt.
Use the install command on this page, plus my endpoint and server-side key. Start with metadata
only in staging. Wrap a real async operation and its tools, passing TraceContext
explicitly to child operations. Flush a synthetic interaction and confirm receipt
in Bench. Show changed files and remaining setup steps.

Beta release. All four languages support tracing, real app tests and scripted simulations.

Install

Package on crates.io · API reference · Source and examples

cargo add trybench-sdk@0.1.0

Record one interaction

Choose Rust on the SDK setup page. Use Rust 1.88 or later and a Tokio async application.

use trybench_sdk::{Bench, Options, SpanInput};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut options = Options::new(
        std::env::var("BENCH_API_KEY")?,
        std::env::var("BENCH_REPOSITORY")?,
        std::env::var("BENCH_BRANCH")?,
    );
    options.endpoint = std::env::var("BENCH_API_BASE_URL")?;
    options.environment = Some("staging".into());
    options.system_name = Some("Support agent".into());
    let bench = Bench::new(options)?;
    let result = bench.trace(
        None,
        SpanInput::new("connection-check").kind("AGENT"),
        |_context| async { Ok::<_, String>(true) },
    ).await;
    bench.shutdown().await;
    result?;
    Ok(())
}

Open the system’s Production tab to confirm receipt.

Wrap your application

Replace the callback with a Rig or custom application operation. Outputs must implement serde::Serialize. Pass Some(&context) to child traces and use .kind("TOOL") for tools. Context is explicit, so concurrent requests remain separate.

For streaming, keep a start_span guard open while reading the stream. Call set_output on success, set_error on failure, then end. Dropping an unfinished guard records an error without capturing the exception message.

Content capture defaults to off. Set capture_content = true and provide SpanInput::input(...) when needed. Built-in filters run before sending; use Options.redact to remove extra application fields. See privacy and redaction.

Await flush() at request boundaries and shutdown() after active requests finish. Delivery retries once, then counts the events in stats().dropped.

Set SpanInput.component_id to a real Bench prompt component for production checks. Automatic framework adapters are coming soon.

Test your application

bench.evaluate_system(options, application).await runs your application’s request handler on pinned JSON cases and returns a redacted local report.

use serde_json::json;
use trybench_sdk::{Application, EvaluationOptions, SystemCase};

let mut case = SystemCase::new("outside-refund-policy", json!({"days": 45}));
case.expected_output = Some(json!({"refunded": false}));
case.forbidden_tools = vec!["issue-refund".into()];
let options = EvaluationOptions::new(
    std::env::var("GIT_COMMIT_SHA")?, "refund-policy-v1", vec![case],
);
let app = Application::new(|input, context| async move {
    // Pass context.trace to your actual nested tool/model traces.
    handle_request(input, context).await
});
let report = bench.evaluate_system(options, app).await?;
assert!(report.passed());
// Explicit upload when desired:
bench.publish_system_evaluation(system_id, &report).await?;

Use Application::with_observer to read authoritative test state independently of the final reply, and case.expected_state to assert it. Some(Value::Null) is an explicit null assertion; None omits it. EvaluationContext provides the case ID, parent trace, is_cancelled() and cancelled().await.

bench.simulate_system(options, create_session).await accepts cases with input: {"initialState": {...}, "turns": [...]} and expected_state. The async factory receives initial state and evaluation context, returning SimulationSession::new(turn, observe, close). The turn callback accepts a JSON message and context. Observe/close callbacks take no arguments; share fixture state with Arc<Mutex<_>> or your test database. Bench snapshots state before close/reset, using a fresh session for each 1-to-20-turn scripted conversation.

The default per-case timeout is 30 seconds, configurable up to five minutes. Missing assertions/state, unfinished spans, capture errors and timeouts remain incomplete and fail passed(). Dropping the evaluation future cancels its task; callbacks must yield and await child work. This is not an operating-system sandbox. Session cleanup is attempted on failure or cancellation and is bounded to five seconds; an abruptly stopped Tokio runtime cannot finish asynchronous cleanup.

Real app tests record redacted content locally, even when production tracing is metadata-only. Use synthetic inputs and isolated test dependencies. Upload and paid production checks remain separate actions. Automatic framework adapters are coming soon.

Run the complete refund simulation example. It sends two refund requests, checks the number of refunds written, and demonstrates the failing implementation and its fix.

Run in CI

Use Bench with your existing pull-request checks. See Run tests in CI for setup, report handling and the features available in each language.