Security and limits

Understand credentials, evaluation usage, and reviewable write actions.

Your account, your permissions

MCP authenticates to Bench with your account or a scoped Bench API key. A client cannot use MCP to bypass repository access, tenant isolation, key limits, or evaluation quotas.

Prefer hosted sign-in when your client supports it. For key-based setup, Bench automatically prepares an account setup key. It is encrypted at rest and can be copied again from your signed-in setup page. Additional keys created with New key are shown once. Keep credentials in personal client configuration or a secret manager. Share setup prompts only with a trusted coding agent; never commit keys or include them in logs, screenshots or public chats. A revoked setup key remains disabled until you explicitly replace it.

Local environments

Use the BENCH_API_BASE_URL supplied by the environment that created your key. A local key will not authenticate against the production API. Do not copy production credentials into a test account.

Actions with consequences

Action	Consequence
Account and result inspection	Reads information available to your credential
Repository scan	Reads authorized source code and stores discovery results
Start or resume evaluation	Uses evaluation allowance and model compute
Open prompt PR	Writes a branch/change and pull request in the connected repository

Ask your coding agent to get confirmation before spending evaluations or creating pull requests. A reviewable candidate is not an applied, deployed, or independently validated fix.

Context and trustworthy results

Business rules define intended behavior. Traces describe observed behavior; they are not automatically correct. Feedback on a single case must not silently become a rule for every case.

Saved evaluations retain their context and artifact references. New information informs a future bench, not a silent rewrite of historical scores. Unknown or unscored checks should remain visible and must not be treated as passed tests.

Revoke a connection

Revoke API keys in MCP → Manage keys and permissions. Remove the MCP entry from clients that no longer need access. If a key appears in a chat, screenshot, or repository, revoke it and issue a replacement.

Headless development access

Authenticated OAuth agents can manage account setup, GitHub connections, SDK keys, workspace membership and billing through the corresponding tools. Scoped API keys cannot mint credentials or change billing/team access. Organization administrator checks still apply. Source-scoped writes require ownership and access to every repository in the system; organization read access alone does not permit key-based edits.

Model/provider selection requires an active Growth or Enterprise plan. Quotas, case ceilings and key spending limits are enforced by the API on every transport. The agent must obtain user authorization before acknowledging source processing, spending evaluations, changing billing, inviting members or publishing code. GitHub permissions and Stripe payment/coupon confirmation remain provider-owned steps. See headless setup.