← 输入框|eve.dev|刷新

Code Extension

eve/extensions/code (eve-code) turns an eve agent into a coding agent. It ships inside the eve package, so you mount it without installing anything else. The extension adds coding instructions that steer the model toward eve's default file tools, the project's AGENTS.md, coding skills, and helpers that install coding tools in the sandbox. GitHub and pull request work lives in the companion eve/extensions/git extension.

Benchmark results

deepswe-lean@v1 · claude-sonnet-5.54 contenders across 1 run
higher is better
0%25%50%75%100%
correctnessvs ref
fx@0.0.16
78%+8 pp
pi@1.5.0
67%−3 pp

Lines are 95% intervals; overlapping lines are not a measured difference. Click a contender to open its run.

These results show baseline performance, with each harness in its standard setup and high reasoning. Other models, settings, or tasks can perform differently.

How the benchmark works

The benchmark uses eve-bench and the DeepSWE-lean dataset. DeepSWE-lean has 12 feature and bug-fix tasks from DeepSWE v1.1.

  • All harnesses use the same model and the same tasks.
  • Each harness does each task 3 times.
  • Each attempt runs in a new Vercel Sandbox.
  • Each attempt has a limit of 20 minutes.
  • The DeepSWE verifier examines the files that the agent leaves.
  • An attempt passes only when the verifier passes.
  • A crash or a timeout is a failure.

The line on each correctness row shows the 95% interval. When the lines of two harnesses overlap, the benchmark does not show a difference between them.

How the results update
  1. The eve-code workflow runs on main with the deepswe-lean dataset every Monday at 06:00 UTC. A maintainer can also start it manually on main.
  2. The workflow runs eve-code, opencode, pi, and fx on each benchmark model, one model at a time. It saves each complete result in the eve-bench result store.
  3. The workflow opens a pull request with a summary of the default Anthropic model's result. This page shows only that model.
  4. After a maintainer approves and merges the pull request, the next docs deployment shows the new results.

The workflow skips incomplete runs, single-task runs, and pull request runs. Results can be a week or more older than main, because the workflow runs weekly and each refresh waits for review. Select a harness name to open the GitHub Actions run for that result.

Mount the extension

Create an extension file in your agent. The file name sets the prefix of every skill the extension adds, such as code__investigate:

agent/extensions/code.ts
import code from "eve/extensions/code";

export default code({});

code({}) mounts the extension without Vercel authentication.

Configure the extension

OptionRequiredEffect
vercel.connectorWith vercelVercel Connect connector that gets a Vercel token for the app, not for a user
vercel.deliveryNo"firewall" (default) or "command"
brokerNobroker(sandbox, rules) callback that merges Vercel credential rules into your network policy
agent/extensions/code.ts
import code from "eve/extensions/code";

export default code({
  vercel: { connector: "vercel/acme-bot" },
});

Credentials stay outside the sandbox

With firewall delivery, Vercel tokens never enter sandbox processes. Firewall delivery needs a top-level broker or a sandbox provider with setNetworkPolicy(). For other providers, set vercel.delivery: "command".

To use a personal access token or another credential provider, omit the connector. Call authenticateGitHub or authenticateVercel from eve/extensions/code/sandbox in your own sandbox setup code. These helpers authenticate commands you run yourself, not the gh tool.

Project instructions

The extension reads AGENTS.md from the workspace root, or CLAUDE.md when there is no AGENTS.md, and adds it to the conversation as project instructions. The content joins the conversation again only when it changes. A file over 64,000 bytes is cut off with a note that tells the model where to read the rest.

The extension reads the file once per session, with one sandbox command at the start of the first turn, which starts the session's sandbox. Later edits to the file reach the model only when it reads the file itself. If the read fails, the turn continues without project instructions, and the next model call tries again. For other directories, the extension's instructions tell the model to read the nearest AGENTS.md or CLAUDE.md before working there.

Publish with the git extension

eve/extensions/git adds the GitHub instructions and the pr skill. Mount it next to the code extension when the agent should open pull requests. code() rejects a github option, so configure GitHub here. Import gh and the GitHub sandbox helpers from eve/extensions/git/tools and eve/extensions/git/sandbox:

agent/extensions/git.ts
import git from "eve/extensions/git";

export default git({});

git({}) adds no tool and assumes the agent's shell already has GitHub credentials. The agent uses the gh and git CLIs in bash, and git__pr commits, pushes, and opens a draft with gh pr create --draft.

Configure github to get GitHub tokens through Vercel Connect instead. The extension then adds the authenticated git__gh tool and tells the agent to use it for every authenticated GitHub operation. git__pr switches to gh-signed-commit for repositories that require verified signatures.

gh-signed-commit adds or updates non-executable regular files and deletes tracked entries. It rejects additions or updates involving executable files, symlinks, submodules, or file-mode changes before contacting GitHub. Your staged changes remain intact. For those changes, use native git commit -S with a configured signing key, then git push through the authenticated tool.

OptionRequiredEffect
github.connectorWith githubConnector that mints a GitHub token
github.brokerWith githubbroker(sandbox, rules) callback that installs, then removes, the GitHub credential rules
github.principalTypeWith github"app" for GitHub App installation tokens, or "user" for the caller's own token
github.authorizeWith githubauthorize(request) returns true to let a gh call run
github.attributionNoattribution(ctx) returns the current turn's { name, email } co-author, or null

Each gh call runs one gh, git, or gh-signed-commit command. Processes in the sandbox see only a placeholder GH_TOKEN. The sandbox firewall replaces it with the real token on GitHub requests. Your github.broker callback receives the header rules before the command runs, then null after it to remove them. The callback must keep your other network rules. GitHub authentication has no "command" delivery option.

Control access with authorize

github.authorize is the agent's only access control. A command runs only when it returns true; otherwise the token never reaches the sandbox. It runs before every gh call and receives the parsed argv, the caller's auth, repository, and the minted token. What the token can reach depends on principalType:

principalTypeTokenauthorize also receives
"app"A GitHub App installation token for the declared repository, or for the whole installation when the call declares only an owner.owner
"user"The caller's own token, which reaches everything the caller can reach on GitHub. eve asks the caller to sign in when needed and when GitHub rejects the token.repository: null, no owner

With "user", authorize always receives repository: null, even when the command declares a repository. Repository allowlists therefore deny user-token calls. Use "app" with a full owner/name declaration when your policy restricts repository access. Use "user" only when your policy permits the caller's full GitHub access, and authorize the caller through auth.

agent/extensions/git.ts
import git from "eve/extensions/git";

import { brokerGitHubLease } from "../lib/github";

export default git({
  github: {
    connector: "github/my-agent",
    broker: brokerGitHubLease,
    principalType: "app",
    authorize: ({ owner }) => owner.toLowerCase() === "acme",
  },
});

Credit the person behind a turn

The GitHub App is the author of commits that the gh tool makes. Return a verified user from github.attribution to add that person's Co-authored-by trailer. eve adds it to gh-signed-commit messages and to gh pr create bodies passed through --body or --body-file. The callback receives the tool context, including ctx.session.auth.current and ctx.getToken. Return null to omit attribution. eve keeps existing co-author trailers. Commands without a body flag and direct gh api calls get no attribution.

For example, resolve the author through your application's user mapping:

agent/extensions/git.ts
import git from "eve/extensions/git";
import { brokerGitHubLease, coAuthorFor } from "../lib/github";

export default git({
  github: {
    connector: "github/my-agent",
    broker: brokerGitHubLease,
    principalType: "app",
    authorize: ({ owner }) => owner.toLowerCase() === "acme",
    attribution: (ctx) => coAuthorFor(ctx.session.auth.current),
  },
});

For a custom GitHub command tool, eve/extensions/git/sandbox exports executeGitHubShell(input, config, dependencies) and githubShellApproval(input). executeGitHubShell parses the command, calls authorize, and installs the credential rules. dependencies supply the session ID and sandbox access. Use githubShellApproval as the tool's approval policy so invalid commands are still rejected.

Prepare the sandbox

Install the CLI tooling in your sandbox environment's prepare callback:

agent/sandbox.ts
import { defineSandbox } from "eve/sandbox";
import { VercelSandbox } from "eve/sandbox/vercel";
import { installCodeTooling } from "eve/extensions/code/sandbox";

export const environment = VercelSandbox.environment({
  prepare: async (sandbox) => {
    await installCodeTooling(sandbox, { vercel: true });
  },
});

export default defineSandbox(() => environment.open());

Preparation installs gh and wrappers for gh, vc, and gh-signed-commit. With the git extension's github configured, repositories that require verified signatures can use gh-signed-commit through the gh tool after staging changes.

eve rebuilds a prepared environment only when the sandbox file or environment options change. A change to an imported helper or to the eve version doesn't rebuild it.

Troubleshooting

What you seeWhat to do
eve-code firewall credential delivery requires a sandbox provider with mutable network policy (setNetworkPolicy).Set vercel.delivery: "command", or pass a top-level broker callback.
The current caller is not authorized to access <repository>.Your github.authorize callback returned false for that caller and repository. Check the callback's rule.
Invalid extension config: eve-code no longer ships a worker subagent; ...Remove the worker option from code(). To delegate work, use the built-in agent tool.
  • Extensions: mount, configure, and override extension packages
  • Sandbox: environments, preparation, and network policy
  • Subagents: delegate work to a child agent with the built-in agent tool
  • Skills: how the model loads skills on demand