Code Extension
eve/extensions/code (eve-code) turns an eve agent into a coding agent. It ships inside the eve package, so you mount it without installing anything else. The extension adds coding instructions that steer the model toward eve's default file tools, the project's AGENTS.md, coding skills, and helpers that install coding tools in the sandbox. GitHub and pull request work lives in the companion eve/extensions/git extension.
Benchmark results
Lines are 95% intervals; overlapping lines are not a measured difference. Click a contender to open its run.
These results show baseline performance, with each harness in its standard setup and high reasoning. Other models, settings, or tasks can perform differently.
How the benchmark works
The benchmark uses eve-bench and the DeepSWE-lean dataset. DeepSWE-lean has 12 feature and bug-fix tasks from DeepSWE v1.1.
- All harnesses use the same model and the same tasks.
- Each harness does each task 3 times.
- Each attempt runs in a new Vercel Sandbox.
- Each attempt has a limit of 20 minutes.
- The DeepSWE verifier examines the files that the agent leaves.
- An attempt passes only when the verifier passes.
- A crash or a timeout is a failure.
The line on each correctness row shows the 95% interval. When the lines of two harnesses overlap, the benchmark does not show a difference between them.
How the results update
- The
eve-codeworkflow runs onmainwith thedeepswe-leandataset every Monday at 06:00 UTC. A maintainer can also start it manually onmain. - The workflow runs eve-code, opencode, pi, and fx on each benchmark model, one model at a time. It saves each complete result in the eve-bench result store.
- The workflow opens a pull request with a summary of the default Anthropic model's result. This page shows only that model.
- After a maintainer approves and merges the pull request, the next docs deployment shows the new results.
The workflow skips incomplete runs, single-task runs, and pull request runs. Results can be a week or more older than main, because the workflow runs weekly and each refresh waits for review. Select a harness name to open the GitHub Actions run for that result.
Mount the extension
Create an extension file in your agent. The file name sets the prefix of every skill the extension adds, such as code__investigate:
code({}) mounts the extension without Vercel authentication.
Configure the extension
| Option | Required | Effect |
|---|---|---|
vercel.connector | With vercel | Vercel Connect connector that gets a Vercel token for the app, not for a user |
vercel.delivery | No | "firewall" (default) or "command" |
broker | No | broker(sandbox, rules) callback that merges Vercel credential rules into your network policy |
Credentials stay outside the sandbox
With firewall delivery, Vercel tokens never enter sandbox processes. Firewall delivery needs a top-level broker or a sandbox provider with setNetworkPolicy(). For other providers, set vercel.delivery: "command".
To use a personal access token or another credential provider, omit the connector. Call authenticateGitHub or authenticateVercel from eve/extensions/code/sandbox in your own sandbox setup code. These helpers authenticate commands you run yourself, not the gh tool.
Project instructions
The extension reads AGENTS.md from the workspace root, or CLAUDE.md when there is no AGENTS.md, and adds it to the conversation as project instructions. The content joins the conversation again only when it changes. A file over 64,000 bytes is cut off with a note that tells the model where to read the rest.
The extension reads the file once per session, with one sandbox command at the start of the first turn, which starts the session's sandbox. Later edits to the file reach the model only when it reads the file itself. If the read fails, the turn continues without project instructions, and the next model call tries again. For other directories, the extension's instructions tell the model to read the nearest AGENTS.md or CLAUDE.md before working there.
Publish with the git extension
eve/extensions/git adds the GitHub instructions and the pr skill. Mount it next to the code extension when the agent should open pull requests. code() rejects a github option, so configure GitHub here. Import gh and the GitHub sandbox helpers from eve/extensions/git/tools and eve/extensions/git/sandbox:
git({}) adds no tool and assumes the agent's shell already has GitHub credentials. The agent uses the gh and git CLIs in bash, and git__pr commits, pushes, and opens a draft with gh pr create --draft.
Configure github to get GitHub tokens through Vercel Connect instead. The extension then adds the authenticated git__gh tool and tells the agent to use it for every authenticated GitHub operation. git__pr switches to gh-signed-commit for repositories that require verified signatures.
gh-signed-commit adds or updates non-executable regular files and deletes tracked entries. It rejects additions or updates involving executable files, symlinks, submodules, or file-mode changes before contacting GitHub. Your staged changes remain intact. For those changes, use native git commit -S with a configured signing key, then git push through the authenticated tool.
| Option | Required | Effect |
|---|---|---|
github.connector | With github | Connector that mints a GitHub token |
github.broker | With github | broker(sandbox, rules) callback that installs, then removes, the GitHub credential rules |
github.principalType | With github | "app" for GitHub App installation tokens, or "user" for the caller's own token |
github.authorize | With github | authorize(request) returns true to let a gh call run |
github.attribution | No | attribution(ctx) returns the current turn's { name, email } co-author, or null |
Each gh call runs one gh, git, or gh-signed-commit command. Processes in the sandbox see only a placeholder GH_TOKEN. The sandbox firewall replaces it with the real token on GitHub requests. Your github.broker callback receives the header rules before the command runs, then null after it to remove them. The callback must keep your other network rules. GitHub authentication has no "command" delivery option.
Control access with authorize
github.authorize is the agent's only access control. A command runs only when it returns true; otherwise the token never reaches the sandbox. It runs before every gh call and receives the parsed argv, the caller's auth, repository, and the minted token. What the token can reach depends on principalType:
principalType | Token | authorize also receives |
|---|---|---|
"app" | A GitHub App installation token for the declared repository, or for the whole installation when the call declares only an owner. | owner |
"user" | The caller's own token, which reaches everything the caller can reach on GitHub. eve asks the caller to sign in when needed and when GitHub rejects the token. | repository: null, no owner |
With "user", authorize always receives repository: null, even when the command declares a repository. Repository allowlists therefore deny user-token calls. Use "app" with a full owner/name declaration when your policy restricts repository access. Use "user" only when your policy permits the caller's full GitHub access, and authorize the caller through auth.
Credit the person behind a turn
The GitHub App is the author of commits that the gh tool makes. Return a verified user from
github.attribution to add that person's Co-authored-by trailer. eve adds it to
gh-signed-commit messages and to gh pr create bodies passed through --body or --body-file.
The callback receives the tool context, including ctx.session.auth.current and ctx.getToken.
Return null to omit attribution. eve keeps existing co-author trailers. Commands without a body
flag and direct gh api calls get no attribution.
For example, resolve the author through your application's user mapping:
For a custom GitHub command tool, eve/extensions/git/sandbox exports executeGitHubShell(input, config, dependencies) and githubShellApproval(input). executeGitHubShell parses the command, calls authorize, and installs the credential rules. dependencies supply the session ID and sandbox access. Use githubShellApproval as the tool's approval policy so invalid commands are still rejected.
Prepare the sandbox
Install the CLI tooling in your sandbox environment's prepare callback:
Preparation installs gh and wrappers for gh, vc, and gh-signed-commit. With the git extension's github configured, repositories that require verified signatures can use gh-signed-commit through the gh tool after staging changes.
eve rebuilds a prepared environment only when the sandbox file or environment options change. A change to an imported helper or to the eve version doesn't rebuild it.
Troubleshooting
| What you see | What to do |
|---|---|
eve-code firewall credential delivery requires a sandbox provider with mutable network policy (setNetworkPolicy). | Set vercel.delivery: "command", or pass a top-level broker callback. |
The current caller is not authorized to access <repository>. | Your github.authorize callback returned false for that caller and repository. Check the callback's rule. |
Invalid extension config: eve-code no longer ships a worker subagent; ... | Remove the worker option from code(). To delegate work, use the built-in agent tool. |
What to read next
- Extensions: mount, configure, and override extension packages
- Sandbox: environments, preparation, and network policy
- Subagents: delegate work to a child agent with the built-in
agenttool - Skills: how the model loads skills on demand