We gave one model, DeepSeek V4.1 Flash, the same 40 questions about two real repositories and changed only the agent around it. The CodeAlive research harness scored 40% higher than OpenCode, and on the large repository it used 3.6 times fewer tokens.
Connecting CodeAlive's tools to OpenCode over MCP raised its score to 50.3 and cut its tokens on the large repository by 30%. The rest of the gain comes from the harness.
What we compared
In July we showed that a small Qwen model inside the CodeAlive harness can outscore Sonnet. This time the model stayed fixed.
The questions come from RepoContextBench v2: 20 on Microsoft Agent Framework (about 513K lines) and 20 on Sapling (about 1.5M lines). An LLM judge scores each answer from 0 to 100 against reference claims taken from the code. The model ran at reasoning effort max in every configuration.
We ran three agents:
- OpenCode, a general-purpose open-source coding agent, with its built-in file search and reading over a local checkout.
- The same OpenCode with CodeAlive's tools connected over MCP, and an
AGENTS.mdthat tells it to use them. - The CodeAlive harness: our research agent, built for answering questions about code, using the same tools directly.
Results
The score gap is 24.7 points on Agent Framework and 12.8 on Sapling. On Sapling the harness also did far less work:
OpenCode spent its steps grepping and reading whole files, and every step sent the growing history back to the model. The CodeAlive agent found the relevant symbols through the index and read only those. Over MCP, OpenCode used 30% fewer tokens than on its own, but it was slower: it kept its own loop and added CodeAlive calls on top.
Why the tools alone are not enough
Over MCP, OpenCode used CodeAlive's tools a lot: on Agent Framework alone it made 186 grep calls, 112 fetches and 74 semantic searches. Its score rose by 3.7 points. The harness, with the same tools, added 18.7.
The difference is in how the agent works with the tools:
- it starts from semantic search instead of walking directories;
- it reads by symbol:
fetch_artifactsreturns the function it asked for, with its callers and callees, not a 2,000-line file; - tool results end with a next-step hint, which helps smaller models most;
- repeated identical calls are caught before they eat the step budget;
- the agent ends the research explicitly once it has enough evidence.
OpenCode is built to edit code, run commands and read files in one loop. Our harness does one job, finding the right code and answering from it.
The savings depend on repository size
On Agent Framework the gain was in answer quality only, not in tokens. The harness used 398K tokens per task against OpenCode's 180K and took 50 seconds against 42. Both agents made about seven model calls per task, but ours starts with a larger prompt and gets larger tool results: whole symbols with their callers instead of a list of matching lines. On a short task that fixed cost dominates. It also bought the 24.7-point lead.
On Sapling OpenCode needed many more steps, its history grew with every file it read, and the balance flipped. The savings do not scale linearly with size: they appear once the repository is large enough that a general agent starts wandering through it.
Caveats
These are internal diagnostic runs, one per configuration, not a published leaderboard entry. A repeated identical run moved the score by about ±2 points, far below the gaps above, and moved token use by about 14%. The judge is an LLM, Muse Spark 1.3 at reasoning effort medium with two votes per task, using a fixed rubric.
Tokens are input plus output as reported by the provider, cached input included. Both OpenCode configurations reached the model through one gateway and the CodeAlive harness through another, so the share of cached input is only roughly comparable. The cost of building the CodeAlive index is not included; it is paid once per repository.
If you already use a coding agent, CodeAlive over MCP is the quickest way to give it repository context. In our runs the biggest gain came from handing the research itself to a dedicated agent. What the call graph adds is in GraphRAG for code, and how accurate the graph is in CodeAlive vs open-source code graph indexers.