When an agent asks "who calls this method?", it trusts the answer. If the graph invents a caller, the agent reads the wrong code and builds its answer on it. If the graph misses one, the agent never learns that the caller exists. So before we put a relationship graph in front of agents, we wanted to know how often it is right.
This summer we measured that for our indexer and for the open-source code graph tools we could run on the same code.
How we measured
For each corpus we generated a reference graph with compiler tooling: the TypeScript Compiler API for TypeScript and a compiler-based index for C#. It contains the relationships the compiler resolves: calls, type references, inheritance, interface implementations and overrides.
Each relationship is identified strictly: the exact place in the source where the call happens, plus the exact declaration it resolves to. If a method calls validate() three times, those are three facts. If the indexer points a call at the wrong overload, it is a miss, even when the name matches.
Precision is the share of the indexer's relationships that the compiler confirms. Recall is the share of the compiler's relationships the indexer found.
Results
Across the three corpora, CodeAlive kept precision between 0.976 and 0.987. Recall varied more, from 0.85 on Playwright to 0.94 on Zod.
The competitor numbers are for DeusData/codebase-memory-mcp, the strongest tool we could score end to end. It does not store relationships the way the compiler does, so we matched its calls by a looser key: the line of the call and the line of the target. The comparison is therefore generous to it, and it still found only 40–50% of the calls, while 35–59% of the calls it reported did not match the compiler.
The other tools
colbymchenry/codegraph found many calls on Playwright (recall 0.72) but most of what it reported did not match the compiler (precision 0.12). Its class hierarchy was much better: extends P 0.86 · R 0.96, implements P 1.00 · R 0.95. We ran it on a slightly smaller Playwright file set (1,307 files), because that was the subset its parser could process cleanly.
vitali87/code-graph-rag publishes its own evaluations, and they are worth reading: Django P 0.977 · R 0.938, Zod v4 P 1.00 · R 0.75. Their key is coarser than ours. A call counts as correct if the calling file and the name of the called function match, so overloads and repeated calls collapse into one fact. These numbers cannot be put in the same column as the ones above. On FluentValidation we ran it ourselves: its interface implementations scored P 0.95 · R 0.62, and its call edges have no call location, so we could not score them strictly.
GitNexus built graphs for Zod and FluentValidation reliably: repeated runs produced identical graphs, about 26,000 edges for Zod. Its stored relationships do not include where a call happens, so two calls between the same pair of functions are one edge, and a strict score is not possible.
Where CodeAlive falls short
Recall on Playwright calls is 0.82, and interface implementations there scored only P 0.71. We track both against the same compiler references, and they are what we are working on now.
Speed is comparable rather than a strength. On FluentValidation, codebase-memory-mcp indexed in 1.5 seconds against 2.8 for CodeAlive. On Playwright, CodeAlive took 22 seconds to index and score, against 31 seconds for codebase-memory-mcp to index.
Caveats
The runs were made between 30 July and 10 August 2026, on pinned commits of each corpus and each tool. Three corpora in two languages is not a general ranking. The competitor keys are approximations of ours, as described above. A fairer comparison would run every tool on one frozen file list with one relationship format, and we plan to do that next.
Does a better graph make a better agent?
Not automatically. When we gave this graph to our research agent, it helped on a large repository and barely mattered on a medium one, and some models would not use it at all. That experiment is in GraphRAG for code. The harness around the tools mattered more than the graph itself: Same model, different harness.