Benchmarks

Real-world and synthetic merge benchmarks. Reproduce with weave bench-repo <path>.

weave vs git merge

git merges lines. mergiraf merges tree nodes. weave merges entities. Read the full deep dive →

Scenario	git merge	mergiraf	weave merge
Two agents edit different functions	CONFLICT (adjacent lines)	auto-resolved	auto-resolved
One adds function, one modifies another	often conflicts	auto-resolved	auto-resolved
Both modify the same function identically	CONFLICT	auto-resolved	detected as identical, uses either
Both modify the same function differently	CONFLICT	CONFLICT	attempts 3-way merge on entity body
One deletes, one modifies	silent data loss possible	depends on context	modify/delete conflict reported
Both add functions at same position	CONFLICT	CONFLICT	auto-resolved (unordered entities)
Python: both add decorators to function	CONFLICT	CONFLICT	auto-resolved (decorator bundling)

Methodology

How the benchmarks work.

Clone a real repo

We pick major open-source repos with long merge histories: git/git (C), Flask (Python), CPython (C/Python), Go (Go), TypeScript (TS).

Walk merge commits

For each merge commit with two parents, extract the base (merge-base), ours (parent 1), theirs (parent 2), and human result (the merge commit itself).

Replay each file merge

For every file that both parents touched, run git's line-level merge and weave's entity-level merge on the same (base, ours, theirs) triple.

Compare against human

A win is when git conflicts but weave resolves cleanly. A regression is when git resolves cleanly but weave's output differs from the human result. Human match checks if weave's output is identical to what the developer wrote.

Summary

Note (0.5.3): this table was regenerated on the 0.5.3 merge engine on 2026-09-01, via weave bench-repo <path-to-clone> --limit 500 against fresh full clones of all five repos. Regressions rose across every repo (86 total, up from 3 on 0.5.2) — expected and by design: 0.5.3 now conflicts on genuinely divergent concurrent additions that 0.5.2 silently merged, and no longer resolves a case that could resurrect a deleted JSON key. Both changes move cases from “weave resolves” into “weave conflicts” under this benchmark's own regression definition (git resolves cleanly, weave doesn't). Wins also rose substantially on every repo.

Across 4,971 file merges from 5 repos, weave resolves 344 merges that git cannot, with 86 total regressions (up from 3 under 0.5.2) — see the per-repo breakdown below.

Repository	Language	Files Tested	Both Clean	Weave Wins	Both Conflict	Regressions	Human Match
git/git	C	1,701	1,224	183	271	23	72%
Flask	Python	67	34	15	17	1	33%
CPython	C / Python	256	191	11	44	10	45%
Go	Go	1,667	1,228	120	282	37	33%
TypeScript	TypeScript	1,280	1,156	15	94	15	53%

Synthetic benchmarks

31 hand-crafted merge scenarios across 7 languages. Run weave bench to reproduce.

Scenario	weave	mergiraf	git
Different functions modified	clean	clean	clean
Different class methods modified	clean	clean	clean
Both add different imports (TS)	clean	clean	CONFLICT
Class: different methods among 4	clean	clean	clean
One adds, other modifies	clean	clean	clean
Adjacent function changes	clean	clean	clean
Python: different class methods	clean	clean	clean
Python: adjacent methods (4-method class)	clean	clean	clean
Both add exports at end of file	clean	CONFLICT	CONFLICT
Reformat vs modify (whitespace-aware)	clean	clean	CONFLICT
Both add functions at end of file	clean	CONFLICT	CONFLICT
Both add methods to class at end	clean	clean	CONFLICT
Rust: both add different use statements	clean	clean	CONFLICT
Python: both add different imports	clean	clean	CONFLICT
Class: modify method + add new	clean	clean	clean
Both add functions between existing	clean	CONFLICT	CONFLICT
Python: both add different decorators	clean	CONFLICT	CONFLICT
Decorator + body change	clean	clean	clean
TS: class method decorators	clean	CONFLICT	CONFLICT
TS: interface field additions	clean	clean	CONFLICT
Rust: enum variant additions	clean	clean	CONFLICT
Java: different methods in same class	clean	clean	clean
Java: both add annotations	clean	clean	CONFLICT
C: different functions modified	clean	clean	clean
TS: method reorder + modification	clean	clean	clean
Python: both add class methods	clean	clean	CONFLICT
Rust: both add impl methods	clean	clean	CONFLICT
Dart: both add class methods	clean	clean	CONFLICT
TS: enum modify + add variant	clean	clean	clean
TS: add JSDoc + modify body	clean	clean	clean
Rust: both add doc comments to different fns	clean	clean	clean
Go: both add different functions	clean	clean	CONFLICT

weave: 31/31 clean (100%) vs mergiraf: 26/31 (83%) vs git: 15/31 (48%). Full benchmark suite runs in 11ms. Individual merges take 65-374µs. Entity extraction powered by sem-core.

git/git

The git source code itself. 1,701 file merges from 500 merge commits. Mostly C header and source files.

183

Wins

Regressions

72%

Human Match

40%

Resolution Rate

131 of 183 wins produce output identical to the human merge. Regressions (23, up from 0 on 0.5.2) are concentrated in a handful of files touched by many merges, e.g. object-file.c and setup.c — consistent with 0.5.3 now correctly conflicting on genuinely divergent concurrent edits that 0.5.2 silently merged.

Common win patterns: both branches add different extern declarations to a header, both branches add functions to different sections of a .c file, import block changes that git sees as overlapping lines.

Flask

Python web framework. 67 file merges from 500 merge commits. Highest resolution rate of all tested repos.

Wins

Regressions

33%

Human Match

47%

Resolution Rate

Flask's codebase is well-structured with clear function and class boundaries, making it ideal for entity-level merge. Nearly half of all git conflicts are resolved by weave. Common patterns: both branches modifying different methods in app.py, import additions to __init__.py. Flask's small sample size (67 file merges) means single cases move the human-match and regression percentages a lot; the one 0.5.3 regression is in tests/test_cli.py.

CPython

The Python interpreter. 256 file merges from 500 merge commits. Mix of C source and Python test files.

Wins

Regressions

45%

Human Match

20%

Resolution Rate

Human match rate improved from 29% to 45% under 0.5.3. Regressions (10, up from 0) land mostly in C source files with heavy macro and preprocessor use — e.g. unicodeobject.c and odictobject.c — where CPython's core devs often make genuinely divergent concurrent edits that 0.5.3 now correctly declines to auto-merge.

Go

The Go compiler and standard library. 1,667 file merges from 331 merge commits (go/go's master branch has fewer than 500 merge commits total; all of them were scanned).

120

Wins

Regressions

33%

Human Match

30%

Resolution Rate

Go's explicit structure (top-level functions, clear type declarations) works well with entity-level merge, and wins rose from 19 to 120 under 0.5.3. Regressions (37, up from 0) concentrate in high-churn runtime and compiler files — e.g. runtime/proc.go, runtime/malloc.go, and the SSA backend under cmd/compile/internal/ssa — where concurrent contributor edits are genuinely divergent, which 0.5.3 now conflicts on rather than silently merging.

TypeScript

The TypeScript compiler. 1,280 file merges from 500 merge commits.

Wins

Regressions

53%

Human Match

14%

Resolution Rate

The TypeScript compiler has very large files with complex entity relationships. Wins and regressions both rose under 0.5.3 (4→15 wins, 3→15 regressions). Several regressions land in bulk-generated tests/baselines/reference/*.js snapshot files from a single commit, alongside genuine source regressions in src/compiler/core.ts; the snapshot cases are lower-stakes than they look since they're machine-generated test fixtures, not hand-written source, but they're counted honestly in the total above.

Glossary

What the numbers mean.

Term	Definition
Files Tested	Number of individual file merges where both branches touched the same file (both-touched files across all merge commits).
Both Clean	Both git and weave merged cleanly. No conflict from either tool.
Win	Git produced a conflict, but weave resolved cleanly. A false conflict eliminated.
Both Conflict	Both git and weave produced conflicts. A real semantic collision that requires human judgment.
Regression	Git merged cleanly, but weave produced a different result than the human. Weave introduced an error where git was fine.
Human Match	Of the wins, how many produce output identical to what the developer actually wrote. Higher = weave's merge matches human intent.
Resolution Rate	Wins / (Wins + Both Conflict). What percentage of git's conflicts weave eliminates.

Reproduce

Run the benchmarks yourself.

run benchmarks

# Clone a repo
$ git clone --bare https://github.com/git/git /tmp/git-bench

# Run benchmark (scans up to 500 merge commits)
$ weave bench-repo /tmp/git-bench

# Show diffs for non-matching cases
$ weave bench-repo /tmp/git-bench --show-diff

# Save base/ours/theirs/human/weave for each case
$ weave bench-repo /tmp/git-bench --save /tmp/weave-bench/git