Study: a coding agent's harness can triple cost per task without improving accuracy
Shandong University researchers held models fixed and ran Claude Code, OpenCode, and the ~100-line mini-SWE-agent on 492 SWE-bench Verified tasks, finding statistically equivalent success rates but up to 3x differences in cost per task. On the 45 hardest tasks, switching harnesses flipped results about as often as simply rerunning the same setup (13%), so small single-run benchmark gaps say little.
