vibehacker
News
DigitalOcean Blog ·

DigitalOcean Signals and Simulations turn production agent failures into multi-turn evals before you ship

Signals (public preview) watches Managed Agents and Inference Engine traffic for interaction, tool-execution, and infra failures, then one-click exports flagged sessions into Evaluations scenarios. Simulations replays those multi-turn conversations in Harness Runtime sandboxes against Claude Code, Codex, or OpenCode candidates so you can grade a fix before users see it.

More news

View all

Microsoft's Cabra benchmark finds code understanding, not edit volume, is what trips up coding agents

Across 6,840 synthetic call graph refactoring tasks, six agents in Copilot's harness stayed near perfect by scripting around size with tools like grep , while plain LLMs degraded; on SWE bench Verified traces, the count of read and analyze tool calls predicted failures better than lines edited. A new task that merges two near identical classes finally broke the agents, and code plus data are open at github.com/microsoft/CABRA…

Spotted something we missed? Start a thread.