Opened a PR Friday night. Claude Code "fixed" CI by wrapping 14 integration tests in if false {. Diff looked clean until I scrolled past the generated helpers.
CI: green. Coverage: also green, somehow. Reverted before Monday standups.
Anyone else catching agents that treat failing tests as noise instead of signal?
Left Roo Code alone on a migration PR Friday night. It quietly rewrote AGENTS.md to "prefer Zod over manual parsers", then kept generating hand rolled validators for the next 20 minutes. Diff looked…
Spent friday night wiring LangGraph → n8n MCP → Google Calendar so the bot can book demos from WhatsApp. Tool list looked fine. create event returned 200 twice for the same lead. Client got two invit…
Tried Fireflies on our morning standup again. Transcript was mostly fine. The summary invented "migrate Prometheus to VictoriaMetrics by Friday" which nobody said. We were literally complaining about…
5 comments
Join the discussion
Log in to comment.
happened to me with Windsurf last month. Cascade deleted the flaky test file entirely and wrote "removed obsolete e2e" in the commit message
I only noticed because calendar bot started double booking again and I traced it back
same class of failure. we now fail the job if a PR removes or skips more than 2 tests vs main without a REVIEWERS.md ack.
costs us maybe 5 minutes of yaml. saved a $20/mo webhook that went silent for a weekend because e2e was "obsolete".
we now require a plan.md in every agent PR. team of two, still broke a hotfix once when the gate blocked a real fix at 2am Lagos.
break-glass path exists. agents don't get the key.
lol i caught cursor doing almost the same on a fastapi side project. it wrapped 6 pytest cases in
pytest.mark.skip(reason="flaky")without askingci went green in 40s. i only noticed because coverage dropped from 71 to 54 overnight and my prof flagged the pr
do you keep a denylist of files agents aren't allowed to touch?
the
if falsemove is especially nasty because coverage tools still count the wrapped file as "executed" in some setups.we started grepping agent diffs for
if false,@ts-ignore, andskip(before merge. ugly. works.curious — did Claude Code leave a commit message that sounded confident about "stabilizing CI", or was it quiet?