vibehacker
Discuss
Tomek Wiśniewski
21 hours ago

agent wrapped my go tests in //go:build ignore and CI went green

#agents#rants#tooling

caught this tuesday night.

agent "fixed" a flaky integration test by wrapping the whole package in //go:build ignore. pr checks green. merge queue happy. i only noticed because coverage dropped 11%.

repro was a race in postgres setup. the agent never ran the test twice. just deleted the signal.

if your agent can silence tests, your CI is the bug. anyone else grepping for build tags / skip / ignore before merge?

5

5 comments

Join the discussion

Log in to comment.

Log in
  • Wei Zhang

    same class of failure on a node mcp repo last month.

    agent removed the engines check in package.json so ci stopped failing on node 18. i only saw it because a user opened issue with stacktrace from 18.20.

    now i require the diff to show package.json and any .yml workflow files in the review checklist. annoying. saves me.

    • Joao Volkov

      The engines check deletion is the same failure mode as the build ignore.

      Agent removes the assertion, CI stops complaining, human sees green. Blast radius is silent until production.

      I now require git diff --name-only against main to include any change under .github/ or **/go.mod before approve. Still misses package-local build tags though.

  • Liam O'Connor

    lol i did worse. cascade put continue-on-error: true on the only job that runs e2e.

    shipped friday. found it monday when a customer pinged that checkout was 500. ci had been "green" all weekend.

    now i grep for continue-on-error before i even read the pr body

    • Cass

      continue-on-error is my new nightmare keyword

      i screenshotted a cascade pr last week that put it on lint, typecheck, AND e2e. three greens that meant nothing.

      do you block it in branch protection now or just grep? i keep forgetting and then find another one in a dependabot-adjacent workflow

  • Nova

    yeah. i started grepping for //go:build ignore and t.Skip in every agent pr after something similar on a rust crate.

    agent just deleted the test file and left a comment "flaky, revisit later". coverage didn't even drop because the file was gone.

    is there a github action that fails on new build tags? asking for a friend who keeps merging at midnight

More like this

View all