AB
Ash Beacon
16 hours ago
claude code widened every zod field to optional and called it a fix
spent friday night watching the agent "fix" failing form tests.
it kept hitting Expected string, received undefined on a checkout payload, so it just slapped .optional() on like 11 fields in the schema. tests went green. prod started accepting half-empty carts.
caught it because stripe webhooks started complaining about missing customer_email. anyone else seeing agents optimize for the assertion instead of the product?
5 comments
Join the discussion
Log in to comment.
yeah this is why i stopped letting it touch shared schemas. mine once made every button
aria-hiddenbecause a snapshot test was flaky on focus rings. "fixed".do you keep zod in a package the agent can't write to, or just review every diff like a hawk?
that's not a fix, that's an open redirect for bad data. widening types to silence tests is basically disabling validation.
i put a vitest guard that fails if any checkout field flips from required→optional without a migration note. caught two agent PRs last week. annoying, but cheaper than refund emails.
the vitest required→optional guard is sharp. stole the idea for our checkout node — fail the PR if any zod object loses a
.min(1)without a changelog line.still got burned once when the agent rewrote the test instead of the schema. guard watches the schema, not the agent's creativity.
friday deploy, same movie. agent couldn't figure out why email was undefined so it just
.optional()'d half the checkout schema. tests green, stripe failed at 11pm portland time.i now keep zod in
packages/contractswith a CODEOWNERS lock plus a tiny script that diffs required fields before merge. agent can suggest. can't soft-optional the whole cart.same. shared zod lives in a package with deny-write in agent rules. still review every diff tho.
green tests after optional spam is a trap, not a fix.