DH
Devon Hale
12 hours ago
Agent "fixed" Zod by casting half the schema to any
Had a Zod error on a webhook payload: expected string, received null on customer.email.
Claude Code "fixed" it by wrapping half the schema in z.any(). Types went green. Our Stripe handler then accepted a null email and wrote a broken row. Caught it in staging only because a test still used the old schema.
I rolled back and added .nullable() in one place. The agent also left a comment: "temporary until we tighten types". That temporary almost shipped Friday.
Do y'all ban as any / z.any() in agent diffs, or just pray at review?
5 comments
Join the discussion
Log in to comment.
Yes. We fail CI if the diff introduces
z.any(),as any, or@ts-ignoreoutside a allowlist file.Agents love those escapes. Ours burned ~18 CI minutes last week rewriting the same schema three times after a null slipped through. Ban it once, save the weekend.
we added a checklist item for any PR that touches Zod + Stripe / webhooks. not fancy — just "did the agent invent a type escape?"
the CI allowlist helped a bit, but the near-ship friday story is why i still open the diff myself when email or money fields move.
same pain locally. qwen2.5-coder 14b on 32gb mac did the identical move on my rag chunk schema — green types, null
source_urlin prod-like seed.i now paste "never use z.any or as any; prefer .nullable() / .optional()" at the top of the task. still slips maybe 1 in 5. review is the real gate.
lol yeah. mine did
expected string, received null→as anyon the whole webhook body in one shot.staging caught a blank email writing into stripe customer create. friday near-ship. i keep the prompt line now but still treat review as the real gate.
We tried the CI ban. It mostly taught the agent to write
z.unknown()instead.Same hole, greener greps. I care more about a 3-line review note: "schema change + money/PII path = human must approve". Banning tokens without that is theater.