MS
Matt Smirnov
9 hours ago
jev scored cancel_sub 0.91 and stripe still charged twice
Jev
System One model for typed decisions with calibrated confidence
ran jev on a churn flow friday. typed decision came back cancel_sub at 0.91 so i wired it straight into the webhook handler.
stripe still fired invoice.paid twice because our idempotency key was the customer id, not the event id. jev was right about the intent. my glue code was not.
anyone actually gating the high-confidence path behind a dry-run queue, or are we all just vibing the floats?
5 comments
Join the discussion
Log in to comment.
repro on my side: confidence looks fine until the side effect isn't atomic. i put a "must see stripe event id once" assert before any cancel. jev doesn't own your webhook design.
same. i screenshot the stripe dashboard before any cancel path goes live now. jev can score 0.99 and i still want to see the event id once. glue code always loses.
0.91 means nothing if your retry loop can call it twice. we log the decision id + request hash and reject duplicates. saved us about $180 last month on accidental refunds.
yeah the decision id trick is the whole game. we hash
intent + invoice_id + window(5m)and drop the second call. confidence floats are just vibes until the side effect is once.we put anything over 0.85 into a dry-run queue with a human checkbox. still got double-charged once because the agent re-submitted after a 502 and our idempotency key was
customer_id + action. switched to stripe event id. boring. works.