Support said our bot sounded like a stiff intern — temperature was 0
Had a user write in Friday: "your assistant sounds like a corporate intern who just discovered the thesaurus."
Spent an hour blaming the system prompt. Turns out our Vercel env had OPENAI_TEMPERATURE=0 from a debug session last month and I never flipped it back. Model was technically correct and somehow still annoying.
Set it to 0.4, redeployed, same prompt. Tickets stopped. Feeling extremely dumb.
Anyone else ship a "prompt fix" that was just a leftover env flag?
5 comments
Join the discussion
Log in to comment.
lol this is exactly my vibe. i once "fixed" a chatty bot by rewriting half the system prompt only to find
temperature=0in a secrets.json that wasnt even in git.do you log the resolved config on boot now? thinking of dumping model + temp into a /healthz just so friday-me cant lie to monday-me.
yeah this bit me too. we had OPENAI_TEMPERATURE in vercel "preview" and somehow it leaked into production via a shared env group.
logging model+temp on /healthz is now a friday checklist item. feels dumb until it saves you.
We measured this on a support corpus last spring. Same system prompt, temp 0 vs 0.35 — mean "sounds human" score went from 2.1 to 3.4 on a 5-pt rater panel (n=40 tickets).
Temp 0 is fine for classification. Terrible for chat tone. Your "stiff intern" diagnosis tracks.
Pin it in code, not env.
Env flags get "temporarily" set during a 2am debug and then live forever. Also print
model+temperatureon every request in staging — caught us once when a preview deploy silently fell back to the platform default.Hard agree on pinning in code. We still keep a soft override for demos (
DEMO_TEMP=0.7) but the default lives in the client init, not Vercel.Filed a Linear ticket titled "stop letting 2am debug flags become product personality" after the last incident. Still open.