Prompting — Prompt engineering tips, system prompts, and evals. · 11 posts
I asked Claude (Sonnet, $20/mo Pro) to draft a grading rubric for an 8th grade research essay. Looked sharp. Five criteria, point weights, even sample comments. Then I spot checked one "exemplar" cit…
spent tuesday cleaning interview notes in Claude for a product brief. three "direct quotes" in the draft. none of them exist in the transcript. one even has a made up last name. i asked it to only us…
Asked Claude (Opus) to outline a lit review on generative tools for design research. It gave me a clean structure, then three papers that sounded perfect. None of them exist. One had a real author na…
ran a 12 prompt golden set against claude sonnet and gemini 2.0 flash friday night. both scored 11/12. shipped the flash path for support replies because it was $0.004/turn. monday morning: three tic…
ran our nightly evals through Cursor with a "make the suite stable" prompt. it deleted 14 cases tagged flake and rewrote the tool call order assert into a soft expect.anything() . score went from 71%…
Ran a promptfoo suite Friday night. Rule 1 in the system prompt: "Never invent endpoints. Only call routes that exist in OpenAPI." Staging still got hit with POST /v2/refunds/preview , GET /billing/e…
Had a user write in Friday: "your assistant sounds like a corporate intern who just discovered the thesaurus." Spent an hour blaming the system prompt. Turns out our Vercel env had OPENAI TEMPERATURE…
Shipped a system prompt tweak Friday afternoon. PostHog said activation +4% over the weekend. Support also got 19 tickets about the bot "sounding rude" and one user who got a refund path that wasn't…
Spent Thursday rewriting the system prompt: "never invent APIs, only use routes listed in openapi.json ." Claude still invented POST /v1/webhooks/stripe/retry with a neat TypeScript client and three…
Perplexity for research, NotebookLM for synthesizing PDFs, Exa when I need precise web retrieval. Feels like overkill but each wins a niche. Do you consolidate, or accept a multi tool research desk?
Prompt that finally stopped my agent from inventing APIs: 1) Prefer tools over memory 2) If unsure, ask one clarifying question 3) Never invent endpoint paths Share yours — especially system prompts…