What you’ll be able to do
- Apply chain-of-thought framing where it earns its cost in accuracy
- Choose few-shot examples that anchor the actual rule, not an incidental pattern
- Write output contracts explicit enough that “done” isn’t a guess
What you need to know
Chain-of-thought is a cost you pay for a specific benefit
Asking a model to reason step by step before concluding measurably improves accuracy on tasks with real multi-step logic — a multi-constraint scheduling problem, a piece of code with several interacting edge cases. It also costs tokens and latency, and on tasks that don't actually require multi-step reasoning it adds cost without adding accuracy.
A separate, architect-level point: reasoning that helps the model arrive at a better answer doesn't need to be shown to the end user. Treat the reasoning and the final response as two different outputs with two different audiences, and don't assume "the model thought about it" is itself useful display copy.
Few-shot examples teach whatever pattern is actually present
Two or three well-chosen examples anchor format and edge-case handling far more reliably than a paragraph describing the format in prose. But the model will learn whatever pattern is consistently present across the examples — including incidental ones you didn't intend to teach.
- If every example happens to be a short input with a short output, the model may learn "keep it short" as a rule, not just as this batch's coincidence.
- Vary examples along the dimension that shouldn't matter (length, subject, phrasing) and hold constant the dimension that should (the actual format and rule you're teaching).
- An edge case that's genuinely rare in production should still appear in the few-shot set if getting it right matters — the model has no other signal that it exists.
An explicit output contract beats an implicit one
"Just answer the question" leaves the model to infer format, length, and tone. An explicit contract states exactly what a correct response contains and excludes: "Respond with exactly one paragraph. No preamble. No meta-commentary about the task. No markdown formatting." Each sentence closes off a specific way real responses have gone wrong.
Standardize the technique, not the wording
The exam-relevant distinction: some prompt techniques generalize across an application and should become a documented, team-wide convention — always using tool-use for structured output, always stating an explicit output contract. Others were reverse-engineered to fix one specific prompt's specific failure and don't generalize; copy-pasting that exact phrasing into every new prompt "because it worked once" is superstition, not architecture. Treat prompt technique changes with the same versioning discipline as code changes, so a regression can be traced to the change that caused it.
Key concept
Chain-of-thought and few-shot examples are tools with real costs and specific failure modes — apply them where the task’s difficulty justifies the cost, and standardize the technique, not one prompt’s exact wording.
When a scenario asks "should every prompt use chain-of-thought," the correct answer keys off task difficulty and cost tolerance, not "more reasoning is always better."
Practice scenario
Work it through, then open this
The examples taught more than the intended routing rule — they also taught “tickets are short and English” as an incidental pattern, because that was consistently true across every example. The fix is to vary length and language across the few-shot set while keeping the actual routing logic constant, so the model can’t infer a spurious correlation between ticket length and category.
Build exercise — Separate a standardizable technique from a one-off fix
Intermediate · 20 min
What you’ll learn
- Distinguishing a generalizable prompt technique from prompt-specific superstition
- Auditing a few-shot set for incidental patterns
- Writing an explicit output contract
-
List every prompt technique currently in use across your application’s prompts and mark each as “standard convention” or “reverse-engineered for one case.”
- Why: Techniques in the second bucket often get copy-pasted into unrelated prompts where they add nothing or actively hurt.
- You should see: A shorter list of true conventions than you expected, and some techniques you can’t explain the original reason for.
-
Take one prompt’s few-shot examples and list every dimension that’s constant across them, not just the one you intended to teach.
- Why: Any accidentally-constant dimension is a pattern the model may learn as a rule.
- You should see: At least one incidental constant — length, tone, language, format — worth varying.
-
Rewrite one prompt’s ending to state an explicit output contract instead of an implicit “answer the question.”
- Why: Explicit contracts close off specific ways real responses have gone wrong; implicit ones leave the model guessing.
- You should see: A contract naming length, structure, and what to exclude — not just what to include.
Exam traps
Standardizing a technique that only worked because of one prompt’s specific wording
Copy-pasting a fix that was really specific to one prompt’s failure mode is superstition dressed as convention.
Choosing few-shot examples that all share an incidental pattern the model overfits to
The model learns whatever is consistently present, including patterns you didn’t intend to teach.
Requesting chain-of-thought reasoning purely as user-facing display copy
Reasoning that helps the model think and reasoning worth showing a user are two different things with two different audiences.
Leaving the output contract implicit instead of stating it explicitly
“Answer the question” leaves format, length, and tone to inference. State them.
Believing more examples always helps
Past a handful, additional examples mostly add cost and can reinforce an incidental pattern rather than the intended rule.
Not versioning prompt technique changes the way code changes are versioned
Without versioning, a regression from a prompt change is much harder to trace back to its cause.