Engineering Practices

Prompt Engineering Patterns That Actually Work in Production

Seven battle-tested prompt engineering patterns for production AI systems — with real examples, failure modes, and implementation guidance.

Algoritmo Lab · 10 min read · November 2025

Most prompt engineering guides teach you how to talk to ChatGPT. They show you how to phrase a question, add context, and coax a better answer out of a language model. That is useful for personal productivity, but it has almost nothing to do with what production prompt engineering looks like. This article is about something different: designing prompts that run thousands of times a day inside production systems, where no human reviews the output before it reaches a customer, updates a database, or triggers a downstream workflow.

When you move from playground experimentation to production deployment, the rules change entirely. You are no longer optimising for a single impressive response. You are optimising for consistency across tens of thousands of varied inputs, graceful handling of edge cases, and predictable failure modes that your monitoring can catch. The prompt becomes a piece of software infrastructure, and it deserves the same engineering rigour you would apply to any critical code path.

Why Production Prompting Is Different

In a playground, you read every output. You can rephrase, add context, and try again if something looks off. In production, that feedback loop disappears. Your prompt receives messy, varied, sometimes adversarial input, and it must produce structured, valid, actionable output every single time — or fail in a way your system can detect and recover from.

The differences are stark. In a playground, you optimise for the best possible single response. In production, you optimise for the worst-case response across a million calls. In a playground, you iterate by reading outputs. In production, you iterate by analysing logs, monitoring structured metrics, and running regression tests against evaluation datasets. In a playground, a creative but slightly wrong answer is fine. In production, a consistently correct but boring answer is far more valuable.

Consider the difference in practice. A playground prompt might say: "Classify this email as spam or not spam and explain why." A production prompt specifies the exact output format, enumerates the classification categories, provides boundary examples, includes instructions for ambiguous cases, and defines what the model should do when it is uncertain. The production prompt is longer, more rigid, and less conversational — but it produces output that downstream systems can reliably parse and act on.

The Seven Patterns

Over the past two years of building production AI systems, we have identified seven prompt engineering patterns that consistently deliver reliable results. Each pattern addresses a specific challenge in production systems, and most real-world applications combine several of them. For each pattern, we cover what it does, when to use it, a concrete example, the most common failure mode, and how to fix it.

1. Structured Output Enforcement

What it does: Forces the model to return data in a strict, machine-readable schema — typically JSON or XML — so that downstream systems can parse the output without guessing. This is the most fundamental production pattern because virtually every production system needs to extract structured data from model responses.

When to use it: Whenever your model output feeds into another system — a database write, an API call, a UI component, or a decision tree. If a human is not reading the raw output, you need structured output enforcement.

Example: A lead qualification system that receives inbound emails and classifies them for a sales team.

Common failure mode: The model wraps the JSON in markdown code fences (```json ... ```) or adds a preamble like "Here is the classification:" before the JSON. Your JSON parser breaks.

Fix: Add explicit instructions to suppress markdown formatting. Better yet, use native structured output APIs (like OpenAI's JSON mode or Anthropic's tool-use responses) which guarantee valid JSON at the API level. Always implement a parsing layer that strips common wrapper patterns as a fallback.

2. Chain-of-Thought with Hidden Scratchpad

What it does: Allows the model to reason step by step inside designated XML or delimiter tags, then produce a clean, structured final answer in a separate tag. You parse only the answer tags, discarding the reasoning. This gives you the accuracy benefits of chain-of-thought prompting without exposing internal reasoning to end users.

When to use it: Classification tasks, routing decisions, and any scenario where the model needs to weigh multiple factors before producing an output — but you only want the final decision downstream.

Example: A customer support routing system that assigns tickets to the right team.

Common failure mode: Internal reasoning leaks into the final answer, or the model puts the answer outside the designated tags. Users see raw chain-of-thought text in the UI.

Fix: Use robust XML parsing that extracts only content between your answer tags. Add post-processing validation that rejects any response where the answer tags are missing or malformed. Log the full response (including reasoning) for debugging, but only forward the parsed answer downstream.

3. Few-Shot with Boundary Examples

What it does: Provides the model with example input-output pairs that include not just typical cases, but deliberately tricky edge cases and examples of what NOT to do. Standard few-shot shows the model the happy path. Boundary few-shot shows it the cliffs on either side.

When to use it: Content moderation, sentiment analysis, and any classification task where the boundary between categories is fuzzy and context-dependent.

Common failure mode: Recency bias — the model over-indexes on whatever example appears last in the list, skewing predictions toward that category.

Fix: Randomise the order of few-shot examples on each call. Include roughly equal numbers of examples per category. Test your prompt against a held-out evaluation set to measure whether category distribution is skewed.

4. Retrieval-Augmented Prompting (RAG-Lite)

What it does: Injects relevant context from your knowledge base into the prompt at runtime, so the model answers based on your actual data rather than its training knowledge. Unlike a full RAG pipeline, this can start as simply as fetching the right document chunk and pasting it into the prompt.

When to use it: Policy Q&A bots, internal knowledge assistants, customer-facing support agents — anywhere the model needs to answer questions about information that changes over time or is specific to your organisation.

Common failure mode: The model hallucinates beyond the provided context, confidently generating plausible-sounding answers that are not grounded in any retrieved document. This is especially dangerous because the answers sound authoritative.

Fix: Add an explicit grounding instruction ("use ONLY the context provided"). Implement a confidence gate: ask the model to rate its confidence that the answer is supported by the context, and route low-confidence responses to a human reviewer. Require the model to cite specific passages from the context to make grounding verifiable.

5. Defensive Prompting (Input Validation)

What it does: Instructs the model to detect and reject off-topic, adversarial, or malformed input before attempting to process it. Think of it as input validation at the prompt level — the same way you would validate API request bodies before processing them.

When to use it: Any user-facing AI system, especially those with access to tools, data, or actions. Essential for HR policy assistants, financial advisors, and any system where a wrong answer has real consequences.

Common failure mode: Prompt injection — a user crafts input that tricks the model into ignoring its system instructions, potentially revealing internal prompts or performing unintended actions.

Fix: Use system prompt separation (system vs. user message roles in the API). Sanitise user input by escaping or removing common injection patterns. Layer defences: combine prompt-level instructions with application-level input filtering. Never rely on the prompt alone for security.

6. Prompt Chaining (Decomposition)

What it does: Breaks a complex task into a sequence of simpler, focused prompts — each with a single responsibility. The output of one prompt feeds into the next. Instead of asking one prompt to do everything, you build a pipeline of specialised prompts.

When to use it: Contract review, document analysis, multi-step reasoning tasks — anywhere a single prompt would need to juggle too many objectives simultaneously. The rule of thumb: if your prompt has more than three distinct objectives, decompose it.

Common failure mode: Error propagation — a mistake in Step 1 cascades through the entire chain. If the extraction misses a key clause, the flagging step cannot catch it, and the summary will be incomplete.

Fix: Add validation between each step. After Step 1, verify the extracted JSON against a schema and check for completeness (e.g., minimum number of clauses expected). Use confidence scores and route uncertain extractions for human review before they enter the next step. Log intermediate outputs for debugging.

7. Self-Evaluation and Retry Loops

What it does: Uses a second model call (or the same model with a different prompt) to evaluate the quality of the first response. If the evaluation fails, the system retries with adjusted instructions. This creates an automatic quality gate without human intervention.

When to use it: High-stakes outputs like customer-facing content generation, financial summaries, and medical information — anywhere the cost of a bad output justifies the additional latency and API spend of a second call.

Common failure mode: The model cannot reliably catch its own hallucinations. If the first pass fabricated a feature, the evaluation pass may accept it because it "sounds right" — the same knowledge gap exists in both calls.

Fix: Combine self-evaluation with retrieval-based fact-checking. Provide the evaluation prompt with the original source material so it can verify claims against ground truth, not just assess coherence. Use a different model or temperature setting for the evaluation pass to reduce correlated errors.

Putting It Together: A Real-World Example

To see how these patterns combine in practice, consider a customer support agent that handles tier-one enquiries for a SaaS company. The agent needs to understand the customer's question, find relevant information in the knowledge base, generate an accurate response, and decide whether to escalate to a human. Here is how the five-layer architecture works.

Layer 1 — Defensive Prompting: The first prompt validates the incoming message. Is it a genuine support request, or is it off-topic, abusive, or an injection attempt? Off-topic messages get a polite redirect. Abusive messages get flagged. Injection attempts get logged and blocked.

Layer 2 — Chain-of-Thought Routing: Valid requests pass to a routing prompt that uses hidden scratchpad reasoning to classify the ticket by product area, urgency, and complexity. The reasoning stays internal; the output is a clean routing decision in JSON.

Layer 3 — RAG-Lite Retrieval: Based on the routing decision, the system retrieves relevant knowledge base articles, recent similar tickets, and any account-specific context. This context is injected into the generation prompt.

Layer 4 — Structured Output Generation: The generation prompt produces a customer-facing response along with structured metadata: confidence score, referenced articles, and a boolean escalation flag. The output follows a strict JSON schema.

Layer 5 — Self-Evaluation: Before sending, a quality-check prompt reviews the response against the retrieved context. Does the answer contradict the knowledge base? Is the tone appropriate? Is the confidence score justified? Responses that fail evaluation get retried once, then escalated to a human agent.

This five-layer architecture handles the vast majority of tier-one tickets automatically while maintaining quality standards that match or exceed human agents. The key is that each layer has a single, well-defined responsibility and a clear failure mode. When something goes wrong, you know exactly which layer failed and why.

Patterns We Deliberately Left Out

You will find many "prompt engineering tips" online that we intentionally excluded. Here is why.

"Just add more context" does not scale. Stuffing everything into one massive prompt increases cost, latency, and the chance of the model losing focus. In production, you want the minimum effective context for each call. That is what retrieval-augmented prompting and prompt chaining solve — they get the right context to the right prompt at the right time, instead of dumping everything into one call and hoping for the best.

Temperature tuning as a strategy is overrated for production. Yes, temperature affects output variability, but it is a blunt instrument. If your prompt produces inconsistent output at temperature 0.7, the fix is rarely to lower the temperature — it is to improve the prompt structure, add better examples, or enforce a stricter output schema. Temperature is a dial you turn after everything else is solid.

"Be creative" or "think outside the box" is the opposite of what production systems need. You want your production prompts to be as boring and predictable as possible. Creativity is a feature in content generation, but in most production use cases — classification, extraction, routing, validation — it is a bug. If your system is being "creative" with its routing decisions, you have a problem.

The seven patterns in this article are not the only ones that work, but they are the ones we reach for most often when building systems that need to run reliably at scale. They compose well, fail predictably, and — critically — they give you the observability you need to debug problems when they inevitably occur. Production prompt engineering is not about writing the perfect prompt. It is about building a system of prompts that degrades gracefully and improves iteratively.