Reasoning Models Guide — OpenAI o1, o3, and Claude Thinking (2026)

Standard LLMs generate tokens left-to-right — each token predicted from the previous ones, with no backtracking. That works for most tasks. But when the problem requires multi-step logic, mathematical reasoning, or verifiable correctness, standard models hit a ceiling. Reasoning models break through that ceiling by allocating extra compute to “think” before answering. If you build LLM-powered systems in production, understanding when and how to use reasoning models is now a required skill.

Who this is for:

• GenAI engineers who need to decide whether a task justifies the cost and latency of a reasoning model or should stay on a standard LLM.
• Backend engineers integrating LLM APIs who want to understand the practical differences between calling o3, Claude with extended thinking, and GPT-4o.
• Senior engineers preparing for interviews — reasoning model architecture and routing decisions appear in system design rounds at staff level and above.
• Architects designing multi-model pipelines who need to reason about cost, latency, and accuracy trade-offs at each routing decision.

Reasoning models represent a fundamental shift in how LLMs handle hard problems. Standard models like GPT-4o and Claude Sonnet generate responses in a single forward pass — fast and cheap, but limited by the model’s ability to “think” in one shot.

Reasoning models add a thinking phase before the final answer. The model generates internal chain-of-thought tokens, checks intermediate steps, and revises its approach — all before producing the visible response. This is not the same as prompting a standard model to “think step by step.” When you prompt chain-of-thought, the model is still generating left-to-right in a single pass, just with more verbose output. Reasoning models have been trained specifically to use this extended thinking phase productively.

Before reasoning models, GenAI engineers had two options for hard problems:

1. Better prompting — techniques like chain-of-thought, few-shot examples, and advanced prompting strategies that squeeze more capability from standard models.
2. Agentic workflows — breaking the problem into smaller steps using multi-agent patterns where each step is handled by a separate LLM call.

Both approaches still work. But reasoning models offer a third option: a single API call that handles multi-step problems natively. This simplifies architectures and, for certain problem types, produces more reliable results than chaining multiple standard model calls.

Reasoning models consistently outperform standard models on tasks with verifiable correctness:

Benchmark scores are approximate ranges as of early 2026. Check provider announcements for current numbers.

The pattern is clear: reasoning models shine on hard, structured problems. They add no value for easy ones. This gap determines when you should use them.

Standard prompting techniques fail on a specific class of problems. Recognizing these failure modes is the first step toward knowing when to reach for a reasoning model.

Standard models generate tokens sequentially without the ability to go back and correct mistakes. This creates predictable failure modes:

Multi-step math and logic: Ask GPT-4o to solve a problem that requires five sequential reasoning steps, and it frequently gets step 3 or 4 wrong — then confidently builds on the wrong intermediate result. The error compounds through remaining steps.

Code generation with dependencies: Ask a standard model to write a function that depends on the correct implementation of three helper functions, and it often generates plausible-looking code where one helper has a subtle bug that makes the whole system incorrect.

Constraint satisfaction: Ask a standard model to generate a solution that satisfies five constraints simultaneously, and it typically satisfies three or four while violating the others — without acknowledging the violations.

Scientific reasoning: Ask about a system with multiple interacting variables, and standard models often oversimplify the interactions or miss second-order effects.

Understanding these failure modes helps you build the right mental model: reasoning models are not “better” across the board — they are specifically better at tasks where correctness requires multiple dependent steps. For everything else, standard models are faster, cheaper, and equally capable.

Reasoning models work by generating chain-of-thought at inference time. The key concepts are how this differs from prompted chain-of-thought and how each provider implements it.

When you add “Think step by step” to a prompt for GPT-4o, you get more verbose output but the same underlying generation process. The model is still predicting the next token based on the previous ones, with no ability to backtrack or verify.

Reasoning models are different. They have been trained through reinforcement learning to use the thinking phase productively — generating multiple candidate approaches, evaluating which approach is most promising, backtracking when an approach leads to a contradiction, and verifying the final answer against the original problem.

Reasoning models introduce a new parameter: the thinking budget. This controls how many tokens the model can use for its internal reasoning process before producing the final answer.

A higher thinking budget allows deeper reasoning — more approaches explored, more verification steps, more backtracking. A lower budget produces faster responses at the cost of potentially shallower analysis.

This creates a new optimization dimension for GenAI engineers: you can tune the reasoning depth per request based on the expected complexity. A simple logic check might need 1,000 thinking tokens. A complex code review might need 30,000.

OpenAI’s reasoning models (o1, o3) use an approach where the thinking process happens internally and the thinking tokens are not exposed to the developer. You see only the final answer and the token count.

Key characteristics:

• Reasoning effort levels (o3): low, medium, high — control how much compute the model spends thinking
• No system prompt support (o1) — these models were initially designed to ignore system prompts; o3 added limited support
• Reasoning tokens are billed — you pay for both thinking tokens and output tokens
• Hidden reasoning trace — you cannot see the intermediate steps, only the final answer and token count

Anthropic takes a different approach with Claude’s extended thinking feature. When enabled, the model generates a thinking trace that can be returned to the developer, giving visibility into the reasoning process.

Key characteristics:

• Explicit thinking parameter — you enable it per request and set a budget_tokens limit
• Visible thinking trace — optionally inspect the model’s reasoning steps
• Works with system prompts — no restriction on system prompt usage
• Same model, different mode — Claude Sonnet with extended thinking is the same model as standard Sonnet, running in a different inference mode
• Streaming support — thinking tokens stream separately from the final response

This section shows the concrete API calls for each provider. The key insight: reasoning models require less prompting, not more. Do not use elaborate system prompts or chain-of-thought instructions — the model handles that internally.

# Requires: openai>=1.30.0
from openai import OpenAI

client = OpenAI()

# Basic o3 call — let the model reason on its own
response = client.chat.completions.create(
    model="o3",
    messages=[
        {
            "role": "user",
            "content": (
                "Review this Python function for correctness. "
                "Identify any bugs, edge cases, or logic errors.\n\n"
                "```python\n"
                "def merge_intervals(intervals):\n"
                "    if not intervals:\n"
                "        return []\n"
                "    intervals.sort(key=lambda x: x[0])\n"
                "    merged = [intervals[0]]\n"
                "    for current in intervals[1:]:\n"
                "        if current[0] <= merged[-1][1]:\n"
                "            merged[-1][1] = max(merged[-1][1], current[1])\n"
                "        else:\n"
                "            merged.append(current)\n"
                "    return merged\n"
                "```"
            ),
        }
    ],
    # o3 supports reasoning_effort to control thinking depth
    reasoning_effort="high",  # "low", "medium", or "high"
)

print(response.choices[0].message.content)

# Check reasoning token usage
usage = response.usage
print(f"Input tokens: {usage.prompt_tokens}")
print(f"Output tokens: {usage.completion_tokens}")
print(f"Reasoning tokens: {usage.completion_tokens_details.reasoning_tokens}")

Prompt strategy for o3:

• Do not add a system prompt with instructions like “think step by step” — the model does this natively and additional prompting can interfere.
• Be direct about what you want. State the task clearly without scaffolding.
• Provide all relevant context in the user message — code, constraints, requirements.

# Requires: anthropic>=0.40.0
import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=16000,
    thinking={
        "type": "enabled",
        "budget_tokens": 10000,  # Max tokens for internal reasoning
    },
    messages=[
        {
            "role": "user",
            "content": (
                "Analyze this distributed system design for race conditions "
                "and data consistency issues. The system has three services "
                "that share state through an eventually consistent database.\n\n"
                "Service A writes user profiles.\n"
                "Service B reads profiles and generates recommendations.\n"
                "Service C updates recommendations based on user feedback.\n\n"
                "Identify all potential consistency issues and propose fixes."
            ),
        }
    ],
)

# Access the thinking trace and final response
for block in response.content:
    if block.type == "thinking":
        print(f"Thinking: {block.thinking[:500]}...")  # Inspect reasoning
    elif block.type == "text":
        print(f"Answer: {block.text}")

print(f"Input tokens: {response.usage.input_tokens}")
print(f"Output tokens: {response.usage.output_tokens}")

Prompt strategy for Claude extended thinking:

• System prompts work normally — use them for role and context.
• Set budget_tokens based on expected complexity: 5,000 for moderate tasks, 10,000-30,000 for complex analysis.
• The thinking trace is useful for debugging — inspect it when the model produces unexpected results.

Standard prompt engineering techniques like few-shot examples, chain-of-thought instructions, and output format scaffolding can actually degrade reasoning model performance. The model has been trained to reason effectively — adding your own reasoning scaffolding creates conflicting signals.

The rule: give reasoning models the problem and the constraints. Let them figure out the approach.

The following diagram shows the fundamental trade-offs between standard LLMs and reasoning models. Use this as a decision framework for your architecture.

Before choosing a model for a task, answer these three questions:

1. Does the task have verifiable correctness? If the answer can be checked against ground truth (math, code, logic), reasoning models add value. If the answer is subjective (creative writing, summarization), they do not.
2. Does accuracy outweigh speed? If a user is waiting for a chat response, 30 seconds is too long. If a CI pipeline is reviewing code, 30 seconds is fine.
3. Is the cost justified? If a wrong answer costs you $100 in debugging time, spending $0.50 on a reasoning model call is a bargain. If a wrong answer costs nothing (draft suggestions), the extra spend is waste.

If the answer to all three is yes, use a reasoning model. Otherwise, stick with standard LLM fundamentals and good prompt engineering.

Concrete examples show where reasoning models earn their cost and where they waste money.

A code review task where a standard model misses a subtle bug but o3 catches it.

The problem: Review a 200-line Python function that implements a concurrent task scheduler with priority queues. The function has a race condition that only manifests when two tasks with equal priority are submitted within 1ms of each other.

Standard model result: GPT-4o identifies surface-level issues (missing type hints, no docstring) and suggests general improvements. It misses the race condition entirely because detecting it requires tracing concurrent execution paths across multiple code blocks.

Reasoning model result: o3 traces the execution flow for concurrent submissions, identifies the unprotected shared state in the priority comparison logic, explains the exact sequence of events that triggers the bug, and proposes a fix using a lock or atomic compare-and-swap.

# Requires: openai>=1.30.0
from openai import OpenAI

client = OpenAI()

# Read the code file
with open("scheduler.py") as f:
    code = f.read()

# o3 with high reasoning effort for thorough code review
response = client.chat.completions.create(
    model="o3",
    messages=[
        {
            "role": "user",
            "content": (
                f"Review this concurrent task scheduler for correctness. "
                f"Focus on race conditions, deadlocks, and data consistency "
                f"issues. Trace concurrent execution paths.\n\n"
                f"```python\n{code}\n```"
            ),
        }
    ],
    reasoning_effort="high",
)

A research task where the model needs to synthesize information from multiple documents and produce a structured analysis with cross-references.

# Requires: anthropic>=0.40.0
import anthropic

client = anthropic.Anthropic()

documents = [
    "Paper 1: Attention mechanism efficiency improvements...",
    "Paper 2: Sparse attention patterns in long-context models...",
    "Paper 3: Hardware-aware transformer optimization...",
]

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=16000,
    thinking={
        "type": "enabled",
        "budget_tokens": 15000,  # Complex synthesis needs more budget
    },
    messages=[
        {
            "role": "user",
            "content": (
                "Synthesize these three papers into a technical brief. "
                "Identify areas of agreement, contradiction, and open questions. "
                "For each finding, cite which paper(s) support it.\n\n"
                + "\n\n---\n\n".join(documents)
            ),
        }
    ],
)

Claude’s extended thinking allows it to cross-reference claims across documents, track which paper supports which conclusion, and identify contradictions — tasks that require holding multiple contexts simultaneously and reasoning about their relationships.

These tasks do not benefit from reasoning models. Using one is a waste of money and time.

The rule of thumb: if a human could answer the question in under 10 seconds without a calculator or reference material, a standard LLM is the right choice.

Every reasoning model call is a trade-off between accuracy, cost, and speed. Understanding the numbers helps you make informed routing decisions.

Reasoning models cost more per request because they generate thinking tokens in addition to output tokens. These thinking tokens are billed at output token rates.

Prices are approximate as of early 2026. Always verify against official provider pricing before production decisions.

A typical production system handling 10,000 requests per day where 15% route to reasoning models:

• 8,500 standard requests at $0.02 avg = $170/day
• 1,500 reasoning requests at $0.30 avg = $450/day
• Total: $620/day vs $200/day if all went to standard models

That 3x premium is justified only if the reasoning model catches errors that would otherwise cost engineering time, customer trust, or revenue.

Use this flowchart for every routing decision:

1. Is the answer verifiable? (Can you check if it is correct?)
   • No → Use standard LLM. Stop here.
   • Yes → Continue.
2. Does a wrong answer have real cost? (Bug in production, wrong calculation, bad decision)
   • No → Use standard LLM. Stop here.
   • Yes → Continue.
3. Can the user tolerate 10-60s latency?
   • No → Use standard LLM with better prompting. Stop here.
   • Yes → Use a reasoning model.
4. How complex is the task?
   • Moderate → Use o3-mini or Claude thinking with low budget (5,000 tokens).
   • Highly complex → Use o3 high effort or Claude thinking with high budget (15,000+ tokens).

This framework prevents the two common mistakes: (1) using reasoning models for everything (expensive, slow) and (2) never using them (missing accuracy gains on hard tasks).

Reasoning models are appearing in GenAI system design interviews. Here are the questions you should be prepared to answer and sample frameworks for each.

Framework: Apply the three-question test — verifiable correctness, cost of errors, latency tolerance. Give a concrete example: “For a code review pipeline in CI/CD, I would use o3 because correctness is verifiable (does the code have bugs?), errors are costly (bugs in production), and the user tolerates 30s latency (it runs async in the pipeline). For the same application’s chat support feature, I would use GPT-4o because speed matters and answers do not need mathematical precision."

Framework: Describe asynchronous architecture. “I separate the user-facing response from the reasoning model call. The user gets an immediate acknowledgment and a streaming status indicator. The reasoning model runs asynchronously, and the result is pushed to the user via WebSocket or SSE when ready. For batch workloads like nightly code reviews, I queue the requests and process them without any latency constraint."

Framework: Describe a three-tier routing architecture:

1. Classifier layer — A cheap model (GPT-4o-mini) classifies incoming requests by complexity in <500ms.
2. Router — Maps complexity levels to model tiers. Simple → GPT-4o-mini, moderate → GPT-4o/Sonnet, complex → o3/Claude thinking.
3. Feedback loop — Log accuracy metrics per model tier. If the standard model handles a “complex” task correctly, downgrade it to save cost. If it fails on a “moderate” task, upgrade it.
4. Cost controls — Per-user and per-feature token budgets that cap reasoning model spend.
5. Fallback — If the reasoning model times out or fails, fall back to the standard model with enhanced prompting rather than returning nothing.

Framework: “I run A/B evaluations on a held-out test set with ground-truth answers. I measure accuracy, cost per correct answer, and latency. The key metric is cost-per-correct-answer — if the reasoning model costs 5x more but produces 2x fewer errors that each require 30 minutes of engineering time to fix, the reasoning model saves money net. I track this using the evaluation frameworks we use for all LLM outputs.”

Production systems that use reasoning models need routing infrastructure, cost controls, and monitoring. These patterns build on the LLM routing and cost optimization foundations.

# Requires: openai>=1.30.0, anthropic>=0.40.0
from enum import Enum
from openai import OpenAI
import anthropic

openai_client = OpenAI()
anthropic_client = anthropic.Anthropic()

class ModelTier(str, Enum):
    FAST = "fast"        # GPT-4o-mini / Claude Haiku
    STANDARD = "standard"  # GPT-4o / Claude Sonnet
    REASONING = "reasoning"  # o3 / Claude Sonnet with thinking

# Classification prompt — runs on cheapest model
CLASSIFIER_PROMPT = """Classify this request's reasoning complexity.
Reply with exactly one word: "fast", "standard", or "reasoning".

- "fast" — simple extraction, formatting, yes/no, classification
- "standard" — summarization, explanation, moderate code generation
- "reasoning" — multi-step math, complex code review, logic puzzles,
  scientific analysis, constraint satisfaction

Request: {query}"""

def classify_complexity(query: str) -> ModelTier:
    response = openai_client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": CLASSIFIER_PROMPT.format(query=query)}],
        max_tokens=10,
        temperature=0,
    )
    tier_str = response.choices[0].message.content.strip().lower()
    try:
        return ModelTier(tier_str)
    except ValueError:
        return ModelTier.STANDARD  # Default to standard on unexpected output

def route_request(query: str, system_prompt: str = "") -> dict:
    tier = classify_complexity(query)

    if tier == ModelTier.FAST:
        response = openai_client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": query},
            ],
        )
        return {
            "tier": tier.value,
            "model": "gpt-4o-mini",
            "response": response.choices[0].message.content,
            "tokens": response.usage.total_tokens,
        }

    elif tier == ModelTier.STANDARD:
        response = openai_client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": query},
            ],
        )
        return {
            "tier": tier.value,
            "model": "gpt-4o",
            "response": response.choices[0].message.content,
            "tokens": response.usage.total_tokens,
        }

    else:  # REASONING
        response = openai_client.chat.completions.create(
            model="o3",
            messages=[{"role": "user", "content": query}],
            reasoning_effort="medium",
        )
        return {
            "tier": tier.value,
            "model": "o3",
            "response": response.choices[0].message.content,
            "tokens": response.usage.total_tokens,
            "reasoning_tokens": response.usage.completion_tokens_details.reasoning_tokens,
        }

Reasoning model responses are expensive and often deterministic for the same input. Caching them aggressively reduces cost on repeated or similar queries. Apply LLM caching strategies with one modification: increase the cache TTL for reasoning model outputs because they are more expensive to regenerate.

import hashlib
import json
import redis

r = redis.Redis(host="localhost", port=6379, decode_responses=True)

REASONING_CACHE_TTL = 604800  # 7 days (longer than standard cache)
STANDARD_CACHE_TTL = 86400    # 1 day

def cached_reasoning_call(query: str, model: str = "o3") -> dict | None:
    cache_key = f"reasoning:{model}:{hashlib.sha256(query.encode()).hexdigest()}"
    cached = r.get(cache_key)
    if cached:
        return json.loads(cached)
    return None

def store_reasoning_result(query: str, model: str, result: dict) -> None:
    cache_key = f"reasoning:{model}:{hashlib.sha256(query.encode()).hexdigest()}"
    r.setex(cache_key, REASONING_CACHE_TTL, json.dumps(result))

Reasoning tokens are invisible to users but drive the majority of the cost. Track them separately from standard output tokens.

Key metrics to monitor:

• Reasoning tokens per request — Track the distribution. If most requests use <2,000 reasoning tokens, you might be routing too many simple tasks to the reasoning tier.
• Cost per tier — Track daily spend broken down by fast/standard/reasoning. If the reasoning tier exceeds 50% of total spend, review your routing classifier.
• Accuracy by tier — Sample responses from each tier and measure accuracy. If the standard tier handles “reasoning” tasks correctly 80% of the time, your classifier is routing too aggressively.
• Latency percentiles — Track p50, p90, and p99 latency for reasoning model calls. A p99 above 60 seconds suggests some requests are too complex even for reasoning models and might need to be broken into smaller steps.

Implement per-feature and per-user limits on reasoning model usage to prevent cost surprises:

from datetime import datetime

DAILY_REASONING_BUDGET = 100.00  # $100/day cap
COST_PER_REASONING_TOKEN = 0.00004  # o3 output rate

def check_reasoning_budget(user_id: str) -> bool:
    """Check if user has remaining reasoning budget for today."""
    today = datetime.utcnow().strftime("%Y-%m-%d")
    budget_key = f"budget:reasoning:{user_id}:{today}"
    spent = float(r.get(budget_key) or 0)
    return spent < DAILY_REASONING_BUDGET

def record_reasoning_spend(user_id: str, reasoning_tokens: int) -> None:
    """Record reasoning token spend against daily budget."""
    today = datetime.utcnow().strftime("%Y-%m-%d")
    budget_key = f"budget:reasoning:{user_id}:{today}"
    cost = reasoning_tokens * COST_PER_REASONING_TOKEN
    r.incrbyfloat(budget_key, cost)
    r.expire(budget_key, 172800)  # Auto-expire after 48 hours

When the budget is exhausted, fall back to the standard tier with enhanced prompting — a graceful degradation rather than a hard failure.

Reasoning models add a new dimension to GenAI engineering: the ability to trade cost and latency for accuracy on hard problems. The core principles are straightforward.

When to use reasoning models:

• Multi-step math, logic, and constraint satisfaction
• Complex code review and generation with dependencies
• Scientific reasoning requiring verification of intermediate steps
• Any task where a wrong answer has a measurable cost

When to stick with standard LLMs:

• Chat, FAQ, and customer support (latency-sensitive)
• Summarization, classification, and format conversion (pattern tasks)
• Creative writing and content generation (subjective quality)
• High-volume workloads where cost per request matters (>10,000 req/day)

The production pattern: Build a routing layer that classifies request complexity and directs each request to the cheapest model capable of handling it. Cache reasoning model outputs aggressively. Monitor reasoning token usage and set budget caps per user and feature.

These pages build on the concepts covered here:

• LLM Fundamentals — How transformer models generate text and why reasoning models extend that process.
• Prompt Engineering — Standard prompting techniques that work with both standard and reasoning models.
• LLM Routing — The full routing architecture for multi-model systems, including reasoning model tiers.
• LLM Cost Optimization — Cost reduction strategies that complement reasoning model routing.
• LLM Caching — Caching strategies for expensive reasoning model outputs.
• LLM API Comparison — Detailed comparison of OpenAI, Anthropic, and other provider APIs.
• Evaluation — How to measure whether reasoning models improve output quality.
• LLM Benchmarks — Benchmark methodology and where reasoning models outperform standard models.
• System Design — Designing production systems that incorporate reasoning model routing.
• Interview Questions — Full GenAI interview question bank including system design questions.