Every Token Counts: A Token-Efficiency Playbook for Claude Apps

From prompt caching to maxTurns: the architectural patterns that keep cost, latency, and context intact to burn less tokens.Token-Efficient Architecture for AI-Powered ApplicationsA reference checklist for engineers building LLM-powered applications with primary focus on Claude / Claude Code /…

From prompt caching to maxTurns: the architectural patterns that keep cost, latency, and context intact to burn less tokens.Token-Efficient Architecture for AI-Powered ApplicationsA reference checklist for engineers building LLM-powered applications with primary focus on Claude / Claude Code / Claude Agent SDK, with honest coverage of open-weight local models for high-volume routing.1. Why token efficiency mattersEvery token you send or receive is billed, and every token you send also counts against a fixed context window and against your account’s rate limits (tokens-per-minute, not just requests-per-minute).An application that stuffs full documents into every turn, re-sends its entire tool catalog on each call, or lets an agent’s tool-output noise accumulate unchecked will hit three walls at once:rising per-request cost,rising latency (more tokens to process before the model can respond), andpremature context-window exhaustion that forces lossy summarization mid-task.Token efficiency is not a cost-optimization afterthought; it’s a load-bearing architectural concern from day one, on par with database indexing or API rate-limit design.Save this infographic for quick reference2. Prompt cachingClaude’s prompt caching lets you mark a prefix of your request (system blocks, tool definitions, message content) with cache_control: {"type": "ephemeral"} so that subsequent requests sharing that exact prefix are billed at a steep discount instead of the full input price [source: Anthropic prompt-caching docs].Mechanics:Caching is a strict prefix match: any byte change anywhere in the cached region (a timestamp in the system prompt, a reordered tool list, non-deterministic JSON serialization) invalidates everything after that point.Render order is fixed: tools → system → messages. A breakpoint on the last system block caches tools and system together.Default TTL is 5 minutes (cache_control: {"type": "ephemeral"}); an extended 1-hour TTL is available ({"type": "ephemeral", "ttl": "1h"}) for workloads with longer gaps between requests.Up to 4 cache breakpoints per request; minimum cacheable prefix length is model-dependent: 1,024 tokens for Sonnet 5 and Opus 4.8 (the models most engineers building against Claude today will use), as low as ~512 tokens on some smaller/older models. Check per-model minimums before assuming a short prompt will cache.Pricing shape: cache writes cost more than a normal input token (roughly 1.25× base price for the 5-minute TTL, 2× for the 1-hour TTL); cache reads cost far less than base input (roughly 0.1× base price). This means caching only pays off with reuse, a single cache write with zero reads is a pure loss.When it helps: repeated system prompts, static tool/function definitions, RAG context or reference documents reused across turns or across many user sessions, and multi-turn conversations where the growing history prefix is replayed on every turn.Concrete before/after shape: Say your system prompt + tool definitions run 3,000 tokens and you make 20 calls against the same session within the TTL window.Without caching: 20 × 3,000 = 60,000 input tokens billed at full price.With caching: 1 cache write (3,000 tokens × ~1.25) + 19 cache reads (3,000 tokens × ~0.1 each) ≈ 3,750 + 5,700 = ~9,450 “effective” input-token-equivalents; an 84.25% reduction on that portion of the request. The break-even point is low: on the 5-minute TTL (1.25× write), the first cache read already nets savings against a second uncached call. On the 1-hour TTL (2× write, a bigger up-front premium), break-even lands after two cache reads — i.e., by your third request in the session.Fork rule: if your architecture spins off side calls (summarization, subagents, compaction passes), the fork must reuse the parent’s exact system/tools/model bytes verbatim, or it silently misses the parent's cache entirely.3. Context management architectureSystem prompt / tool definition minimizationEvery tool definition and every line of system prompt is billed on every call that includes it (subject to caching above, but caching only reduces cost — it doesn’t reduce the context-window footprint).Trim tools the current task doesn’t need rather than loading your entire tool catalog by default; drop few-shot examples once the model demonstrates it doesn’t need them for a given task class.Anthropic documents a tool search pattern: tool schemas are discovered and loaded on demand rather than all declared upfront as one way to keep a large tool catalog from bloating fixed context; confirm current availability/mechanics against the latest tool-use docs before depending on it in production.RAG retrieval vs. full-document stuffingRetrieve the relevant passages and pass those, rather than concatenating whole source documents into every prompt. Full-document stuffing wastes tokens on irrelevant content, degrades the model’s ability to locate the relevant part (needle-in-haystack effects grow with unnecessary context), and defeats caching if the “irrelevant” portion varies per query. A retrieval step that returns 3–5 targeted chunks is usually both cheaper and more accurate than stuffing 50 pages and hoping.Conversation compaction/summarizationLong-running agent sessions eventually approach the context window limit. Claude Code and the underlying Claude API support server-side compaction: as a conversation nears a trigger threshold, earlier turns are automatically summarized into a compaction block that replaces the raw history on subsequent requests, rather than the client having to truncate blindly.The implementation detail worth internalizing for your own agents: you must append the full response content (including compaction blocks) back into the message history on each turn, not just the extracted text. The API needs the compaction block itself to correctly replace prior history next time.This is a general pattern any long-running agentic application should replicate even outside Claude’s specific API: summarize old turns down to their decision-relevant conclusions, discard the token-heavy scaffolding (full tool outputs, exploratory reasoning) that led there.Subagent / fork pattern: orchestrator/worker context isolationThis is the single highest-leverage architectural lever for agentic applications, and it is exactly how Claude Code’s own subagent mechanism works: a subagent runs in an isolated context window; it starts fresh, without your main conversation’s history, and receives only a delegation task description (plus static project context like CLAUDE.md).It does its own token-heavy work: reading files, searching logs, crawling documentation entirely inside that isolated context. Only the subagent’s final summary flows back into the parent/orchestrator’s context; the intermediate exploration, raw file contents, and search noise never touch the main conversation’s token budget.A fork (as opposed to a fresh subagent) is the deliberate exception: it inherits the parent’s full conversation history and tool set, used when continuity matters more than isolation (e.g., continuing a task with full prior context).This generalizes directly to any agentic application you build on any model: adopt an orchestrator/worker pattern. The orchestrator holds only decision-relevant state and issues delegated tasks; workers absorb the token-expensive search/read/crawl operations in their own disposable context and return condensed findings.This keeps the orchestrator’s context small and stable, cacheable, cheap to re-process every turn, and far less likely to hit the context-window ceiling on long-running tasks. Treat “does this operation risk flooding my main context with output I won’t need again” as the trigger question for whether a step belongs behind a subagent boundary rather than inline.Tool output truncation/paginationNever return raw dumps by default. A file-read tool should support line-range reads; a search tool should return match counts and snippets, not full file bodies; a diff tool should return the diff, not before/after full-file pairs.Design tool outputs the way you’d design a paginated API response; return what’s needed to decide the next action, and let the caller ask for more if needed.4. Skills and agent configuration as built-in efficiency mechanismsClaude Code’s own configuration surface implements several of the levers above natively, worth using before hand-rolling your own version.Skills load lazily. A skill (.claude/skills//SKILL.md) only puts its name and description in context by default; the description is capped at 1,536 characters. The full body (instructions, examples, reference files) is not parsed or loaded until the skill is actually invoked (via /skill-name or Claude auto-triggering it by relevance). This is the direct opposite of stuffing every possible instruction into a static system prompt: you can maintain dozens of skills, and only the ones triggered in a given session cost tokens.Two related settings tune this further: skillListingBudgetFraction (how much of the context budget the skill listing itself may consume) and skillListingMaxDescChars (a per-skill description cap) [source: code.claude.com/docs/en/skills.md, code.claude.com/docs/en/settings-reference.md].Subagent tools: / disallowedTools: frontmatter fields restrict which tools a subagent can see and call:tools: Read, Grep, Glob, BashdisallowedTools: Write, EditThis isn’t just an authorization control; it directly reduces the tool-schema tokens sent on every call that subagent makes, since tools outside the allowlist are never declared to the model. disallowedTools is applied first, then tools resolves against what remains; MCP tool patterns (mcp__, mcp____*, mcp__*) are supported for bulk exclusion [source: code.claude.com/docs/en/sub-agents.md, verified live].Subagent model: frontmatter field pins a specific agent to a specific model, independent of the main conversation:model: haikuValid values are model aliases (sonnet, opus, haiku, fable), a full model ID, or inherit (use the main conversation's model). Resolution order: a per-invocation model parameter, then the agent's frontmatter, then the CLAUDE_CODE_SUBAGENT_MODEL environment variable, then the main conversation's model.This is model routing (§4 below) expressed declaratively at the agent-definition level: a narrow, well-scoped agent (formatting, extraction, lookup) can be permanently pinned to Haiku while the orchestrating conversation runs Sonnet or Opus, with no runtime branching logic required [source: code.claude.com/docs/en/sub-agents.md, verified live].Subagent maxTurns: frontmatter field bounds worst-case token spend on a runaway agentic loop:maxTurns: 10When a subagent hits the limit, Claude Code returns its output marked as partial rather than continuing indefinitely; the orchestrator can then decide whether to resume it. This is the direct token-budget analog of a request timeout: without it, a subagent stuck in a retry/search loop has no ceiling on tool-call round-trips (and therefore no ceiling on token spend) [source: code.claude.com/docs/en/sub-agents.md, verified live].settings.json context-management knobs apply at the session level rather than per-agent:autoCompactEnabled — turns automatic context compaction on/off.autoCompactWindow — how full the context window gets before compaction triggers.bashOutputMaxChars — caps how much of a shell command's output is returned inline into context, preventing one verbose command from silently consuming a large token budget.[source: code.claude.com/docs/en/settings-reference.md, verified live; exact default values are not stated on the reference page — check your installed version's claude config output rather than assuming a number.]Practical takeaway: if you’re building on the Claude Agent SDK or Claude Code directly, prefer these declarative controls over ad hoc prompt engineering tools: allowlist and a model: haiku line on a narrow subagent get you tool-schema trimming and cost-aware routing for free, and maxTurns gives you a hard backstop against the one failure mode none of the other techniques in this article address: an agent that simply won't stop calling tools.Before/after: unoptimized vs. configured context loadSame capability, far smaller fixed cost per call: the top side pays full price for context it may never use; the bottom side pays only for what a given task actually invokes.5. Model routing / right-sizingNot every step in a pipeline needs your most capable (and most expensive) model. Route by task complexity:Cheap/small models (e.g., Claude Haiku) for classification, entity extraction, intent routing, simple formatting, and other narrow, well-specified sub-tasks where reasoning depth doesn’t move the needle on quality.Escalate to larger models (Sonnet/Opus-tier) only for steps that genuinely require deep reasoning, multi-step planning, or nuanced judgment — code review, complex synthesis, ambiguous instruction-following.A common production shape: a cheap model classifies/routes the request, and only the branch that needs it invokes the expensive model — most traffic never touches the costly path.Open-weight local models (Llama, Mistral, and similar Western open-weight families; Chinese-origin models/labs are out of scope here) suit one slice of this problem: high-volume, low-stakes classification or routing, where self-hosting amortizes compute cost across enough volume to beat per-token API pricing, and error tolerance covers a smaller model’s lower accuracy ceiling. It’s a volume-and-risk tradeoff, not a blanket substitute — and self-hosting shifts cost from per-token billing to your own ops burden (GPUs, updates, monitoring).Batching for async workloads: Claude’s Message Batches API processes large volumes of non-latency-sensitive requests asynchronously (submit a batch, poll for completion; most batches finish within about an hour) at roughly 50% lower cost than standard synchronous requests [source: Anthropic batch-processing docs, https://platform.claude.com/docs/en/build-with-claude/batch-processing].This is the right lever for bulk classification, offline evaluation runs, large-scale content generation, or any workload where you don't need a response within seconds.6. Output-side efficiencyOutput tokens are typically priced several times higher than input tokens, so trimming verbose output has outsized cost impact:Prefer structured/constrained output (JSON schemas via structured-output/tool-call mechanisms) over free-form prose when the consumer is a program, not a human. Structured output is shorter, more predictable in length, and removes the token cost of natural-language padding (“Sure, here’s the JSON you requested…”).Set explicit max_tokens appropriate to the task — don't leave it at a large default for a task that should produce a short answer; conversely, don't set it so low that legitimate output gets truncated mid-response, which just forces a retry and doubles your cost.Avoid exposing chain-of-thought/reasoning output when the caller doesn’t need it. Extended reasoning is useful for improving answer quality on hard problems, but streaming or returning the full reasoning trace to end users (or storing it wholesale in logs) burns tokens and storage for content that’s rarely consumed after the fact. Default reasoning visibility to summarized or omitted, and turn on full visibility only where a human explicitly needs to audit the model’s reasoning path.7. Measurement & governanceYou can’t optimize what you don’t measure. Build token accounting into the engineering workflow, not just into a post-hoc billing dashboard:Token counting API (messages.count_tokens on the Claude API) gives an exact, model-specific token count for a prompt before you send it — use this for pre-flight budget checks, not a third-party tokenizer (tokenizers are model-specific; a count from one model's tokenizer can be meaningfully wrong for another).Per-feature token budgets. Assign an expected token envelope to each feature/endpoint (e.g., “the summarization endpoint should average under N input tokens and M output tokens per call”) the same way you’d assign a latency SLO.Cost dashboards broken down by feature/endpoint/model, not just aggregate spend — aggregate spend tells you nothing about which feature regressed.Catch regressions in code review. A prompt-template change that removes a cache breakpoint, a new tool added to every request’s tool list, or a RAG step that silently switched from top-3 retrieval to whole-document context are the kinds of changes that should trip a reviewer’s attention the same way an added N+1 query would. Track cache-hit-rate (usage.cache_read_input_tokens vs. total) as a monitored metric — a silent drop to zero on a previously-caching prompt is a regression, not noise.8. Reference checklist9. DiagramsArchitecture: orchestrator + subagent/fork pattern (context isolation)Key point: raw search results, file contents, and logs stay inside the worker’s box and are discarded when it finishes. Only the small arrow labeled “summary only” crosses back into the orchestrator’s context.Flow: request lifecycle and where each optimization appliesEach numbered step maps to a section above:cachingcontext assembly/RAG/minimizationmodel routingSend request for tool calltool output truncationoutput-side efficiencymeasurement & governance.Sources UsedAnthropic prompt caching mechanics (cache_control, ephemeral TTL of 5 minutes default / 1-hour extended, prefix-match invalidation, cache write vs. read pricing multipliers, minimum cacheable prefix): claude-api skill reference file shared/prompt-caching.md (Anthropic-maintained, sourced from platform.claude.com/docs/en/build-with-claude/prompt-caching).Message Batches API — 50% cost reduction for async processing: verified live via WebFetch, https://platform.claude.com/docs/en/build-with-claude/batch-processing ("cutting costs by 50% and increasing throughput", "reducing costs by 50%").messages.count_tokens token counting API usage: claude-api skill reference file shared/token-counting.md.Subagent/fork context isolation mechanics (isolated context window, delegation-only task message, summary-only return path, fork as full-history exception): verified live via WebFetch, https://code.claude.com/docs/en/sub-agents.md.Compaction (server-side summarization at context-window trigger threshold, requirement to replay full response.content including compaction blocks): claude-api skill quick-reference section, sourced from platform.claude.com/docs/en/build-with-claude/compaction.Tool search / on-demand tool schema loading as a context-minimization pattern: claude-api skill reference file shared/agent-design.md.Output pricing asymmetry (output tokens priced higher than input) and structured-output mechanism (output_config.format): general Claude API architecture, claude-api skill SKILL.md.Skills lazy-loading (description-only baseline cost, 1,536-char cap, full body loads only on invocation), subagent tools:/disallowedTools: frontmatter, model: override with resolution order, and maxTurns: partial-output behavior: verified live via WebFetch, https://code.claude.com/docs/en/skills.md and https://code.claude.com/docs/en/sub-agents.md.settings.json context-management keys (autoCompactEnabled, autoCompactWindow, bashOutputMaxChars, skillListingBudgetFraction, skillListingMaxDescChars): verified live via WebFetch, https://code.claude.com/docs/en/settings-reference.md. Default values are not stated on that reference page — check your installed version rather than assuming a number.Open questions for the userThe before/after cache-savings example in Section 2 uses illustrative multipliers (~1.25×/~2× write, ~0.1× read) drawn directly from the sourced skill reference — worth double-checking against the live Anthropic pricing page if this article will be published externally, since multipliers can shift with pricing updates.The open-weight local model section is deliberately brief per your instruction to exclude Chinese-origin proprietary/hosted models — confirm this reads as sufficiently “honest secondary mention” rather than too thin; I can expand with a concrete self-hosting cost-breakeven example if you want more depth.Flag if you want the compaction trigger threshold (150K tokens, mentioned in the skill's quick reference but not restated in the article) added explicitly.Suggested next stepSend to adversarial-reviewer (or the equivalent scrutiny pass) to check:(1) whether the cache pricing multipliers should be re-verified against the current live pricing page rather than the skill's cached reference before publishing,(2) whether the open-weight-models section stays appropriately brief without reading as dismissive, and(3) whether the Mermaid diagrams render cleanly in your target viewer (GitHub-flavored Markdown vs. other renderers can differ slightly on subgraph styling).This story is published under the Generative AI publication. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories. Let’s shape the future of AI together!Every Token Counts: A Token-Efficiency Playbook for Claude Apps was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Source: Generative AI Pub — Published September 16, 2026 at 02:20 UTC — Category: Image AI