Optimizing Cost and Latency with Amazon Bedrock Prompt Caching

In a post published by Daniel Abib, Specialist Solutions Architect for Generative AI at AWS, it is explained how using the prompt caching mechanism in Amazon Bedrock can reduce input token costs by up to 90 percent when repeatedly sending the same context to foundation models, as well as shorten time-to-first-token (TTFT). According to the post, without caching, sending a 10,000-token contract alongside 50 different user questions results in billing for 500,000 input tokens at full price for content the model has already processed.

The author notes that alternative approaches for managing costs include shortening prompts (which might compromise context quality), reducing context windows (which degrades the model's ability to reason over complete information), and response caching (which provides no benefit when the same context is paired with different questions). Caching prompts at the infrastructure level enables the model to read cached tokens instead of reprocessing them, without altering the model or prompt quality.

How Prompt Caching Works and Its Pricing

When a request to the Converse API includes a cachePoint marker, Amazon Bedrock evaluates whether the content preceding that marker matches an existing cache entry. In the case of a cache hit, the model skips reprocessing and begins generating output from the cached state. In the event of a cache miss, the model processes the full content and writes the result to the cache for potential future requests.

Four core principles define cache behavior:

1. Cache scope: Cache entries are scoped strictly to individual AWS accounts and specific AWS Regions.
2. Minimum token thresholds: Each checkpoint must meet a minimum token threshold to activate caching. For example, Anthropic Claude Sonnet 4.5 and Sonnet 4.6 require at least 1,024 tokens per checkpoint, while Opus models require at least 4,096 tokens.
3. Time-to-live (TTL): Cache entries expire according to the TTL specified in the request. The default is 5 minutes, with select models supporting up to 1 hour.
4. Model-agnostic syntax: The cachePoint syntax in the Converse API is identical across supported model families, including Anthropic Claude and Amazon Nova.

In terms of pricing, two new token categories are introduced:

• cacheWriteInputTokens: Tokens written to the cache on the first request, billed at a cost 25 percent higher than standard input. When utilizing a 1-hour TTL, the cost is 100 percent higher (2x) than standard input.
• cacheReadInputTokens: Tokens read from the cache on subsequent requests, billed at a cost 90 percent lower than standard input.

For workloads with repeated context, total savings on input token costs reach approximately 75 percent (for instance, a 10,000-token document queried with 10 different questions within the TTL window).

Converse API Use Cases

The post outlines six practical implementation scenarios in Python code using the Converse API:

1. Message content and long document caching: Placing a cachePoint marker after static document content and before the dynamic user question. In RAG or codebase analysis workflows, Amazon Bedrock caches the document on the first call and might reuse it on subsequent calls. For Claude models, simplified cache management is supported, where a single checkpoint checks for matches across up to approximately 20 preceding content blocks.
2. System prompt caching: Persona definitions, detailed instructions, and embedded knowledge bases within the system prompt are preserved by placing a cachePoint inside the system parameter, separate from user messages.
3. Tool definition caching: In agentic workflows, numerous JSON tool schemas can comprise thousands of tokens. Appending a cachePoint at the end of the tool array within toolConfig enables reuse without reprocessing schemas on every conversation turn.
4. Mixed TTL caching: Assigning different expiration times to different content tiers within the same request—such as 1 hour for reference materials and tool definitions, and 5 minutes for momentary conversation context. The API enforces an ordering constraint where longer TTL checkpoints must appear before shorter TTL checkpoints.
5. Tenant isolation in multi-tenant systems: To prevent cross-tenant cache access within the same AWS account and Region, a SHA-256 hash prefix of the tenant ID is prepended to the cached content. This creates an independent cache entry for each tenant with an overhead of only approximately 16 tokens (64 characters).
6. LangChain integration: Utilizing the ChatBedrockConverse class and the create_cache_point function to generate cache checkpoints directly within message arrays or within ChatPromptTemplate structures and LCEL chains.

Comparing the Converse API and InvokeModel API and Best Practices

The author compares the interfaces and highlights that while the Converse API offers uniform syntax across all supported models, places cache markers as standalone blocks, and provides a cacheDetails breakdown in responses, the InvokeModel API uses differing syntax for Anthropic models (utilizing ephemeral cache_control inside content blocks without cacheDetails response data). Therefore, the Converse API is recommended for new applications.

In conclusion, best practices include profiling prompts to identify static components, ensuring requests satisfy the required token threshold for each model, selecting suitable TTLs, monitoring cache metrics in application logs, incorporating control mechanisms such as Amazon Bedrock Guardrails, and combining simultaneous caching of messages, system prompts, and tool definitions within the same request.