> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dari.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt Caching

> Keep a warm model when switching would cost more than it saves.

Cache-aware routing stops a policy from switching away from a warm model for a tiny apparent saving. Give the policy real prices and the conversation's recent cache state; it removes switches that do not pay for their cold prefix.

## Use It In Your Policy

Use `createDariRoutingPolicy`, not a plain inline `RoutingPolicy`:

```ts theme={null}
import { createPiRuntime, createRouter } from "@mupt-ai/dari-router";
import { createDariRoutingPolicy } from "@mupt-ai/dari-router/policy-engine";

const pi = await createPiRuntime({ apiKey: process.env.OPENAI_API_KEY! });

const policy = createDariRoutingPolicy({
  runtime: pi,
  selectorModel: "openai/gpt-5.4-mini",
  selectorContextWindowChars: 400_000,

  // Your per-million-token provider prices, including cached-input prices.
  pricing: (model) => myPricing[model] ?? null,

  // Your observed output averages. Router Core does not provide these.
  averageOutputTokensByModel: myAverageOutputTokensByModel,

  // Read the state saved for this conversation after earlier provider calls.
  state: async ({ request }) =>
    request.cacheKey ? cacheState.read(request.cacheKey) : {},
});

const router = createRouter({
  models: [pi.model("openai/gpt-5.4-mini"), pi.model("openai/gpt-5.4")],
  policy,
  executor: pi,
});
```

`cacheState.read()` returns the state the policy engine needs:

```ts theme={null}
{
  chainsByModel, // prompt-prefix chain visible to each candidate model
  prefixHits,    // recently observed provider prefixes and the model that served them
  nowMs: Date.now(),
}
```

After each provider call, save the updated prompt chain, provider-prefix observation, and selected model under the same key. That persistence is application code: Router Framework does not store conversations or cache observations for you.

## What You Get

For a conversation with a warm incumbent, the policy engine compares each switch's cold-prefix cost with its expected savings. It automatically removes a switch unless it saves at least **10%** on the fixed-turn cost.

So when `gpt-5.4-mini` looks a little cheaper but switching would reprocess a large warm prefix, it is removed before the selector chooses. A genuinely cheaper switch stays available.

This protection works only when all of these are true:

* The request has a stable `prompt_cache_key`.
* Your saved state identifies the previous model and a still-warm prefix.
* You provide normal and cached-input prices plus observed output-token averages for each model/effort.

Router Core does not ship, infer, or persist output averages. Self-hosted users own those estimates; Dari's managed platform can maintain its own. If an input is missing or stale, the engine makes no cache-savings claim and does not prune on cache cost.

## Send The Same Key Every Turn

Send `prompt_cache_key` on every request in one conversation:

```json theme={null}
{
  "model": "my-router",
  "prompt_cache_key": "conversation-42",
  "messages": [{ "role": "user", "content": "Continue." }]
}
```

Both the OpenAI Chat Completions and Anthropic Messages adapters accept this field. They normalize it to `request.cacheKey`; `createPiRuntime()` passes it to Pi as the provider session ID. Custom executors receive `request.cacheKey` and should map it to the cache or session field supported by their provider.

Provider cache hits remain the provider's decision. Executors return observed usage as `cacheReadTokens` and `cacheWriteTokens`; the protocol adapters serialize those values in the caller's native usage format.

## Leases Are A Simpler Fallback

A lease keeps later turns on one model, but it is not a cache-cost calculation or a cache hit. Use `leaseTurnsRemaining` when you only want short-term stickiness. Use cache-aware routing when you persist prefix observations and want the policy to decide whether a switch is actually worth it.

Read [Policies](/framework/policies) for the full policy setup and [Decision Pipeline](/framework/decision-pipeline) if you need to control persistence and selection phases yourself.