Skip to main content
Speculative routing starts a follow-up request on the previously selected compatible route while Dari chooses the route for the next turn. It overlaps provider execution with selection so the selector does not add its full latency to every continuation. New routers enable speculative routing by default. It is available with both SLM Router and Describe Your Router configurations.

​
How It Works

On the first turn, Dari has no previous route and selects a model before provider execution. On an eligible follow-up, Dari:
  1. Recovers the model and reasoning effort previously saved for this follow-up.
  2. Confirms that pair still satisfies the current request and router configuration.
  3. Starts that provider request while the selector evaluates the conversation.
  4. Serves the current turn from the model that was started.
  5. Stores the selector’s decision as the route to start on the next follow-up.
The model shown in X-Dari-Selected-Model and the response’s routing metadata is always the model that served the current turn. If the selector chooses a different model, that change applies on the next turn.
Streaming follows the same rule. Dari can send content from the continuing model while selection is still running, but waits to finish the stream until the next routing decision has been recorded. An immediate continuation can therefore resolve the correct state.

​
When Dari Does Not Speculate

A request remains selector-first when:
  • it is the first turn;
  • there is no recoverable previous decision;
  • the previous model or reasoning effort was removed from the router;
  • the current request needs unsupported images, tools, structured output, streaming, or reasoning;
  • a Describe Your Router rule no longer permits the previous pair;
  • speculative provider setup cannot begin safely.
Dari falls back to normal selection automatically in these cases.

​
Failure Behavior

If the speculative provider attempt fails, Dari waits for the selector and executes the selected route using the router’s retry and fallback settings. If selection fails after a valid speculative route has started, the current turn can continue on that route. Dari keeps it for the next turn rather than recording a selector decision that never completed. Provider usage, retries, and model activity are attributed to the model that actually served the request.

​
Configure It

Open the router in the Dari dashboard, find Speculative Routing, and switch it on or off. Disabling it restores selector-first execution for every turn. Speculative routing is independent from Primary Retries and Model Fallback. Those settings determine recovery after provider failure; speculation determines whether an eligible provider call may begin before selection finishes.

​
Framework Implementations

The self-hosted framework does not perform this orchestration through createRouter(). Its lower-level prepareRoute() and finalizeRoute() APIs expose the phases needed for a custom host to implement it. See Framework Speculative Routing.