Rescue & hedging

Two ways the exchange keeps a call alive when a seller doesn't.

Failover

If an offer fails before responding, the router tries the next eligible offer, up to max_attempts (default 3). thin.attempts tells you how many it took. If all fail, you get upstream_failed and are not charged.

Rescue

If a stream breaks after the seller has started answering, the exchange continues the same answer on the next eligible offer and keeps streaming into your connection. Offers that declare prefill support continue from the partial assistant message directly. The response is marked thin.rescued: true.

Rescue is on by default; turn it off per request:

json

"thin": { "rescue": false }

Hedging

For latency-sensitive calls, set hedge_after_ms. If the first offer hasn't produced a first token by then, the exchange starts the next offer in parallel and keeps whichever answers first; the other is cancelled. The response is marked thin.hedged: true.

The cancelled offer already received your prompt, so its seller is paid for the input tokens it processed (an estimate, at that offer's input price). That amount is shown separately as hedge_cost_micro on the receipt and included in thin.cost_micro. Hedging only starts a second leg when the first is slow, so most hedged calls cost nothing extra.

json

"thin": { "hedge_after_ms": 800 }

Or the hedge800 policy-path token, or X-Thin-Hedge-Ms: 800.