Inference protocols

One account and signed receipts across text, embeddings, images, and audio.

Responses API

POST /v1/responses supports stateless text and image inputs, function calls and function outputs, JSON formats, and server-sent events. It uses the same routing, spending reservations, masking, and receipts as chat completions. Send the conversation in input. Storage defaults to false; stored response IDs, background jobs, hosted tools, and reasoning controls are rejected explicitly.

Responses

const response = await client.responses.create({
  model: "your-model",
  input: "Explain vector search",
  store: false,
  max_output_tokens: 300,
});

Media and embeddings

Find available configurations at GET /v1/media/offers or the media marketplace. Offers are separate from chat listings. Their endpoint probes validate response structure; they do not establish model identity, image quality, or transcription accuracy. Select thin.verification: "probed" explicitly. A key requiring verified models will continue to reject them.

POST /v1/embeddings: text or token arrays, float or base64 vectors. Fixed price per request, up to 1 MiB of serialized input.
POST /v1/images/generations: JSON image generation, billed per image, 1–10 images. Size and quality must match the offer configuration.
POST /v1/audio/speech: binary audio, billed per 1,000 Unicode characters, prorated and rounded up to a microdollar. Maximum 4,096 input characters.
POST /v1/audio/transcriptions: multipart file upload up to 25 MiB, JSON output, fixed price per request.

These are marketplace quotes, not upstream token or minute pricing. For a request-based offer, the provider agrees to serve any valid input within these limits for the listed price. Model-specific limits may still reject a request without charging the buyer.

Embedding with a total cost ceiling

POST /v1/embeddings
{
  "model": "provider/embedding-model",
  "input": ["First document", "Second document"],
  "max_cost_micro": 10000,
  "thin": {"verification": "probed", "regions": ["eu"]}
}

For multipart transcription, pass thin as a JSON string. Speech responses expose the request, cost, offer and receipt IDs in x-thin-* headers. JSON responses include a thin object. Signed receipts include usage_units with quantity, unit, and price. Upstream embedding token counts are informational; the fixed quote determines the charge.

Explicit scope

Media supports price routing, sequential failover, region and seller restrictions, declared ZDR, and total cost caps. Media streaming, image edits, translation, price locks, batch jobs, token-price filters, caching, masking, and latency objectives are not supported. Unsupported request options fail before billing. The gateway does not retain media prompts, files, vectors, or outputs in traces.

Connect capacity

Open Provider → Media endpoints. Set a model, protocol, fixed options, unit price, serving region, and encrypted provider credential. Publishing runs one small probe against your endpoint; your upstream may charge for it. Pause or remove an offer at any time. Removal destroys its stored credential. Change prices or options by publishing a new offer.