Reference

Inference API

One HTTP surface for every hosted checkpoint. OpenAI-compatible chat completions, plus a raw generate route for non-chat models.

Authentication

Send a bearer token. Free-tier tokens are rate limited per hour and per model class.

Authentication

curl https://api.osai.dev/v1/chat/completions \
  -H "Authorization: Bearer $OSAI_TOKEN" \
  -d '{"model":"Qwen/Qwen2.5-72B-Instruct","messages":[{"role":"user","content":"hi"}]}'

Streaming

Set stream to true for server-sent events. Each event carries a delta and a cumulative token count.

Streaming

for chunk in client.chat.stream(model=MODEL, messages=msgs):
    print(chunk.delta, end="", flush=True)

Limits and errors

429 responses carry a Retry-After header. Cold checkpoints return 503 with an estimated warm-up time rather than queueing indefinitely.

← Quickstart Model cards →