Install, configure, and route your first request.

Full Flux isn't publicly available. Custom-plan users get their copy and install steps from us; talk to us about a Flux custom plan to get access. To try routing today, install flux-os, free and MIT licensed (github.com/vbc1406/flux-os):

pip install "git+https://github.com/vbc1406/flux-os"

In a full Flux copy, the bundled demo routes 25 sample prompts through the full stack with mocked provider calls, with no API keys needed:

python -m router.demo

Export keys for the providers you want eligible. Flux dispatches the right one based on the model it picks. You only need keys for providers whose models you actually want in the candidate pool.

export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export GOOGLE_API_KEY=...
export GROQ_API_KEY=gsk_...
export MISTRAL_API_KEY=...

Full Flux copies can also take keys programmatically. That reference ships with your copy.

If you bind the Flux server to a non-loopback address (anything other than localhost/127.0.0.1), authentication is required unless you explicitly opt out with the unsafe override. Don't disable it on anything reachable from outside your machine.

# Local-only, no auth required
flux serve

# Non-loopback bind: needs a token
export FLUX_AUTH_TOKEN=your-flux-token
flux serve --host 0.0.0.0

flux serve

Defaults to http://localhost:8000. Point any OpenAI-compatible client at /v1.

No SDK swap. Point your existing OpenAI client at the local Flux server and use model="flux-auto".

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="your-flux-token",
)

response = client.chat.completions.create(
    model="flux-auto",
    messages=[{"role": "user", "content": "Summarize this document"}],
)

Because the endpoint is OpenAI-compatible, the integration looks the same as other gateways; the difference is what Flux tracks across a run. See how Flux compares with OpenRouter and LiteLLM.

Full Flux also has a Python API for programmatic control over routing. Its reference ships with your custom-plan copy.

Add routing_priority to shift the cost/quality/latency weighting:

Pass a run ID and step type so Flux can track spend per run and apply per-step quality floors. Supported step types include plan, tool_select, tool_result, reflect, and final. Planning, tool-selection, reflection, and final-answer steps can retain a higher quality floor; mechanical steps like tool_result can use cheaper models.

response = client.chat.completions.create(
    model="flux-auto",
    messages=[{"role": "user", "content": "Plan the next step"}],
    extra_headers={
        "X-Flux-Run-Id": "run-123",
        "X-Flux-Step-Type": "plan",
    },
)

Flux only routes tool-calling requests to models in the catalog marked as supporting function/tool calling. Pass tools the same way you would to the OpenAI SDK directly.

response = client.chat.completions.create(
    model="flux-auto",
    messages=[{"role": "user", "content": "What's the weather in Boston?"}],
    tools=[{
        "type": "function",
        "function": {"name": "get_weather", "parameters": {...}},
    }],
    extra_headers={"X-Flux-Step-Type": "tool_select"},
)

Set a ceiling on a single agent run. Flux reserves the estimated cost of each step before dispatch and blocks any request predicted to exceed the remaining budget, returning a structured summary instead of erroring out.

Before the cap is reached, Flux shifts to cheaper models: degradation starts at 70% of the run budget, a warning is raised at 90%, and Flux hard-stops with a 429 once the budget is fully used. The Flux FAQ covers these thresholds.

resp = asyncio.run(flux.complete(
    prompt,
    user_id="u123",
    run_id="run-123",
    max_run_cost=1.00,          # cap for this whole agent run
    max_cost_per_request=0.01,  # filters out any model whose estimate exceeds this
))

Add customer_id on requests to get per-customer adaptive scores and per-day cost tracking.

resp = asyncio.run(flux.complete(
    prompt,
    user_id="u123",
    customer_id="acct-42",
    max_daily_cost=5.00,
))

• Set FLUX_AUTH_TOKEN before binding to a non-loopback address.
• Run Redis and point Flux at it if deploying more than one worker process.
• Pass user_id / customer_id only from authenticated context, never raw user input.
• Persist adaptive state with adaptive_state_file= so learning survives restarts.
• Export Prometheus metrics to your existing observability stack.
• Only export keys for providers you intend to route to.

All savings and latency numbers on this site come from an offline synthetic evaluation using registry pricing and catalog-rated model quality, run locally against a fixed request set. It is not graded on live production traffic. Benchmark data as of Aug 25, 2026. The full benchmark methodology lists sample count, baselines and environment.

• Up to 80% estimated savings in our offline benchmarks on synthetic requests. Per-baseline offline synthetic estimates: 84.8% vs GPT-4o, 89.3% vs Claude Sonnet 4.6, 82.2% vs Gemini 2.5 Pro.
• 500 requests, fixed set, same set across all three baselines; 30 catalog models across five providers, priced from the committed registry snapshot (Aug 2026).
• 500/500 routing decisions completed under 10 ms in the local benchmark.
• 0.31 ms median, 0.60 ms P95 routing-decision latency. Excludes provider inference, network latency and full HTTP gateway overhead.

• No PyPI package yet. Install from a clone in editable mode.
• Budgets are enforced per run, not pooled across concurrent agents over a time window.
• Not a medical, legal, or safety decision system. High-stakes detection is rule-based, not a substitute for application-level safety controls.
• Native incremental streaming isn't implemented for every provider yet.
• Savings and latency numbers are from an offline synthetic benchmark, not production traffic.