Install, configure, and route your first request.

Flux isn't published to PyPI yet. Install from a clone in editable mode.

git clone https://github.com/vbc1406/flux-router.git
cd flux-router
pip install -e .

Want to try it with no API keys first? The bundled demo routes 25 sample prompts through the full stack with mocked provider calls:

python -m router.demo

Export keys for the providers you want eligible. Flux dispatches the right one based on the model it picks — you only need keys for providers whose models you actually want in the candidate pool.

export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export GOOGLE_API_KEY=...
export GROQ_API_KEY=gsk_...
export MISTRAL_API_KEY=...

Prefer passing keys programmatically instead of env vars?

# One key for all providers (e.g. an OpenAI-compatible gateway)
flux = make_flux(api_key="sk-...")

# Explicit per-provider keys (overrides env vars)
flux = make_flux(api_keys={
    "openai": "sk-...",
    "anthropic": "sk-ant-...",
})

Resolution order per request: explicit api_keys= / env var → per-request provider_api_key → legacy single api_key=.

If you bind the Flux server to a non-loopback address (anything other than localhost/127.0.0.1), authentication is required unless you explicitly opt out with the unsafe override — don't disable it on anything reachable from outside your machine.

# Local-only, no auth required
flux serve

# Non-loopback bind — needs a token
export FLUX_AUTH_TOKEN=your-flux-token
flux serve --host 0.0.0.0

flux serve

Defaults to http://localhost:8000. Point any OpenAI-compatible client at /v1.

No SDK swap — point your existing OpenAI client at the local Flux server and use model="flux-auto".

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="your-flux-token",
)

response = client.chat.completions.create(
    model="flux-auto",
    messages=[{"role": "user", "content": "Summarize this document"}],
)

Prefer programmatic control over routing? Use the Python API directly.

import asyncio
from router.flux import make_flux

flux = make_flux()  # reads keys from the env vars above

resp = asyncio.run(flux.complete(
    "Explain backpropagation in two sentences.",
    user_id="u123",  # required, from your authenticated context
))
print(resp.text)
print("routed to:", resp.model.display_name)

Add routing_priority to shift the cost/quality/latency weighting:

Pass a run ID and step type so Flux can track spend per run and apply per-step quality floors. Supported step types include plan, tool_select, tool_result, reflect, and final. Planning, tool-selection, reflection, and final-answer steps can retain a higher quality floor; mechanical steps like tool_result can use cheaper models.

response = client.chat.completions.create(
    model="flux-auto",
    messages=[{"role": "user", "content": "Plan the next step"}],
    extra_headers={
        "X-Flux-Run-Id": "run-123",
        "X-Flux-Step-Type": "plan",
    },
)

Flux only routes tool-calling requests to models in the catalog marked as supporting function/tool calling — pass tools the same way you would to the OpenAI SDK directly.

response = client.chat.completions.create(
    model="flux-auto",
    messages=[{"role": "user", "content": "What's the weather in Boston?"}],
    tools=[{
        "type": "function",
        "function": {"name": "get_weather", "parameters": {...}},
    }],
    extra_headers={"X-Flux-Step-Type": "tool_select"},
)

Set a ceiling on a single agent run. Flux reserves the estimated cost of each step before dispatch and blocks any request predicted to exceed the remaining budget, returning a structured summary instead of erroring out.

resp = asyncio.run(flux.complete(
    prompt,
    user_id="u123",
    run_id="run-123",
    max_run_cost=1.00,          # cap for this whole agent run
    max_cost_per_request=0.01,  # filters out any model whose estimate exceeds this
))

Add customer_id on requests to get per-customer adaptive scores and per-day cost tracking.

resp = asyncio.run(flux.complete(
    prompt,
    user_id="u123",
    customer_id="acct-42",
    max_daily_cost=5.00,
))

• Set FLUX_AUTH_TOKEN before binding to a non-loopback address.
• Run Redis and point Flux at it if deploying more than one worker process.
• Pass user_id / customer_id only from authenticated context, never raw user input.
• Persist adaptive state with adaptive_state_file= so learning survives restarts.
• Export Prometheus metrics to your existing observability stack.
• Only export keys for providers you intend to route to.

All savings and latency numbers on this site come from an offline synthetic evaluation using registry pricing and catalog-rated model quality, run locally against a fixed request set — not live-graded production traffic. Last updated Aug 25, 2026.

• Up to 89% estimated savings in our benchmarks: 84.8% vs GPT-4o, 89.3% vs Claude Sonnet 4.6, 82.2% vs Gemini 2.5 Pro.
• 500 requests, fixed set, same set across all three baselines; 30 catalog models across five providers, priced from the committed registry snapshot (Aug 2026).
• 500/500 routing decisions completed under 10 ms in the local benchmark.
• 0.31 ms median, 0.60 ms P95 routing-decision latency — excludes provider inference, network latency and full HTTP gateway overhead.

• No PyPI package yet — install from a clone in editable mode.
• Budgets are enforced per run, not pooled across concurrent agents over a time window.
• Not a medical, legal, or safety decision system — high-stakes detection is rule-based, not a substitute for application-level safety controls.
• Native incremental streaming isn't implemented for every provider yet.
• Savings and latency numbers are from an offline synthetic benchmark, not production traffic.
• Flux is not a community project in the contribution sense — issues are welcome, pull requests are not currently accepted.