Show HN: Flux, an open-source LLM router with per-agent-run budgets
Hi HN, I'm Vandith, an engineering student building this mostly on my own.
Most LLM routers and cost controls I've looked at operate one request at a time, and when they do try to control spend, it's usually one of two things: a hard meter that cuts you off the moment you hit a number, or just a dashboard. It shows you what you're spending, but that's visibility, not control.
Agentic and wrapper workflows both break that model, just differently. A wrapper call is simple but unpredictable. The person asking is your customer, not you, so you can't assume every request is the same difficulty. An agentic run is worse: it might retry, spawn subagents, or swing from an easy step to a genuinely hard one halfway through, and none of that is visible from any single call in isolation.
So I built Flux around a different unit for the agent case: the run, not the request.
Flux is a self-hosted, OpenAI-compatible LLM proxy. For a single wrapper call, it just picks the cheapest model that can actually handle that request. For an agent run, every call sharing a run ID also shares a budget, so the whole run is governed as one economic unit instead of many independent decisions. What a call can afford depends not only on what that call needs, but on what the agent has already spent getting there. As teams start caring as much about spend per output as the output itself, treating every call like it needs your most expensive model stops making sense. That's the gap I'm trying to close.
How the budget works
0-70% normal routing
70% cost pressure increases, prefer cheaper capable models
90% aggressive cost conservation
100% hard stop
Calls and subagents sharing the same run ID share the same budget state. So a 30-call workflow is governed as one economic unit rather than 30 independent routing decisions.
70/90/100 is a heuristic, not something I think is mathematically optimal. That's actually one of the things I'm hoping to get feedback on.
Routing
The current routing table comes straight from published benchmark results. When I add a model, I add its benchmark numbers and it's eligible for routing. Nothing fancier than that.
For each call, Flux filters to models meeting a minimum quality floor for the task, then ranks the survivors by cost. As the run consumes more of its budget, the routing policy becomes increasingly cost-sensitive.
Try it
git clone https://github.com/vbc1406/flux-router cd flux-router pip install -e . python -m router.demo # no API keys needed
Add provider keys when you want to run it for real with python examples/chat.py. It's not on PyPI yet.
What I've actually tested
On 7 live agentic-eval samples, Flux cut cost by roughly 80-90% versus a Claude Sonnet 4.6-only baseline while maintaining comparable evaluator scores. I'm giving a range rather than a precise decimal on purpose. n=7 doesn't earn that much precision, and I haven't tested this widely enough yet to stand behind a single number. It's a sanity check, not a statistically meaningful benchmark.
I also have a larger fixture-based result, but because it's evaluated against the same routing table Flux uses for selection, I consider it circular and don't treat it as evidence.
The more useful test used a parent agent plus two subagents on a weather + air-quality task. Nothing mocked. 21 HTTP steps against live Mistral calls, run-ID propagation working across all three agents, tool round trips landing on actual Open-Meteo responses.
Building this test broke Flux twice: a trailing tool-message handling bug, and a missing OpenAI→Anthropic tool-call format translation. Either one would've thrown a real 400 in production. Fixed now.
I also forced OpenAI failures mid-run. Flux failed over to Claude Sonnet 4.6 in one retry. After five consecutive failures, the provider circuit breaker opened and the next request routed around OpenAI without another failed call.
Where it's still wrong
An independent audit of the routing table against published benchmarks found two real routing errors.
Flux currently ranks gpt-oss-120b above gemini-3-flash-preview for reasoning despite a 9-10 point GPQA gap, and picks Claude Haiku over Qwen3.6-27B for code review despite Qwen scoring higher on SWE-bench (77.2% vs. 73.3%).
They're on the list to fix.
Why another router?
LLM routing itself isn't new. RouteLLM helped establish that routing between models can materially reduce inference cost, and newer benchmarks and production routers have pushed request-level routing much further since then.
I'm not trying to claim Flux has a universally better prompt-to-model classifier.
What I'm exploring is a different control problem:
A router asks: which model should handle this request?
Flux also asks: given what this agent has already spent, what should its next request be allowed to cost?
Whether run-level budget-aware routing deserves to be its own abstraction, or just ends up as a standard gateway feature eventually, I honestly don't know yet. That's part of why I'm posting this.
One question for HN
The part I'm least confident about is the degradation policy.
If you're running agents in production, what should happen when a run approaches its budget? Degrade to cheaper models? Stop outright? Ask for approval? Something else?
And would you want that policy globally, per agent, or per task?
If you're running this at a scale where the defaults don't quite fit, different quality floors, different degradation behavior, whatever it is, I'm happy to talk through it.
Flux is AGPL-3.0 and open source. The mocked demo doesn't require API keys. I'm around today and happy to go deep on any of the implementation or routing decisions. Reply here, or if something's broken or you'd rather just tell me directly, email me at vandith@flux-llm.com.