we spent a week trying to make automatic routing smarter. the funniest result was how often the dumb deterministic thing won.

everything was green. that was the problem.

the first version of this looked great. outputs came back, the harness said success, and the totals looked believable. then a fallback serialization bug showed us something annoying: the path we were reporting on was not always the path that actually ran.

i kept that busted batch in the evidence set instead of deleting it. it is basically the reason this whole post exists. selected is not executed. if i cannot prove the worker physically ran, i do not get to count it.

  1. configured
  2. selected
  3. executed
  4. consumed
  5. receipted
RECEIPTreceiptedone canonical record exists

falsified here replay, conflict rejection and refusal all needed separate proof

A route being selected is not evidence it executed. This is the chain every claim in this study had to survive, and the point at which each stage was actually falsified at least once.

hover or tab through that thing. every step has been wrong at least once. that is way more useful than a pretty green pipeline pretending nothing weird ever happened.

nanojev was fast. it was also taking the wrong test.

C-Tianyu/NanoJev @ 4a19595e looked awesome at first: about 31 ms p50 on cpu, the right interface, and an early auroc that looked promising.

then i checked what it was actually trained to do. Maze, Snake, ViZDoom Basic, Predict Position. action choice. the boolean / ordered-score stuff was an interface affordance, not the training objective. lol.

fast is not the same as trained for the job. an interface can fit perfectly while the capability underneath it is totally wrong.

jev was the part that actually survived

also, the first “reproduction” was not really a reproduction. we had tested proxy models instead of the model from the paper. reproducing a different system very carefully is still reproducing a different system.

authored144 · famBalAcc
0.9514
recomputed from the raw per-example file
authored144 · ECE
0.0334
calibration, not just accuracy
withheld: AUROC
0.9982
only on the 48-example binary slice — see below
recomputed from the raw per-example capability-evidence file. not copied from a nice-looking summary number.
authored144
144
3 families x 48
direct qwen3.5-4b
0.8056
ece 0.0678
reranker
0.6319
ece 0.0883
perturbations108
1.0000
ece 0.0029

i also dropped the original 146:4 abstention slice as the main target. it is not in this checkout, and with a 97% majority class the headline accuracy mostly tells you the majority class exists. not super helpful for the three-way runtime contract we actually route on.

i stopped asking which model is best

this was the architecture change. asking which model is best? is kind of a fake question. best at what? verification? tool choice? image choice? cheap worker stuff?

now z0intelligence asks the smaller question: what implementation has evidence for this exact job?

JEV_FUNCTIONjev-1.13.0MEASURED
evidence
0.9514 famBalAcc / 0.0334 ECE on authored144; 0.9982 AUROC on the binary slice
consumers
codex, hermes (opt-in)
caveat
reference contract is supported/insufficient/contradicted
same idea everywhere: do not ask which model wins globally. ask what has evidence for this exact job.

the image lane is real. the benchmark win is extremely not real.

we do have a pinned multimodal decision path now: z0int Image Decision 3B, backed by Qwen/Qwen2.5-VL-3B-Instruct at 66285546d2b8. it maps whatever options the task gives us into a bounded label set and reads the option logits directly. no prose parsing. no pretending a generated paragraph is a probability distribution.

published examples
4/8
eight published example images only
generated answer tokens
0
normal single-label path
local target
3080 ti
12 gb
official score
unknown
NO_OFFICIAL_SCORE

a token count needs a provenance tag too

another stupid bug class: counting the last assistant message and calling it the whole run. omp now accumulates provider-reported input/output usage across the whole trace, and it only gets to say complete when every observed turn actually carried provider usage.

routing is boring until you prove the worker actually ran

one measured batch returned 20/20 outputs. 6 providers physically completed work across 25 attempts, with 5 real fallbacks. that is the number i care about now, not just “router said provider x.”

  • cerebrasqwen-3.8-27b7 taskscap 32 policy
  • groqopenai/gpt-oss-20b3 taskscap 16 policy
  • deepseekdeepseek-chat3 taskscap 32 policy
  • grokgrok-4.20-0309-non-reasoning2 taskscap UNKNOWN
  • nousinclusionai/ling-3.0-flash-sante:free2 taskscap UNKNOWN
  • nvidiaopenai/gpt-oss-20b2 taskscap 12 policy
  • localqwen2.5:3b1 taskcap 1 policy
Admission control — measured
32 admitted32 capped before provider HTTPpeak inflight 32

counted local HTTP fixture — NOT 64 paid provider calls and NOT two Kubernetes replicas

six providers physically did work. the cap numbers are policy inputs; only the 32-admission test is measured here.

we also forced a real failure. an invalid groq credential produced real HTTP 401, then Cerebras returned the expected output. OpenRouter and Vercel produced real 403 responses followed by successful Cerebras calls.

free is nice. free is also not an architecture.

the free path is intentionally boring: compiled python route, one authority, one executor. replay and conflict rejection were proven by replacing the pod instead of trusting an assertion.

provider / model
nemotron-3-super-120b-a12b:free
openrouter
returned
42
55 in / 38 out
provider-reported cost
$0.00
free_only route
receipt latency
790.449 ms
1 observation
  1. harness event native before-agent-start hook
  2. authority identity/fingerprint CAS + admission
  3. executor stateless, no ledger
  4. provider OpenRouter free route, max_price 0
  5. canonical receipt exactly one physical start/completion
the three boring invariants that keep duplicate work from becoming real money
rulestatusevidence
completed duplicate → replayverifiedstored output returned after pod replacement; exactly one canonical physical start/completion
same trace, different request → rejectverifiedHTTP 400, no second physical execution
uncertain prior execution → refuseverifiedprior uncertain execution refused after pod replacement

cerebras was good. delegation still lost.

The worker was excellent. Parent work still went up.

worker raw quality
60/60
zero required repairs or fallbacks
worker latency p50
625 ms
p95 1,149 ms
parent tokens, equal cache
+522.1
report summary, 12 pairs
parent tokens, equal cache
+1008.1
evidence gate, 8 pairs
same frozen 60-pair cohort. positive parent delta means delegation made the parent do more work.
uncached parent input + output, parent-only (a) vs delegated (b)
measureab
uncached parent input325,894396,074
parent output32,68131,065
cached parent input884,480861,056
uncached input + output358,575427,139
worker input / output0 / 015,960 / 44,220

the parent cannot save tokens if it still reads the whole original task plus a giant worker response.the next thing to test is typed / compressed worker output, not just an even better worker model.

its worker output did contribute — the reviewed B-arm implementation produced its own report. Contribution is not savings.

then five lines of dumb rules beat the learned routers

this is the result that made me laugh. on a grouped sealed split of 77 real decision states, the deterministic keyword rule beat both learned fast paths.

sealed real-trace tool-family routing · 4 classes · chance 25%
systemaccuracymacro f1p50
majority READ_SEARCH51.95%0.1709—
5-line keyword rule74.03%0.6906—
Laya 421M19.48%0.2095366 ms
Julia-1 R110.39%0.091247 ms

julia was not broken. honestly that made the result more useful.

before throwing julia away, i fixed the adapter to render state exactly the way julia does. typed cpu went back to 1451/2000, exactly matching the publisher. ag news reproduced too at 94/100.

so this is not “oops the adapter was bad.” the model is integrated faithfully. it just does not fit this decision domain. confidence did not save it either: authored144 uncertainty aurocs sit around chance (0.470 p1, 0.469 entropy), with only 1.79% coverage at the ≤10% error budget. confidence ranking is at or below chance; calibration cannot make this a safe cascade gate.

session files sampled
300
of 2,477
outcome-backed actions
35,248
101 sessions
compactions
169
observable events
reachable capabilities
4/5
at the requested >=500-row bar

decision dataset v2 rebuilds per-turn state from omp transcripts and pairs each tool call to its real toolResult. four capabilities already have enough outcome-backed rows:tool_family_select, context_compress_needed, model_family_select, retry_or_escalate.

so what would z0 actually do?

pick a task shape and this shows the route the frozen cohort actually produced, plus the reason it recorded. no imaginary “ideal router” in this widget.

JEV_FUNCTIONjev-1.13.0MEASURED
evidence
Exact evaluated choice contract; operator-authorized paid Jev verification exception — observed: latency 299.9 ms, 372/45 tokens, one canonical physical call
caveat
paid exception under explicit operator authorization; free_only still governs generic workers
frozen decisions from the measured cohorts. the status is evidence level, not a vibes score.