advith —

k3s is fine. The layer above it isn't.

The bottleneck is placing thousands of small, bursty AI jobs across a strange, mixed fleet, and learning from every run so placement gets better. That's a scheduling, capacity and scaling-policy problem, which is your home turf.

One night, 624 job logs
Hover or tap a band. Each one is a different kind of failure.
624jobs
01 What we're building

Canary releases, but for models.

The shadow network. Small local models learn to impersonate Jev, our fast judgment model, on narrow tasks: tool calling, routing, classification. Each one runs in shadow until its evals reach at least 95%. Only then does it take real traffic.

The training data is five years of AI and harness conversation logs. They're labelled and replayed through the shadow network to find what each small model is good at, then fine-tuned.

The workload is thousands of short experiments (replays, evals, fine-tuning runs, builds, test suites), with about 20 agent "leads" submitting jobs at the same time.

Try the promotion rule
tool-call SLMshadowing Jev
88.0% eval agreement
95% gate
request
Jevanswers 100%
SLMmirror only

In shadow: it sees every request and answers none. Its answers are only scored against Jev's.

An illustration of the rule. Real gates are per task, and the open question is what the promotion analysis should look like at scale (chapter 04).

Kerdoios places work across three kinds of resource at once

The rule: autoscale typed capabilities against capacity offers, not sessions. z0intelligence#131

02 The fleet

Two k3s nodes, two phones, and a queue nobody uses.

How a job actually runs

    Kueue is installed with three ClusterQueues, factory, lanes and z0, but they're empty. Jobs actually run through a host-lane wrapper: fleet-build checks heat, memory and disk, then starts a systemd-run scope on the build laptop. They never become Kubernetes Jobs.

    The phones run Linux inside PRoot on Android: no cgroups, no netlink, no systemd. They can't run a kubelet, so they're host-lane workers with battery and thermal gates.

    Observability installed
      03 Failures we actually hit

      Nine failures, most of them already solved by Kubernetes.

      Each card shows what happened, what we did, and the Kubernetes-native answer we suspect exists. That last part is our guess, and it's what we'd like you to correct.

      Failure 1, interactive: why DiskPressure wouldn't clear
      317 GB build disk. The kubelet sets pressure below the threshold, then reclaims until free space reaches threshold + minimum reclaim.

      Scenario: the disk starts at 9% free and a cleanup "deletes" 32 GB. With zstd compression on btrfs, du overstated what deleting would free: about 32 GB deleted, about 4 GB actually freed.

      04 Where a Kubernetes expert changes the trajectory

      Six places we'd rather not reinvent.

      05 Questions we'd love your take on

      Five questions. Tell us where we're wrong.

      Under each question is what our own agents' research leaned toward. It's research only: none of it is deployed, and it's exactly the kind of reasoning we'd like you to check.

      06 Pointers for your agent

      Everything is public.

      Brief for your coding agent