Akka Token Shredder

Economics, architecture,
and the optimization loop

How token cost is reduced, how the system is deployed, how the optimization loop runs, and how cost is governed.

Economics
01 · Baseline
annual spend = developers × $ / dev·month × 12
$60.0Mper year · the annual bill
$2,500per developer · month (assumed)
2,000developers
cost drivers: GPT-5.5 · Codex (⅔) + Opus 4.8 · Claude Code (⅓)
02 · Price risk
ONE DEVELOPER · 3 MONTHS · 57.9B TOKENS metered at frontier API GPT-5.6 Sol $170K Fable 5 $150K ≈ $320K · peak days ~$10K billed on subsidized seats $1,300 ~99.6% how far frontier API prices must fall to match subscriptions
03 · Unit cost
cost/task = price/tok × tok/task 1.0 0 1.00 leased 0.30 self-provide 0.17 adapted × 0.30 price/tok · × 0.55 tok/task
04 · Savings from SLMs
$60.0M leased today $11.2M serve $10.2M train $1.0M self-provide $48.8M saved · 81% your GPUs: capex · managed inference: metered opex
05 · Break-even
time (days) → leased self-provided fixed $1.0M crossover ≈ 8 days cost / verified task $0.0152 $0.0031 −80% · +2 pts quality 3,900 → 760 tokens / task · fewer tokens is why the cost falls
Sources: OpenAI & Anthropic API pricing — GPT-5.5 $5/$30, Opus 4.8 $5/$25, GPT-5.6 Sol $5/$30, Claude Fable 5 $10/$50 per 1M tokens (Jun–Jul 2026) · Fireworks serverless & fine-tuning pricing · OpenRouter, "Open Weight Models that Matter," Jun 2026. Baseline: 2,000 developers × $2,500/dev·month (assumed effective rate) = $60.0M/yr. Price risk: one developer, 3 months = 57.9B tokens, billed ≈$1,300 on subsidized seats vs ≈$320K metered at today's frontier (≈190–250×).
Architecture

Route, serve, train, and govern

CLIENT ENVIRONMENT HARNESSES Cursor Claude Code Codex GitHub Copilot CONNECTORS BYO-agent pluginendpoint config · trace capturePLUGIN Log forwarderproduction logs → AI GatewayLOGS Repo & CI connectioncodebases · sandbox RL deploysREPO AKKA MANAGED PLANE · SINGLE TENANT · SOVEREIGN Governance consent · policy · RBAC · audit · checks POLICY Scorecards & dashboards quality · velocity · cost · autonomy SCORECARD AI Gateway PROXY OpenAI-compatible ingress tier select · adapter hot-swap · cache canary · shadow · rollback Frontier fallback TEACHER GPT-5.5 · Opus 4.8 external API Continuous training Trace storeagent traces Scrub & curatePII · secrets Judges & evalsreward models RL envssuites · sandbox Synthetic datateacher · RLVR SFT · DPO · RLLoRA out · held for eval INFRASTRUCTURE · COMPUTE Inference endpoints · elastic auto-scale · base + adapters SERVING GLM-5adapted · high DeepSeek V4 Proadapted · high Kimi K2adapted · economy adapters specialize by domain — code review · deploy · security · SRE SPECIALISTS Your GPUsvLLM serving + verl training · in your VPCsovereign · air-gapped · you own the weightsYou Serve FireworksFireworks serving + Fireworks tuning · their cloudmanaged · open-weight · you own the adaptersThey Serve serve · route · canary · shadow serves rollouts adapter traces

solid yellow = serving / request flow

grey dashed = training / data flow

The orchestration flows

Functional Requirements

T1 – T4

Training loop

Which agents are worth training, the run itself, the on-policy rollout cycle, and the ladder that proves a candidate.

Akka orchestration · hours to days
S1 – S3

Serving loop

The hot path, how a candidate earns real traffic, and the drift watch that sends a decayed agent back to T1.

Gateway · milliseconds
G1 – G3

Governance

The one orchestrated pass that admits and captures every request, the standing requirements, and which controls need a person.

Gateway + policy
D1 – D2

Scorecards

What the loop costs and whether quality held, per audience — and the decisions it is waiting on a person to make.

Read side
Training loop · 1 of 4
T1

Target & Admit

Akka orchestration

Which agents are worth training. A policy filter, two checks the agent must pass, and two scores that rank whatever survives — returning a named reason for every verdict.

TouchesTRACE STOREJUDGES & EVALSPOLICYSCORECARD

Flowpolicy filter → four scheduled scorers → one verdict

1 · WHICH AGENTS MAY BE SCORED AT ALL Trace storepast requests and repliesgrouped by agent Eligibility filterregulated task · training-consent scope · data residencyplus anything the customer has ruled off-limitsA9 Vetoedscoring is skipped · the reason isrecorded against the agent 2 · SCORE EACH AGENT — FOUR SCORERS, EACH ON ITS OWN SCHEDULE the agents allowed through — scopes who each scorer looks at TRIGGER · NIGHTLY TIMER NarrownessA1 how varied are its tasks:shape · wording · tools · output WRITES · narrowness scoreAFFECTS RANK · 0.6 of the score TRIGGER · ON GRADER-DEF CHANGE GradeabilityA2 MUST PASS · a zero disqualifies the agentcan a machine check the answer? WRITES · gradeability scoreif only a person can, this fails TRIGGER · HOURLY TIMER Volume sufficiencyA3 MUST PASS · a zero disqualifies the agentenough distinct examples, not repeats WRITES · volume scoreand arriving fast enough to keep up TRIGGER · EVERY TICK Token economicsA4 spend = Σ tokens × pricesplit by context, reasoning, retry WRITES · economics scoreAFFECTS RANK · 0.4 of the score POST /v1/embeddings POST /v1/chat/completions POST /v1/embeddings inference tier — endpoint and serving path in the table below reads the four current scores · any new score re-runs this 3 · COMBINE INTO ONE VERDICT ADMISSION = gradeability_passed × volume_passed × ( 0.6·narrowness + 0.4·economics )A6the two must-pass checks multiply, so either can zero the result · spend stops counting above 10M tokensevery verdict ships with its per-dimension breakdown ADMIT→ T2 training run KEEP_FRONTIER + reason"no deterministic grader for this task shape""only 31 distinct examples after removing repeats"

solid = invocation  ·  grey dashed = read  ·  red dashed = failure path  ·  green = must pass, a zero disqualifies the agent  ·  yellow = affects rank only

Trigger

A schedule, or a hand-back from S3 when a promoted model decays. Each run scores one agent across a window of its recent traffic, so there is one verdict per agent rather than one per request.

State & durability

Each scorer saves its result against the agent it scored. The final verdict is a live view over those four saved scores, so whenever any scorer saves a new one, the verdict recomputes. Nothing schedules the verdict itself — it moves whenever a scorer moves. A crash part-way through resumes at the last completed step.

Failure & compensation

A failed check renders KEEP_FRONTIER with the reason named — "ungradeable, no deterministic grader for this task shape", "only 31 distinct examples, below the minimum", "policy veto: regulated task". The verdict and its reason are written to the record.

Inference-tier interfaces

Called byEndpointYour GPUs · you serveFireworks · they serveWhat comes back
NarrownessPOST /v1/embeddingsvLLM serving an embedding model inside your VPCFireworks embeddings endpointvectors it can cluster, to measure how much the wording varies
GradeabilityPOST /v1/chat/completionsvLLM serving the judge model inside your VPCFireworks, or the frontier TEACHER tiera yes/no on whether a checkable rubric can be built for an unseen task shape
Volume sufficiencyPOST /v1/embeddingsthe same vLLM endpoint, results cachedthe same Fireworks endpointwhich requests are near-duplicates, so repeats collapse to a count of distinct examples
Token economics— nonea tokenizer library and a price book, both in-processtoken counts and spend per task shape

Both postures speak the same OpenAI-compatible API, so only the base URL differs between them — this workflow's code is identical either way.

Components to buildevery component T1 needs, and the scaffolding each one depends on

That code was written for the sales assessment tool — a one-shot run over a customer's exported logs, ranking which use cases and how many tokens are eligible for training. T1 is the same algorithm running continuously. The scoring logic ports directly; the orchestration does not.

TMRTimed Action — fires on a schedule
CONConsumer — reacts to a stream of changes
ENTEntity — durable state, one instance per agent
VIEWView — a query that recomputes when its sources change
WFWorkflow — durable multi-step process, resumes after a crash
ComponentTypePurposeScaffolding needed
Workflows4durable multi-step processes — each resumes at its last completed step
NarrownessScoringWorkflowWFReads the agent's recent window, embeds a bounded sample, clusters it, and writes the A1 sub-scoreTrace-window reader · embedding client · reservoir sampler
GradeabilityScoringWorkflowWFSorts the agent's traffic into grader tiers and writes the A2 sub-scoreGrader-definition registry
VolumeScoringWorkflowWFCollapses near-duplicates to a distinct-example count, measures accrual, and writes the A3 sub-scoreNear-duplicate collapse over the window
EconomicsScoringWorkflowWFDecomposes spend across system, context, reasoning, output and retry, and writes the A4 sub-scorePrice book · tokenizer
Timed Actions3schedule only — each starts a run and does no work itself
NarrownessTimedActionTMRNightly. Starts one narrowness run per eligible agent — it does no scoring of its ownSelf-rescheduling timer, bootstrapped at startup
VolumeTimedActionTMRHourly. Starts one volume run per eligible agentSelf-rescheduling timer
EconomicsTimedActionTMREvery tick — cadence undecided, see below. Starts one economics run per eligible agentSelf-rescheduling timer
Consumers2react to a stream of changes
GraderDefinitionConsumerCONReacts to a grader-definition change and starts a gradeability run for each agent it affectsGrader-definition registry to subscribe to
NewCandidateConsumerCONA8. Streams admission changes and emits who newly qualifiesChange feed off the reducer view
Entities8durable state, one instance per agent
NarrownessScoreEntityENTHolds the A1 sub-score for one agent
GradeabilityScoreEntityENTHolds the A2 sub-score for one agent
VolumeScoreEntityENTHolds the A3 sub-score for one agent
EconomicsScoreEntityENTHolds the A4 sub-score for one agent
AdmissionEntityENTPer-agent verdict — TARGET or KEEP_FRONTIER, its named reason, and the A7 estimate
TrainingStateEntityENTLifecycle: none → admitted → training → in-ladder → promoted or retired. Control plane only; the serving path never reads it
EligibilityPolicyEntityENTA9 pre-filter vetoes and late modulators, per agentPolicy source of record
InteractionEntityENTOne sampled interaction — the trace substrate every scorer reads
Views2queries that recompute when their sources change
AdmissionReducerViewVIEWA6 composite over the four sub-score entities. Any write re-runs it; nothing schedules it
InteractionsByAgentViewVIEWGroups interactions by agent so a scorer can read one agent's window

Four sub-score entities, not one. The assessment tool holds all four scores in a single entity and gets away with it because one process computes them serially. Under T1's independent cadences, four writers against one entity is last-writer-wins — the hourly volume write would clobber the nightly narrowness write. Splitting them is what makes the reducer-as-a-view design work at all.

The trigger is not the work. A timed action has no journal and no steps — on failure it re-runs from the top, indefinitely by default. So each trigger does exactly one thing: start the run. The run is a workflow, which is what makes "resumes at the last completed step" above true rather than aspirational. There is no workflow that orchestrates all four; each scorer runs on its own schedule, and the view recomputes whenever any of them writes.

How often is "every tick"? Engineers must decide this, and it is the biggest open question on this slide. It sets how many workflow instances exist at once, so it is a cost decision before it is a correctness one. The instinct to batch is probably wrong: with a large fleet under evaluation, many small incremental runs are likely better than few large ones — a long cadence lets a backlog of un-scored agents build up, and the deeper that backlog, the staler every verdict downstream of it. Start frequent and incremental, then measure.

What we build vs. what we wire inwhich parts are our algorithms, and which are someone else's capability

Algorithms we buildno external service — this is the IP
CapabilityNodeWhy it exists
Template extraction + structural entropyA1Measures how much the request shape varies
Tool-path entropyA1Measures how much the call sequence varies
Output-space classificationA1A fixed label, JSON or SQL scores narrow; open prose scores wide
Semantic clustering over embeddingsA1Turns vectors into a dispersion number — streaming, over a bounded sample
Near-duplicate collapseA3Repeats must not count as distinct training examples
Grader-tier detectionA2Sorts traffic into deterministic, model-judgeable, human-only, or ungradeable
Token accounting + waste decompositionA4Splits spend across system, context, reasoning, output and retry
Composite gatingA6Two must-pass checks multiply; two weights rank whatever survives
Estimate waterfall + feasibilityA7Magnitude and confidence, computed separately and never multiplied together
Eligibility policy evaluationA9Applies vetoes and modulators, always carrying a reason
Capabilities we wire insomeone else's, called for a named reason
CapabilityKindWhy T1 needs itConstraint
Text embeddingModel inferenceA1 semantic entropy and A3 near-duplicate collapse both need vectorsMust run in-VPC. Logs may not leave the customer boundary — that rules out hosted embedding APIs
TokenizerLibrary, in-processA4 counts tokens whenever the source log doesn't carry themMust match the frontier model's encoding, or the spend number is wrong
Price bookReference data, not an APIA4 turns token counts into moneyNeeds a refresh path — prices move
Judge / chat inferenceModel inferenceA2 only, and only in V2 — asks whether a checkable rubric could be built for a task shape it hasn't seenOptional. V1 ships without it: a task with no deterministic grader simply fails the gradeability check
Trace storeData substrateEvery scorer reads past requests and replies, grouped by agentWindowed and sampled reads, never full scans
Grader-definition registryData substrateA2 reads which graders exist; its change events are what trigger the scorerNot built — and nothing today subscribes to it
Governance / policy sourceData substrateA9 reads regulated status, training consent, and data residencyCustomer-owned. A wrong answer here is a compliance failure

Sanitization — and where it must not be applied. Akka's sanitizer masks matched patterns (email, phone, card, IBAN, IP, plus custom rules) and can be called from any component, not only an agent. Enable it on logs, where it costs nothing, and apply it to the A2 judge call if that judge is not served in-VPC — the one place in T1 where text leaves the boundary.

Do not sanitize before tokenizing or embedding. Masking rewrites the text, and this workflow exists to measure text: a masked span changes the token count that A4 prices, shifts the vectors A1 and A3 cluster, and can alter the extracted template. It would corrupt the scores silently, since nothing would fail. Measure first, sanitize after.

How the embedding capability is wired today — a local daemon if one is running, else an embedding model bundled in-process, else a deterministic fallback that reports itself as degraded. All three keep logs inside the boundary. Production resolves the same capability against the vLLM-on-KServe stack the product already deploys: same interface, different base URL.

Training loop · 2 of 4
T2

Training Run

Akka orchestration

Days long and resumable at every stage. The orchestrator holds the run, so a GPU failure costs a step. The two deployment postures both train — they simply do it through different interfaces.

TouchesSCRUB & CURATESYNTHETIC DATASFT · DPO · RLTEACHERYOU SERVETHEY SERVE

Flowone trigger · sequential stages · posture branch

TRIGGER · T1 ADMITS AN AGENT AND TrainingState(agent) SHOWS NO RUN ALREADY IN FLIGHT 1 · BUILD THE TRAINING SET Dataset buildcurated traces from the record, plustrajectories harvested from the frontierWRITES · hashed dataset Synthetic augmentationadded where a reward can be checkedautomatically, and skipped where it cannotWRITES · augmented set POST /v1/chat/completions on the TEACHER tier, routed through the gateway 2 · TRAIN — SFT · DPO · RL. THE POSTURES DIVERGE HERE Your GPUsYOU SERVElaunch a verl job on capacity you scheduleyou hold the split between training and servingcheckpoints are written to your own storerollouts run through T3 against the live checkpoint FireworksTHEY SERVEpost a fine-tuning job and poll the handlecapacity is theirs to manage — you reserve nothingthe finished adapter is fetched when it is readyrollouts run through T3 the same way 3 · EXIT Emit checkpoint~50MB adapter · base weights unchangedevery stage hashed, so a failure resumes here Exit check — did the run work?converged · no collapse · better than where itstarted, on the trainer's own held-back split CheckpointReady→ T4 starts the ladderT4 compares it to the incumbent, not this RetiredTrainingState → retiredthe incumbent never moved

solid = invocation  ·  grey dashed = read  ·  red dashed = failure path  ·  green = the run passed its own check

Trigger

T1 admits an agent, and TrainingState(agent) shows no run already in flight. That state is a control-plane entity — the serving path never reads it.

State & durability

Long-running, with durability at every checkpoint. Each stage records its inputs by content hash, so a completed run is reproducible and a failed one resumes at its last checkpoint. TrainingState moves admitted → training → in-ladder, or to retired.

Failure & compensation

Infrastructure failure resumes from the last checkpoint. A run that fails its exit check is retired and never reaches the ladder. The incumbent keeps serving throughout — the request plane is untouched until T4 promotes.

Inference-tier interfaces

Called byEndpointYour GPUs · you serveFireworks · they serveWhat comes back
Dataset buildPOST /v1/chat/completionsthe frontier TEACHER tier, reached through the gateway like any other callteacher trajectories to learn from
Training job— not an inference calllaunch a verl job against capacity you reservedpost a fine-tuning job; capacity is managed for youa job handle to poll, then an adapter

Inference is uniform across postures, so the teacher call is identical on both. Job submission is not — these are two different services, and the own-GPU path is the only one where you reserve capacity.

Components to buildevery component this workflow needs, and the scaffolding each one depends on

One long-running run per admitted agent. The substrate branch is real — own-GPU and managed tuning are not one API behind a flag, so the training step forks and everything around it stays common.

WFWorkflow — durable multi-step process, resumes after a crash
CONConsumer — reacts to a stream of changes
ENTEntity — durable state, one instance per key
VIEWView — a query that recomputes when its sources change
ComponentTypePurposeScaffolding needed
Workflows2durable multi-step processes — each resumes at its last completed step
TrainingRunWorkflowWFDataset build → augmentation → train → emit checkpoint → exit check. Long-running; resumes from the last completed stageContent-hash manifest per stage · substrate branch
TeacherHarvestWorkflowWFReplays logged tasks through the frontier tier to harvest teacher trajectories for the datasetGateway route to the TEACHER tier
Consumers1react to a stream of changes
AdmissionConsumerCONReacts to a TARGET verdict from T1 and starts a run — but only when training state shows none in flightAdmission change feed
Entities3durable state, one instance per key
TrainingStateEntityENTShared with T1. The interlock that stops a second run starting while one is in flight
CheckpointRegistryEntityENTCheckpoints produced for an agent, addressed by content hashArtifact store
DatasetManifestEntityENTWhat went into a run — sources, filters, split — by content hash, so a result is reproducible
Views1queries that recompute when their sources change
RunsByAgentViewVIEWRun history per agent; what the in-flight interlock and the dashboards read

The substrate branch is not a flag. Own-GPU launches a training job against capacity you schedule and writes a checkpoint to your store. Managed tuning posts a job and hands back an adapter, and has no capacity reservation at all. Only the training step differs; dataset build, augmentation and the exit check are common to both.

The exit check is not T4's first rung. This asks whether the run produced a usable checkpoint — converged, no collapse, better on the trainer's own validation split. Whether it beats the incumbent is a different question, asked by T4 on different data. Conflating them is how a bad candidate reaches real traffic.

What we build vs. what we wire inwhich parts are our algorithms, and which are someone else's capability

Algorithms we buildno external service — this is the IP
CapabilityNodeWhy it exists
Dataset assembly from tracesSelection, filtering and the train/validation split over captured production traffic
Synthetic augmentationExpands thin task shapes so a run has enough coverage to learn from
Exit checkConverged, no collapse, improved on the run's own validation split
Content-hash stage manifestMakes a run reproducible and lets a failed run resume rather than restart
Capabilities we wire insomeone else's, called for a named reason
CapabilityKindWhy this workflow needs itConstraint
Frontier teacher inferenceModel inferenceHarvests teacher trajectories to learn fromRouted through the gateway so training traffic is governed like production traffic
RL training runtimeCompute jobRuns the training itself on the own-GPU postureYou schedule the capacity; the split between training and serving is yours to make
Managed fine-tuning serviceManaged jobThe same outcome on the managed postureNo capacity call exists here — the provider owns scheduling
GPU capacity schedulingInfrastructureReserves the share of the cluster a run may useOwn-GPU only. This has no counterpart on the managed path
Artifact storeData substrateHolds checkpoints and adaptersContent-addressed, so a checkpoint is identified by what it contains
Training loop · 3 of 4
T3

Rollout Loop

Akka orchestration

Learning from the real harness. The agent works in a sandbox at a pinned commit, the environment marks what it did, and the result feeds the next training step.

TouchesRL ENVSJUDGES & EVALSSFT · DPO · RLPROXYREPO

Flowclosed cycle · repeats until the trainer stops asking

TRIGGER · RolloutRequested — THE TRAINER IN T2 NEEDS FRESH EXPERIENCE FROM THE CURRENT CHECKPOINT 1 · Stagethe checkpoint is put behind atemporary endpoint of its ownthe fleet is untouched 2 · Provisiona sandbox at one exact commit,with that commit's test suitethrown away afterwards 3 · Actthe agent drives the same harnessa developer would useunder a scoped, read-only identity 4 · Mark the workrun the tests where they exist,ask a judge model where they do notone score between 0 and 1 5 · Returnwhat it did, and howwell it did, go backto the trainer the cycle closes — the trainer takes the result and asks again POST /v1/chat/completions judge model call only where no test can answer it Rollout traffic goes through the gatewayso training calls are metered, scrubbed, and recorded on exactlythe same path as production — G1 has no exceptions What the agent may touch is fixed by the setupanything undoable runs for real inside the sandbox · anything thatwould reach a real system is faked, so it cannot A rollout that hangs is stopped and its attempt thrown away, so the run loses one sample and nothing outside the sandbox changes. Sandboxes are temporary, but the record of them is not — one left running is found and shut down.

solid = invocation  ·  grey dashed = result returning  ·  red = the boundary the agent cannot cross

Trigger

RolloutRequested from the trainer in T2. Every cycle is one request, so a training run may drive thousands of them.

State & durability

Event-driven between the trainer and the orchestrator. Sandboxes are temporary; their lifecycle is durable, so one that is orphaned gets reclaimed rather than left running.

Failure & compensation

Anything reversible runs for real inside the sandbox. Anything irreversible is faked. That is set by how the sandbox is built rather than by configuration, so it holds even when the model tries something unexpected. A rollout that hangs is stopped and its attempt discarded.

Inference-tier interfaces

Called byEndpointYour GPUs · you serveFireworks · they serveWhat comes back
ActPOST /v1/chat/completionsvLLM holding the staged checkpoint, reached through the gatewayFireworks holding the staged checkpoint, same paththe agent's next move in the harness
Mark the workPOST /v1/chat/completionsvLLM serving the judge model in your VPCFireworks, or the frontier TEACHER tiera score where no test can decide it
Mark the work— not an inference callthe commit's own test suite, executed inside the sandboxpass or fail, with no model involved

Marking the work is two different things wearing one name. Where the commit has tests, they decide it and nothing is asked of a model. A judge is called only for work no test can settle.

Components to buildevery component this workflow needs, and the scaffolding each one depends on

One rollout is one episode: stage the checkpoint, provision a sandbox at a known commit, act, grade, resync. This is the only workflow that runs a model against a live workspace, which is why the containment sits in the authorization layer rather than in the prompt.

WFWorkflow — durable multi-step process, resumes after a crash
AGTAgent — one model call with tools and session memory
CONConsumer — reacts to a stream of changes
ENTEntity — durable state, one instance per key
VIEWView — a query that recomputes when its sources change
ComponentTypePurposeScaffolding needed
Workflows1durable multi-step processes — each resumes at its last completed step
RolloutWorkflowWFOne episode — stage → provision → act → grade → resync. Owns the sandbox lifecycle, so an orphan is always reclaimedSandbox runtime · repo at a pinned commit
Agents1model calls with tools, guardrails and session memory
RolloutAgentAGTDrives the task inside the sandbox against the pinned checkpointVendor harness parity · scoped tool set
Consumers1react to a stream of changes
RolloutRequestConsumerCONReacts to a rollout request from the training backend and starts an episodeBackend event contract
Entities2durable state, one instance per key
SandboxEntityENTDurable lifecycle of an ephemeral sandbox — what makes orphan reclamation possibleSandbox runtime
TrajectoryEntityENTThe captured episode: model I/O and workspace state, without which a verifiable reward cannot be recomputed
Views1queries that recompute when their sources change
RolloutsByRunViewVIEWEpisodes per training run — progress, sample count, discards

Containment is authorization, not prompting. The rollout runs as a scoped, read-and-sandbox-only projection of the real persona's identity and tool access. Reversible in-workspace tools run for real inside an ephemeral clone; irreversible or external tools are mocked. Enforced at the authorization layer — never by trusting the model. Akka's guardrails do not implement this: they inspect text at the model boundary, and this is an access-control question.

Train must equal serve. If the rollout harness differs from the harness at serving, the reward optimizes an environment that does not exist. Prefer driving the real vendor harness in the loop, and snapshot workspace state rather than model I/O alone.

What we build vs. what we wire inwhich parts are our algorithms, and which are someone else's capability

Algorithms we buildno external service — this is the IP
CapabilityNodeWhy it exists
Tool taxonomy enforcementDecides what runs for real in the sandbox and what is mocked
Verifiable reward computationTest execution inside the sandbox — the part of the reward that is not a model opinion
Trajectory captureRecords model I/O together with workspace state at a pinned commit
Sandbox lifecycle and reclamationEphemeral by design, durable in bookkeeping, so nothing leaks
Capabilities we wire insomeone else's, called for a named reason
CapabilityKindWhy this workflow needs itConstraint
Pinned-checkpoint inferenceModel inferenceThe rollout traffic itself, against the staged checkpointReached through the gateway, so it is metered and recorded like production traffic
LLM judgeModel inferenceScores only the subjective portion of the rewardThe verifiable portion is test execution and is not a model call
Sandbox runtimeInfrastructureProvides the ephemeral, isolated workspace an episode acts inMust be disposable and reclaimable; an orphan costs money silently
Source hostData substrateSupplies the repository at a pinned commitThe commit is part of the trajectory — a moving branch makes rewards irreproducible
Identity and access controlGovernanceProjects a scoped, least-privilege persona for the episodeThis is the containment boundary. Nothing else prevents a real write
Vendor agent harnessRuntimeThe loop the agent actually runs inMust match serving, or the reward optimizes the wrong environment
Training loop · 4 of 4
T4

Prove & Promote

Akka orchestration

Four steps, each risking a little more than the last, and any of them can send the candidate back. Two of the four are carried out by S2 inside the gateway; this workflow keeps the score.

TouchesJUDGES & EVALSPROXYSERVINGSCORECARDPOLICY

Flowfour steps · rising risk · one shared way back

TRIGGER · CheckpointReady — T2 FINISHED A RUN AND IT PASSED ITS OWN CHECK 1 · Compare it offlinereal past requests it has never seen,answered by both the candidate andthe model in service todaynobody is served by it yet 2 · Shadowlive requests are copied to it andscored side by side with the answerthe user actually receivedcarried out by S2 · nobody is served by it 3 · Canarya small share of real traffic isanswered by the candidate andmeasured on what users do nextcarried out by S2 · real people, real answers 4 · Promotethis workflow asks the gateway toswitch; the gateway makes the changein one move, and says it didundoing it is the same one move POST /v1/chat/completions the candidate answers the held-back set Every step fails the same way back to the model in service · keep answering · set the candidate aside · raise an alert at the canary step S2 makes the switch back on its own, in milliseconds, and tells this workflow afterwards — serving never waits on the control plane SOME PROMOTIONS WAIT FOR A PERSON A candidate that has passed everything still stops here when the agent is one where a mistake would be costly, or when it saves money without answering any better. it holds at the last step it passed, and appears in D2 as a decision with an owner and a cost of waiting Because promoting is one small change, undoing it is a change of the same size — nothing to drain, nothing to redeploy.

solid = invocation  ·  grey dashed = read  ·  red dashed = the way back  ·  green = no user is exposed at this step

Trigger

CheckpointReady, emitted by T2 when a run passes its own check. The ladder starts on that event rather than by polling for new checkpoints.

State & durability

Each step records its verdict and the evidence behind it. Promotion is a single request to the gateway, which applies it in one move — that is what makes undoing it the same size of change. The gateway is the only writer, so every switch has one order and one record.

Failure & compensation

Any step failing compensates identically — back to the model in service, keep answering, set the candidate aside, raise an alert. At the canary step S2 reverts on its own without waiting for this workflow, then reports the breach so the verdict and its evidence are still recorded here.

Inference-tier interfaces

Called byEndpointYour GPUs · you serveFireworks · they serveWhat comes back
Compare it offlinePOST /v1/chat/completionsvLLM holding the candidate, with no traffic pointed at itFireworks holding the candidate, sameanswers to score against the model in service
Compare it offlinePOST /v1/chat/completionsvLLM serving the judge model in your VPCFireworks, or the frontier TEACHER tiera quality score for each answer
Shadow · Canary— delegated to S2this workflow sets the share and reads the outcome; the gateway does the mirroring and splittinga verdict per step
Promote— not an inference calla request to the gateway, which is the only thing allowed to change what servesconfirmation of what changed

Components to buildevery component this workflow needs, and the scaffolding each one depends on

Four rungs, each risking a little more than the last, and any of them can send the candidate back. Two rungs are carried out by S2 inside the gateway; this workflow owns the saga and keeps the score.

WFWorkflow — durable multi-step process, resumes after a crash
CONConsumer — reacts to a stream of changes
ENTEntity — durable state, one instance per key
VIEWView — a query that recomputes when its sources change
ComponentTypePurposeScaffolding needed
Workflows2durable multi-step processes — each resumes at its last completed step
PromotionWorkflowWFThe four-rung saga. Sequences the rungs, records each verdict with its evidence, and emits the promote intent at the endRung acceptance thresholds · S2 split control
OfflineEvalWorkflowWFRung 1 — loads the candidate and scores it over held-out production traffic. No traffic at riskHeld-out eval set · adapter load
Consumers2react to a stream of changes
CheckpointReadyConsumerCONReacts to a checkpoint from T2 and opens a promotion sagaT2 event contract
CanaryBreachConsumerCONReceives a breach from S2, records the rung verdict, and quarantines the candidateS2 breach event
Entities3durable state, one instance per key
CandidateEntityENTThe candidate under promotion — rung history, evidence, and current hold state
PromotionEvidenceEntityENTThe standard the candidate actually proved at promotion. Retained, because S3 measures drift against it later
TrainingStateEntityENTShared with T1 and T2. Moves to promoted or retired as the saga ends
Views2queries that recompute when their sources change
CandidatesByAgentViewVIEWCandidates and their rung position per agent
DecisionQueueViewVIEWCandidates holding at a rung because a person must decide — the queue D2 renders

Promotion is one intent, which is what makes rollback cheap. This workflow does not write the routing table; it asks. The gateway applies the change atomically, so undoing it is the same size of change — no drain, no redeploy. A rollback that costs more than a promotion does not get used in an emergency.

Some holds are deliberate. A candidate stops at its last passed rung when the agent is high blast radius, or when cost improves while quality is only level. A person decides that; no threshold does. The candidate keeps its evidence while it waits.

What we build vs. what we wire inwhich parts are our algorithms, and which are someone else's capability

Algorithms we buildno external service — this is the IP
CapabilityNodeWhy it exists
Rung sequencing and saga stateAdvances or retires a candidate, and compensates when a rung fails
Acceptance-bar comparisonQuality against the incumbent, and token reduction against the target
Paired-score aggregationTurns per-request comparisons into a rung verdict
QuarantineSets a failed candidate aside with its evidence rather than deleting it
Capabilities we wire insomeone else's, called for a named reason
CapabilityKindWhy this workflow needs itConstraint
Adapter loadModel servingStages the candidate so the offline batch can run against itServing-tier operation; the posture decides how it is expressed
Batch inferenceModel inferenceScores the candidate over the held-out setOffline — no production traffic is exposed at this rung
LLM judgeModel inferenceSupplies the quality half of the acceptance barMust be the same judges S3 later measures drift with, or the standard moves
Routing intentControl planeAsks the gateway to switch what servesThe gateway is the sole writer; this workflow never writes routing itself
Serving loop · 1 of 3
S1

Route & Serve

Gateway

The hot path. Every request from a coding tool arrives here and leaves with an answer; a lookup table decides which model produced it.

TouchesPLUGINPROXYSERVINGSPECIALISTSTEACHER

Flowone pass, in order · one way out to the frontier

TRIGGER · A REQUEST ARRIVES FROM A CODING TOOL — CURSOR, CLAUDE CODE, CODEX, COPILOT Request inthe tool needs no changebeyond one addressit points here instead Recognise the taskwhich of this agent's knownjobs is this one Look up the modelthe table says which modeland which version to usewritten only by the gateway Load the specialista ~50MB add-on is swappedonto the model already loadedno restart, no redeploy Answerserved from your owntuned model Back to the toolevery request getsan answer Out to the frontier insteadon a timeout, an error, or an answer thepolicy says is not confident enoughrecorded, and audited like any other call WHAT COUNTS AS "NOT CONFIDENT ENOUGH" IS SET BY A PERSON Per agent, someone chooses the test: an exact check where the task allows one, a threshold on the model's own certainty, or a second opinion from a judge. until that is set, the agent has no automatic way out — see G3, and the queue in D2 Nothing about a single request is kept. The one lasting thing is the lookup table — which model and version answers for each agent. The gateway is the only thing that writes it. T4 asks on promotion, S3 asks when a model decays, a person asks when they approve — the gateway makes the change. How often it falls out to the frontier is itself a signal: a rising rate re-opens S3.

solid = invocation  ·  grey dashed = read  ·  red dashed = the way out to the frontier

Trigger

A request arrives from a coding tool, in the same format those tools already speak. Pointing them here is an address change and nothing else.

State & durability

Nothing per request is kept — this path is measured in milliseconds. The lasting object is the lookup table: which model and version answers for each agent. The gateway is its only writer; T4, S3, and human approvals ask, and the gateway makes the change.

Failure & compensation

A timeout, an error, or an answer below the confidence policy goes out to the frontier instead. Every request comes back with an answer. How often that happens is itself measured — a rising rate re-opens S3.

Inference-tier interfaces

Called byEndpointYour GPUs · you serveFireworks · they serveWhat comes back
AnswerPOST /v1/chat/completionsvLLM serving the base model with the agent's add-on loadedFireworks serving the same pairingthe answer, plus token counts and timing
Out to the frontierPOST /v1/chat/completionsthe frontier provider's own API, reached through the gateway so it is recorded like anything elsean answer from the strongest available model
Look up the model— not an inference callthe gateway's own lookup table, held in memory on the request pathwhich model and version to use

Components to buildevery component this workflow needs, and the scaffolding each one depends on

The request path. Nothing here is durable per request — this path is measured in milliseconds. The lasting object is the routing table, and the gateway is the only thing allowed to write it.

EPEndpoint — HTTP surface — the inline request path
CONConsumer — reacts to a stream of changes
ENTEntity — durable state, one instance per key
VIEWView — a query that recomputes when its sources change
ComponentTypePurposeScaffolding needed
Endpoints1the inline request path — no durable per-request state
GatewayEndpointEPOpenAI-compatible ingress. Classifies, selects the tier, binds the adapter, serves, and returns usage. The only inline hopServing tier · routing table held in memory
Consumers1react to a stream of changes
RoutingIntentConsumerCONApplies intents from T4, S3 and human approvals, then emits what it actually didIntent event contract
Entities2durable state, one instance per key
RoutingTableEntityENTAgent → tier → adapter version. One writer, one ordering, one audit path
EscalationPolicyEntityENTThe per-agent low-confidence policy a person defines. Until it is set, that agent has no automatic escalationPolicy source of record
Views1queries that recompute when their sources change
RoutingTableViewVIEWThe read side the request path consults on every call

One writer, by design. T4 asks on promotion, S3 asks when a model decays, a person asks on approval — and the gateway makes every change. That is what gives the routing table a single ordering and a single audit trail. A second writer would make rollback ambiguous exactly when it matters most.

Escalation is configuration. The low-confidence signal is whatever a person configured for that agent: an exact check where the task allows one, a confidence threshold, or a judge call. There is no default, and an agent without a policy simply never escalates automatically.

What we build vs. what we wire inwhich parts are our algorithms, and which are someone else's capability

Algorithms we buildno external service — this is the IP
CapabilityNodeWhy it exists
Request classificationWorks out which agent and task shape a request belongs to
Tier selectionChooses the small model, the incumbent, or the frontier tier
Escalation policy evaluationApplies the per-agent rule that sends a hard request upward
Routing-intent applicationApplies asks in one order and emits what changed
Capabilities we wire insomeone else's, called for a named reason
CapabilityKindWhy this workflow needs itConstraint
Small-model servingModel inferenceAnswers the request when the routing table says the small model serves itIn-VPC or managed; the same OpenAI-compatible call either way
Adapter hot-swapModel servingBinds the right small adapter onto a resident base modelThe adapter is small; the base stays loaded. This is what makes per-agent models affordable
Frontier providerModel inferenceServes escalations and anything with no promoted modelReached through the gateway, so it is recorded like everything else
Usage accountingTelemetryReturns tokens and latency on every responseThis is the raw material for every cost number downstream
Serving loop · 2 of 3
S2

Shadow & Canary

Gateway

How a new model earns real traffic. The gateway is the one place where a candidate and the model in service can answer exactly the same request, so the comparison is like for like.

TouchesPROXYSERVINGSCORECARD

Flowtwo phases · both answer the same request at once

TRIGGER · T4 MOVES A CANDIDATE TO THE SHADOW OR CANARY STEP AND SETS THE SHARE Live requestfrom a real user PHASE 1 · SHADOW — NOBODY IS SERVED BY THE CANDIDATE In service todayanswers the user The candidateanswers a copy · thrown away Scored side by sidesame request · same moment PHASE 2 · CANARY — A SMALL SHARE OF REAL USERS In service todaystill answers most people The candidateanswers the rest, for real Measured on outcomeswhat users did next To the recordcost and quality delta If the candidate starts answering worse, this happens here rather than in the control plane the gateway puts every request back on the model in service, in milliseconds, without asking anyone then it tells T4, which records which step failed, sets the candidate aside, and raises the alert serving never waits on the control plane to protect itself · T4 keeps the score, S2 pulls the handle

solid = real traffic  ·  grey dashed = a copy, or a result being read  ·  red dashed = the way back  ·  green = nobody is exposed

Trigger

T4 moves a candidate onto the shadow or canary step and sets the share of traffic it may take. This workflow carries that out; it does not decide it.

State & durability

The share is durable and versioned. Paired results carry a matching id, so a comparison made today can be reproduced from the record months later — the same request, the same moment, both answers.

Failure & compensation

Shadow answers are scored and thrown away, so no user ever receives one. If the candidate starts answering worse, the gateway puts everything back on the model in service immediately, then reports the breach to T4, which records the failed step and sets the candidate aside. Serving continues throughout.

Inference-tier interfaces

Called byEndpointYour GPUs · you serveFireworks · they serveWhat comes back
ShadowPOST /v1/chat/completionsthe candidate answers a copy of the request; the answer is scored and discardedan answer nobody receives
CanaryPOST /v1/chat/completionsthe candidate answers its share of real requests, on the same endpoint as the model in servicean answer a real user receives
Set or revert the share— not an inference callthe gateway's own traffic policy, changed in placeconfirmation of the new share

Components to buildevery component this workflow needs, and the scaffolding each one depends on

Where a candidate meets real traffic. The revert is inline and does not wait on the control plane — safety belongs to whatever is closest to the request.

EPEndpoint — HTTP surface — the inline request path
CONConsumer — reacts to a stream of changes
ENTEntity — durable state, one instance per key
VIEWView — a query that recomputes when its sources change
ComponentTypePurposeScaffolding needed
Endpoints1the inline request path — no durable per-request state
GatewayEndpointEPShared with S1. Mirrors traffic for shadow, splits it for canary, and reverts in place on a breachSplit policy read on the request path
Consumers1react to a stream of changes
PairedScoringConsumerCONScores paired captures and discards the shadow output, so no user ever receives oneJudge access · correlation ids
Entities2durable state, one instance per key
SplitPolicyEntityENTDurable and versioned. What share of traffic a candidate may see — the exposure cap
PairedResultEntityENTBoth arms of a comparison under one correlation id, so a result is reproducible months later
Views1queries that recompute when their sources change
CanaryHealthViewVIEWAcceptance metrics over the canary window — what the breach check reads

S2 acts, T4 judges. On a canary dip S2 reverts the split to the incumbent immediately, without waiting for the control plane, and only then reports the breach. T4 owns the saga and records the verdict; S2 owns the safety action. Splitting it this way means the fast path is never blocked on a slower one.

Shadow costs money and shows nothing to users. Every shadow request is a second inference that is scored and thrown away. That is the price of a risk-free comparison, and it should be a deliberate, bounded spend rather than a permanent background cost.

What we build vs. what we wire inwhich parts are our algorithms, and which are someone else's capability

Algorithms we buildno external service — this is the IP
CapabilityNodeWhy it exists
Traffic mirroringDuplicates live requests to the candidate without affecting the answer served
Traffic splittingSends a bounded share of real traffic to the candidate
Paired capture and correlationTies both arms of a comparison together for later reproduction
Acceptance evaluationDecides in-window whether the candidate is still within its bar
Immediate revertReturns all traffic to the incumbent without a control-plane round trip
Capabilities we wire insomeone else's, called for a named reason
CapabilityKindWhy this workflow needs itConstraint
Model serving, both armsModel inferenceRuns incumbent and candidate over the same inputShadow doubles inference cost for the duration
LLM judgeModel inferenceScores the paired resultsSame judges as T4's bar, or the comparison is not like-for-like
Serving loop · 3 of 3
S3

Drift Watch

Akka orchestration

A promoted model keeps being marked for as long as it serves, against the same scores it had to beat to go live in the first place. Where it slips sharply the system acts on its own; where it slips slowly, a person decides.

TouchesSCORECARDJUDGES & EVALSTRACE STOREPROXY

Flowon a timer · three ways it can end

TRIGGER · A TIMER, ON A FIXED CADENCE, FOR EACH PROMOTED MODEL — OVER A ROLLING WINDOW OF ITS LIVE TRAFFIC Mark the answersreal answers from real users,marked by the same judgesT4 used before promotion Watch the trendquality over time, cost pertask, and how often S1 hasto fall out to the frontier Compareagainst the scores this modelhad to beat to be approved,not a target that moved since POST /v1/chat/completions the judge model marks live answers Still answering well enoughkeep serving, keep marking · this is where a healthy model staysthe timer fires again next cycle and nothing changes Dropped sharplythe system asks the gateway to go back, without waiting for anyoneto the previous version, or out to the frontier · training re-opens Slipping slowly — a person decidestrain it again, or send this agent back to the frontier for goodit keeps serving while the decision waits · queued in D2 with a cost a healthy model simply comes round again on the next tick What this model proved at promotion is kept, so slipping is always measured against what it actually did — never against a target that moved underneath it. Going back is always a request to the gateway. It is the only thing that changes what serves, whether the system asks or a person does.

yellow = waiting on a person  ·  grey dashed = read  ·  red dashed = the system acts on its own  ·  green = nothing to do

Trigger

A timer, on a fixed cadence, for every promoted model — reading a rolling window of that model's live traffic. Nothing about it is event-driven, so a quiet agent is still checked.

State & durability

A rolling window per agent, durable across restarts. What the model proved at promotion is kept alongside it, so slipping is measured against the scores it actually had to beat.

Failure & compensation

A sharp drop is treated exactly like a failed canary: ask the gateway to go back to the previous version or to the frontier, and re-open training. A slow slide is a decision for a person — train it again, or return the agent to the frontier for good — and it keeps serving while that decision waits.

Inference-tier interfaces

Called byEndpointYour GPUs · you serveFireworks · they serveWhat comes back
Mark the answersPOST /v1/chat/completionsvLLM serving the judge model in your VPCFireworks, or the frontier TEACHER tiera quality score for live answers, on the same scale T4 used
Watch the trend— not an inference callread from the record, not from the models — quality, cost per task, and how often S1 fell out to the frontierthe trend over the window
Go back— not an inference calla request to the gateway, which is the only thing that changes what servesconfirmation of what changed

Components to buildevery component this workflow needs, and the scaffolding each one depends on

A promoted model is not finished. This watches it against the standard it actually proved at promotion, and separates a cliff from a slide — because they need different responses.

WFWorkflow — durable multi-step process, resumes after a crash
TMRTimed Action — fires on a schedule
ENTEntity — durable state, one instance per key
VIEWView — a query that recomputes when its sources change
ComponentTypePurposeScaffolding needed
Workflows1durable multi-step processes — each resumes at its last completed step
DriftCheckWorkflowWFGrades a window of production responses, compares against the retained promotion bar, and renders an outcomeJudge access · retained evidence
Timed Actions1schedule only — each starts a run and does no work itself
DriftWatchTimedActionTMRFixed cadence per promoted agent. Starts one drift check and does no grading itselfSelf-rescheduling timer
Entities2durable state, one instance per key
RollingWindowEntityENTThe window of recent graded production traffic per agent, durable across restarts
PromotionEvidenceEntityENTShared with T4. The standard the model proved, retained so drift is measured against the real thing
Views1queries that recompute when their sources change
DriftByAgentViewVIEWCurrent position against the standard it proved, per promoted agent

A cliff and a slide are different events. Sharp degradation is handled without asking anyone: emit a revert intent and re-open training. Gradual drift is a person's decision — retrain, or return the agent to the frontier for good — and the agent keeps serving while that decision waits. Automating the second one would either revert too eagerly or too late.

Measured against the retained standard. The acceptance evidence from promotion is kept precisely so this comparison is against what the model proved. Re-deriving a baseline later would let the standard drift along with the model.

What we build vs. what we wire inwhich parts are our algorithms, and which are someone else's capability

Algorithms we buildno external service — this is the IP
CapabilityNodeWhy it exists
Rolling-window maintenanceKeeps a bounded, durable window of recent graded traffic per agent
Drift classificationSeparates sharp degradation from gradual slide
Comparison against the retained barMeasures against promotion evidence rather than a recomputed baseline
Capabilities we wire insomeone else's, called for a named reason
CapabilityKindWhy this workflow needs itConstraint
LLM judgeModel inferenceGrades production responsesThe same judges T4 used at promotion, or the comparison is meaningless
Routing intentControl planeAsks the gateway to go back on sharp degradationThe gateway makes the change; this workflow only asks
Metrics recordData substrateSupplies cost and latencyRead from the record, not from the inference tier
Governance · 1 of 3
G1

Govern & Capture

Gateway

Checking and recording happen in one pass. Every request is checked before it leaves and written down after, at the same point — so neither can be skipped, and there is nowhere else to go.

TouchesPOLICYPROXYLOGSTRACE STORESCRUB & CURATE

Flowone pass · one way to be stopped

TRIGGER · EVERY CALL IN BOTH LOOPS — SERVING AND TRAINING ALIKE — PLUS PRODUCTION LOGS ARRIVING IN BATCHES A user's requestfrom S1 A training callfrom T2 and T3 Batched logsfrom the forwarder ONE POINT EVERYTHING PASSES THROUGH — INCLUDING TRAINING TRAFFIC 1 · Allowed?has this data beencleared for this useis this person and thistenant allowed to askchecked against policy 2 · Strip itpersonal details andsecrets come out herebefore anything at allis written down 3 · Count ittokens spent, againstthis agent's ceilingtraining calls countthe same as serving ones 4 · Write it downone record per call,carrying what it wascleared foradded to, never edited The recordadded to, never editedwhat the dashboards read Not allowedthe call is stopped, or forced out to the auditedfrontier path instead — and written down either way Stripping failedthe record is dropped rather than written, so nothingunscrubbed ever reaches the store A refused call is written down alongside the ones that went through — what was asked for, and why it was not allowed.

solid = invocation  ·  grey dashed = written to the record  ·  red dashed = stopped here

Trigger

Every call in both loops — serving traffic from S1 and training traffic from T2 and T3 — plus production logs arriving in batches through the forwarder. Rollouts route through here too, so there is no path around it.

State & durability

The record is added to and never edited. What each call was cleared for travels with it, so T1 can rule an agent out later without having to work out where the data came from a second time.

Failure & compensation

A call that is not allowed is stopped, or forced out to the audited frontier path. It is written down either way — a refused call is part of the record just as a served one is. If stripping fails, the record is dropped rather than written, so nothing unscrubbed reaches the store.

Inference-tier interfaces

Called byEndpointYour GPUs · you serveFireworks · they serveWhat comes back
All four steps— none of its ownthis workflow calls no model. It wraps the calls the other workflows make, before and after, and everything it does is deterministicnothing from a model

Checking, stripping, counting, and recording are all ordinary code. Listing them as inference-tier interfaces would suggest a model is involved in deciding whether a call is allowed, and none is.

Components to buildevery component this workflow needs, and the scaffolding each one depends on

The band around everything else. Every call in both loops transits this path — including rollout traffic — so there is no route that escapes metering, scrubbing or the record.

EPEndpoint — HTTP surface — the inline request path
CONConsumer — reacts to a stream of changes
ENTEntity — durable state, one instance per key
VIEWView — a query that recomputes when its sources change
ComponentTypePurposeScaffolding needed
Endpoints2the inline request path — no durable per-request state
GatewayEndpointEPShared with S1 and S2. Admits, scrubs, meters and emits, inline, before the model callPolicy source · sanitizer · spend ledger
LogForwarderEndpointEPBatch ingest for agents that cannot be pointed at the gateway. Capture and audit only, with no steeringTrace ingest format
Consumers1react to a stream of changes
TraceCaptureConsumerCONWrites traces to the store with their governance tags attached
Entities3durable state, one instance per key
PolicyEntityENTConsent, RBAC and tenancy for an agent — what admit checks againstPolicy source of record
SpendLedgerEntityENTToken spend against the agent's ceiling
TraceEntityENTOne append-only trace, carrying the governance tags that let T1 veto later without re-deriving provenance
Views2queries that recompute when their sources change
TracesByAgentViewVIEWThe substrate every T1 scorer reads
SpendByAgentViewVIEWSpend and ceiling per agent — what D1 renders

Scrub before persist, and drop on failure. PII and secrets are removed at the boundary, before anything is written. If scrubbing fails the trace is dropped rather than stored unscrubbed — the record is allowed to have a hole in it, but never a leak. This is the natural home for Akka's sanitizer, which masks matched patterns and can be called from any component.

Governance tags travel with the trace. Regulated status and training consent are written onto the record here, at capture. That is what lets T1 veto an agent months later without re-deriving where the data came from — and it is why a wrong tag at this step is a compliance failure rather than a scoring error.

Two onboarding paths, two levels of control. Pointing an agent at the gateway gives inline steering, enforcement and capture. The log forwarder gives capture and audit only — there is no steering until that agent is proxied, and the difference should be explicit to whoever is buying.

What we build vs. what we wire inwhich parts are our algorithms, and which are someone else's capability

Algorithms we buildno external service — this is the IP
CapabilityNodeWhy it exists
Admission policy evaluationChecks consent, RBAC and tenancy before a call proceeds
Metering against a ceilingCounts spend per agent and enforces the limit
Trace emission and taggingWrites the append-only record with governance tags attached
Capabilities we wire insomeone else's, called for a named reason
CapabilityKindWhy this workflow needs itConstraint
SanitizerPlatform capabilityMasks PII and secrets at the boundary before anything is persistedAkka's own, injectable into any component — not a third party
Identity providerGovernanceSupplies the identities RBAC and tenancy are checked againstCustomer-owned
Trace ingest formatData substrateHow forwarded logs arrive for agents that cannot be proxiedOpen tracing formats; the customer's exporters decide the shape
Governance · 2 of 3
G2

Standing policy requirements

Akka Verify

G1 runs per request as a workflow. These are standing requirements evaluated against the record on a schedule. Akka Verify reads the cost and quality record and enforces them:

  • 1Model eligibility. A task type is served only by models at quality above a floor and cost below a ceiling.
  • 2Lowest-cost routing. Route each task to the cheapest qualifying model, updated as candidates prove out.
  • 3Budget guardrail. Throttle or fail over when projected spend exceeds a cap.
  • 4Per-task ceiling. Cap tokens or cost per task; downgrade or halt on breach.
  • 5Cost as a promotion criterion. Hold a candidate that passes quality but costs more than the incumbent.
  • 6Re-evaluate on price drift. Reopen routing when frontier prices or a model's blended cost move past a threshold.
  • 7Value realization. Report savings against the frontier baseline from the record.
Governance · 3 of 3
G3

Every policy, and who decides it

Akka Verify

Three kinds of control, and the difference is who acts. A guardrail fires on its own. An evaluation produces a verdict that something else acts on. A human decision holds the system until a named owner chooses — and every one of those is a queue somebody has to work.

17GUARDRAILfires automatically · blocks, throttles, or reverts with no one in the loop
7EVALUATIONmeasures and renders a verdict · acts on nothing by itself
11HUMAN DECISIONthe system holds and waits · surfaced in D2 as a queue with an owner
PolicyWhereKindWhat happens when it fires
Training loop
Regulated task may not leave the audited modelT1GUARDRAILAgent is removed before scoring; reason recorded
Training-consent scope and data residencyT1GUARDRAILAgent is removed before scoring; logged data stays audit-only
Customer ring-fence — what may never be touchedT1GUARDRAILAgent is removed before scoring
Minimum gradeability and minimum distinct examplesT1EVALUATIONRenders KEEP_FRONTIER with the failing dimension named
Override a KEEP_FRONTIER verdict for a wanted agentT1HUMANAgent stays on frontier until someone accepts the risk in writing
Contractual pin — "this customer's contract specifies a named model"T1HUMANAgent is excluded from optimization until the pin is lifted
Order of the training queue under a grader budgetT1HUMANRanked plan is proposed; nothing starts until the order is confirmed
Allowed base-model menuT2GUARDRAILBases outside the menu are never proposed
Choice of base on the size frontier — cheaper vs saferT2HUMANA recommendation with a risk curve; the run waits for the pick
Training spend cap per runT2GUARDRAILRun halts at the cap and reports where it stopped
Teacher distillation and synthetic data permittedT2GUARDRAILExcluded data classes never enter the dataset build
Reversible actions run for real, irreversible are mockedT3GUARDRAILEnforced by the sandbox topology, not by configuration
Which repositories and environments rollouts may useT3GUARDRAILSandbox provisioning refuses anything off the list
Quality against baseline and token reduction against targetT4EVALUATIONCandidate advances a rung, or is retired with its evidence
Canary traffic shareT4GUARDRAILCaps exposure at the configured percentage
Promotion of a high-blast-radius agentT4HUMANCandidate holds at the last passed rung until approved
Promotion when cost improves and quality is only levelT4HUMANHeld — the saving is real but nobody gains quality
Serving loop
Model eligibility — which models may serve which taskS1GUARDRAILIneligible models are never routed to
Route to the cheapest qualifying modelS1GUARDRAILRouting table updates as candidates prove out
Per-task token and cost ceilingS1GUARDRAILDowngrades the tier or halts the task on breach
Frontier fallback permitted for this agentS1GUARDRAILWhere it is off, the request fails rather than escalating
What counts as low confidence for escalationS1HUMANReviewed and set per agent — an exact check where the task allows one, a logprob threshold, or a judge call. Until it is defined the agent has no automatic escalation
Projected spend exceeds the period capS1HUMANThrottle and degrade, or accept the overspend — someone must choose
A candidate on live traffic starts answering worseS2GUARDRAILSplit reverts to the incumbent immediately; T4 records the verdict
Paired shadow scoring against the incumbentS2EVALUATIONProduces the comparison T4 reads at the shadow rung
Marking live answers against the scores the model was approved onS3EVALUATIONProduces the drift signal; acts on nothing itself
Sharp degradation of a promoted modelS3GUARDRAILReverts to the prior adapter or to frontier without waiting
Gradual drift — retrain, or return to frontier for goodS3HUMANThe agent keeps serving while someone decides whether it is still worth optimizing
Re-admission after repeated driftS3HUMANA third failure stops automatic re-entry into T1
Every request, both loops
Consent, RBAC, and tenancy on each callG1GUARDRAILBlocks the route or forces the audited frontier path
PII and secrets removed before anything is storedG1GUARDRAILOn failure the trace is dropped rather than written
Hold a candidate that passes quality but costs moreG2EVALUATIONBlocks promotion on economics alone
Re-open routing when model prices moveG2EVALUATIONRecomputes the cheapest qualifying model per task
Savings reported against the frontier baselineG2EVALUATIONFeeds the record behind every figure on the dashboards
Standing exception to any guardrail aboveG2HUMANGranted with an owner and an expiry, or the guardrail stands
Scorecards & dashboards
D1

Continuous Cost Governance

Read side

Budget owner

Am I within budget — and is the saving real?

Spend this period
$48.2Kof $60K cap
projected $57.1K at period end · within cap
Realized saving
$181K
vs pinned frontier
counterfactual · 79% below
Cost per task
$0.0031
▼ 68% vs 90d · per task
Where the spend sits
claims-extract$19.1K
code-review$11.4K
deploy-agent$7.2K
all others$10.5K
Rollback exposure
71%of traffic on SLMs
if every adapter reverted to frontier tomorrow,
run rate goes $12K/mo → $47K/mo
Waiting on you
2 open
Spending is on track to reach $63.1K against a $60K cap
open 6 days · at the cap the throttle engages and ~12% of requests degrade
Approve the overspendLet it throttle
code-review went over its cost-per-task ceiling 14 times this week
open 2 days · $0.0041 against a $0.0035 ceiling
Raise its ceilingMove it to a cheaper tier

AI owner

What is worth training next — and is what I promoted still holding?

Candidate pipeline · 90d
observed47
admitted12
in training3
in ladder2
promoted9
Why 35 were not admitted
ungradeable19
volume11
consent5
19 agents become eligible once a deterministic
grader exists for their task shape
Promoted models — how quality has moved
Agentfrontier → proved → now$/taskState
claims-extract v50.93 → 0.96 → 0.96$0.0028holding
code-review v30.91 → 0.94 → 0.89$0.0041drifting
deploy-agent v20.88 → 0.92 → 0.92$0.0035holding
code-review still beats the model it replaced, but answers
worse than when it was approved — so S3 re-opens T1
Waiting on you
4 open
claims-extract v6 passed all four steps, and this agent is one where a mistake would be costly
open 9 days · holding $1,240 / mo · sitting at 5% of traffic
Promote to the fleetHold at canary
code-review is answering worse than it did when you approved it, though still better than the model it replaced
open 3 days · scored 0.94 when approved, 0.89 now · holding $890 / mo
Train it againReturn it to frontier
Which base model sre-triage should be built on
open 1 day · the run cannot start until this is picked
8B · saves more32B · safer
Order of the next training queue — 6 agents admitted, budget covers 3
open 4 hours · nothing new starts this cycle until it is set
Confirm the orderReorder

System behavior

Is the loop running — and is it failing safe when it fails?

Loop throughput · 30d
Training runs started → completed12 → 9
Promotions to fleet6
Rollbacks1
Candidates retired at a check4
Evaluation ladder — pass rate per rung
offline eval14/19
shadow9/14
canary7/9
each rung that rejects a candidate stops a
regression before it reaches the fleet
Serving health
Latency p50 / p95340ms / 1.2s
Frontier fallback rate2.1% ▲
Error rate0.02%
Policy vetoes · scrub failures14 · 0
GPU allocation
train 38%serve 54%idle 8%
the same GPUs do both — T2 reserves its share within
the serving tier's latency budget
Waiting on you
2 open
sre-triage has no rule for when an answer is too weak to send, so it never escalates
open 7 days · it fails open rather than falling back to the frontier
Set the ruleLeave it off
Whether the claims-v2 data class may be trained on, not only logged
open 2 days · 19 agents cannot be scored until it is answered · $3,100 / mo
Clear it for trainingKeep it audit-only
Scorecards & dashboards
D2

Decisions waiting on a person

Read side

The eleven human decisions in G3 are the only places the loop stops on its own. Each one holds something — a candidate, a saving, an agent's routing — and holding has a price. This view carries that price, so the queue is worked in the order the money says.

8decisions open now
$5.2K / mosaving held up while the queue waits
3past their review window
2.5 daysmedian age of an open decision
Waiting onOwnerOpenIf nobody actsCost of waiting
Promote claims-extract v6 to the fleet
passed all four steps · a mistake on this agent would be costly, so it needs sign-off
AI owner9dCandidate holds at canary, serving 5% of traffic$1,240 / mo
sre-triage has no rule for when an answer is too weak to send
without one the agent never falls back to the frontier
Platform7dWeak answers are served instead of escalatingfails open
Accept projected spend of $63.1K against a $60K cap
driven by frontier fallback on code-review, which is drifting
Budget owner6dThrottle engages at the cap; roughly 12% of requests degrade3.1K requests
code-review is answering worse than when it was approved — train it again, or return it
still better than the model it replaced, but worse than it was on the day it went live
AI owner3dKeeps serving at a widening quality gap$890 / mo
Whether the claims-v2 data class may be trained on
logged for governance today; not yet cleared as training data
Compliance2d19 agents stay ineligible and are never scored$3,100 / mo
code-review went over its cost-per-task ceiling 14 times this week
$0.0041 against a $0.0035 ceiling
Budget owner2dIt keeps serving, at the higher cost per task$0.0006 / task
Which base model sre-triage is built on — cheaper, or safer
8B saves more and carries more quality risk; 32B is the safe point
AI owner1dThe training run does not startstart delayed
Order of the next training queue
6 admitted agents, grader budget covers 3 this cycle
AI owner4hNothing new enters training this cycle1 cycle
Routed to the budget owner
2open · oldest 6 days
spend against the cap, and per-task ceilings —
decisions denominated in money
Routed to the AI owner
4open · oldest 9 days
promotion, retrain-or-retire, base model, queue order —
decisions denominated in quality and risk
Routed to platform & compliance
2open · oldest 7 days
escalation rules, consent scope, standing exceptions —
decisions that settle whether an agent may run at all
Getting started · 01

A 90-day sequence

Days 1–30 · Install

Connect

Deploy in the chosen environment, pass InfoSec, connect the datalink and trace sources.

Days 31–60 · Observe

Target

Targeting scores live traffic and returns a ranked plan with quality baselines.

Days 61–90 · Prove

First model

The first adapted model is trained, evaluated, and shadow-tested, with a before/after benchmark on your traffic.

The sequence is the same for both deployment options; the training and inference engine differs.

Getting started · 02

What you supply, what Akka runs

Customer supplies

Inputs

  • GPUs, or a Fireworks account.
  • The private datalink to the environment.
  • Persona and tool scopes for the loop.
  • Evaluation definitions.
  • Workspace access for training and testing.
Akka runs

The rest

  • Capture, targeting, and the optimization workflow.
  • Training orchestration and serving.
  • Evaluations, gating, and rollback.
  • The interaction record and Improvement Policy.