How one firm's look at its AI bill turned into a system for continuous AI governance.
Where we ran the study
A top-tier Japanese financial group.
$2.5T
assets under management
40+
AI systems in production
4
regions, worldwide
Claims, KYC, research, support, code — dozens of solutions.
Their AI bill
$60M
a year · model fees across every system
Growing roughly 40% a year.
Open-weight models on MMLU
By mid-2026, the gap on standard benchmarks had all but closed.
Time from frontier release to comparable open-weight release
2 – 4
months
Kimi K3 · GLM-5 · DeepSeek V4 · Qwen3-6 · MiniMax M3 · Doubao — one after another, meeting or beating the closed benchmarks a few months on.
How do you build a strategy that exploits this?
Every call sent to a frontier model carried
The prompt.
The context around it — records, documents, past work.
And how the work is done.
Millions of calls a day, across every system.
Designing for cost
Capture every call
across every AI system
→
Tag it
agent · task · tokens
→
Cost per agent, per task
tokens × price, summed
$60M / year, attributed by task
Extraction · KYC · claims
$0
Classification · moderation
$0
Code
$0
Summarization
$0
Routing · retrieval
$0
Open-ended reasoning
$0
Of that $60M
70%
$42M · tasks with a checkable answer
One more measurement
$1,300
→
$320K
Same usage. Subsidized seat vs metered list rate — about 200× higher.
On one narrow task
0.80
tuned 4B model
0.77
frontier A
0.76
frontier B
Same task · cost per verified task
$0.0152
→
$0.0031
Trained again as new traffic comes in
$0saved per year · 81% · no measurable quality degradation
How the system works
Your agents & tools
Cursor · Claude Code · Copilot · custom stacks
▼
AI Gateway
every call routes through one point route · serve · fall back to frontier
▼
Train
capture · evaluate · train on your traffic
Serve
base model + specialized adapters
Governance
policy · audit · cost — on every step
The technology stack
Base models · open-weight
Kimi K3Moonshot AI · 2.8T MoE, 41B active
GLM-5Zhipu AI · 744B MoE, MIT license
DeepSeek V4 ProDeepSeek · reasoning, top SWE-Bench
Llama 4 · Qwen3-6Meta · Alibaba
Where they run
In-region VPCGPUs owned or rented in the firm's cloud
Fireworksmanaged serving, when in-VPC isn't required
Frontier fallbackGPT-5.5 · Claude Fable 5 · via API
What Akka runs
The gatewayroute · serve · fall back
The training loopcapture · evaluate · train · promote
Governance & the consolepolicy · audit · cost, per call
One request, through the system
All of it, at once
Budget owner
Am I within budget — and is the saving real?
80%
of budget used · projected within cap
$181K
saved vs frontier, this quarter
$0.0031
cost per verified task · down 68%
Where the spend sits
claims-extract
$19.1K
code-review
$11.4K
deploy-agent
$7.2K
all others
$10.5K
Waiting on you · 2
Spending on track to $63.1K against a $60K cap
Approve overspendThrottle
code-review over ceiling 14× this week
Raise ceilingDowngrade tier
AI owner
What's worth training next — and what's holding?
9
models promoted to the fleet
3
in training now
1
drifting · flagged for review
Promoted models — how quality has moved
agent
frontier → proved → now
$ / task
claims-extract v5
0.93 → 0.96 → 0.96
$0.0028
code-review v3
0.91 → 0.94 → 0.89
$0.0041
deploy-agent v2
0.88 → 0.92 → 0.92
$0.0035
Waiting on you · 3
Promote claims-extract v6 to the fleet
ApproveHold at canary
code-review is answering worse than at approval
RetrainReturn to frontier
Base model for sre-triage — cheaper or safer
8B32B
Operations
Is the loop running — and failing safe when it fails?
12→9
training runs · started to completed
1.2s
p95 latency · within budget
2.1%
frontier fallback rate
Evaluation ladder — pass rate per rung
offline eval
14 / 19
shadow
9 / 14
canary
7 / 9
errors0.02%
frontier fallback2.1% ▲
policy vetoes14
Waiting on you · 2
Train on the claims-v2 data class?
ApproveAudit-only
Set escalation rule for sre-triage
Set ruleLeave off
Three kinds of control
17
Guardrails
fire automatically
7
Evaluations
measure and report
11
Decisions
require a person
What they now call it
Continuous cost governance.
Ongoing measurement of what every AI call costs, policies that route work to the cheapest capable model, and human review of the decisions that carry risk.
Their models, run in their own environment, continuously trained.