Microsoft Token Wars: A 4-Hour Microhack to Cut AI Cost Per Answer

Microsoft has put a ready-made workshop for cutting LLM cost per answer on GitHub. The microsoft/Token-Wars repository is a four-hour Azure "microhack" in which teams take a deliberately wasteful AI support copilot and make it cheaper without letting quality slip. It appeared on GitHub's trending list in week 42 of 2026, and its README is a compact checklist of what actually moves token bills.

You do not need to join an event to learn from it. This post summarises the scenario, the three challenges, the model tiers, the scoring rule, and how to run it, and we flag what the repo does not say.

Quick reference

Why the baseline is so wasteful

The README lists exactly why the starting copilot is expensive, and it is a good checklist for your own app:

• It always calls the premium model, even for easy questions.
• It stuffs the whole knowledge base and the whole orders database into every prompt.
• It puts dynamic data at the top of the system prompt, so prompt caching never kicks in.
• It has no output limits and no caching.

Each item maps to a standard fix: route by difficulty, retrieve instead of stuffing, keep stable text first so providers can cache the prefix, cap output tokens, and cache answers. If you have read our notes on context-window pricing, the last three are the same levers described from the price side.

The three challenges

Routing and escalation

In the default design, a classifier on the Economy tier labels each question SIMPLE, STANDARD or COMPLEX. SIMPLE goes to Economy, STANDARD to Balanced and COMPLEX to Premium, and escalation follows economy, then balanced, then premium when an answer fails. This is the same pattern as LangChain's Open SWE model router, which reported 64 percent lower cost across 973 threads, and the workshop gives you a safe sandbox to see how much quality you lose by routing wrongly.

Routing sorting easy questions to cheap models and hard ones to premium ones in the Token Wars cost microhack

Gateway policies

Every inference call in the kit goes through an Azure API Management AI gateway (via_gateway: true), except the independent judge. Policy templates in the repo cover a starter gateway, a solution gateway and a multi-provider failover policy for the stretch goal. Gateways are where rate limits, retries and provider failover live in real deployments, so having a working example is the most reusable part of the repo.

Bring your own model

Challenge 2 compares open-weight and self-hosted options, including a Llama-3.3-70B-Instruct deployment, optional Ollama on Container Apps, and optional Fireworks on Foundry. A coach-only demo, "Own the Weights", fine-tunes a small open-weight model and runs a break-even calculator between self-hosting and per-token pricing. For a real-world example of small models beating larger ones on a narrow task, see Coinbase's fine-tuned Qwen3.5 9B fraud agent.

Pay-as-you-go principle

The README says every model in the kit is billed per token in Global or Data Zone Standard, nothing requires provisioned throughput, and provisioned capacity appears only as a reference line in the coach's break-even demo. Each team uses its own Azure subscription so that quota, feature registration and cost stay isolated, while the coach uses a separate subscription for the leaderboard and the fine-tuning demo.

The scoring rule is the lesson

Most cost-cutting fails because it optimises the wrong number. Token Wars scores cost per successful answer under a hard quality floor: at least 85 percent of the 100 questions judged correct by an LLM judge. A cheap model that answers wrongly scores zero. The judge is a separate GPT-5.5 deployment that the README says to leave unchanged, which prevents a team from tuning against its own judge.

Two caveats a careful reader will want. An LLM judge is a proxy; it has biases and can be gamed by length or style. And 100 questions is a small sample, so a result at 85 percent has real noise around it. For your own production evaluation you would add held-out questions and human spot checks.

How to run it

The README's quick start is:

1. Sign in with the Azure CLI and export your subscription ID.
2. In the infra folder, copy the tfvars example, set your team name, run terraform init and terraform apply (10 to 20 minutes; the API Management service is the slowest part). It writes the .env and models.json for you.
3. In Python, install requirements and run python -m tokenwars doctor, then a 20-question quick baseline, then the full scored run with the submit flag. In .NET, use the equivalent dotnet run commands.

Prerequisites: your own Azure subscription with Owner, or Contributor plus User Access Administrator; model quota in the default region (swedencentral) for the three GPT-5.6 tiers, GPT-5.5, text-embedding-3-small and Llama 3.3 70B; Terraform 1.6 or newer; and the Azure CLI. Fireworks challenges need a registered feature flag, which the README says to request at least a day ahead.

No Azure subscription? The README says every command works offline in mock mode with simulated token counts. That is enough to read the code and the strategy flags, but not to learn real prices.

A worked way to think about it

Suppose the baseline answers 100 questions and 90 are judged correct. Cost per successful answer is total spend divided by 90, not by 100, so wrong answers still cost money. Now imagine you route 60 simple questions to the Economy tier and the judge still marks 88 correct. If spend falls by half, cost per successful answer falls by a bit more than half only because the correct count barely moved; but if the correct count dropped below 85 you would be disqualified outright. These numbers are illustrative, not from the repo, and they show why the quality floor and the cost metric must be tracked together.

The same arithmetic explains why caching is the first thing to try. A cached prefix cuts input cost on every call with no change in answers, so it improves the numerator while leaving the denominator alone. Output caps are similar for chatty models. Routing and model swaps come later because they can change the denominator.

Teams using the kit with GitHub Copilot will find each exercise in the participant guide comes with a collapsed agent prompt and a collapsed solution, tailored to the Python or .NET track. That makes it usable as self-study: attempt the exercise, check the result, and only then open the solution.

What it does not tell you

• Prices are a snapshot. The README lists GPT-5.6 Standard Global short-context rates per 1M input, cached input and output tokens (Sol $4.00 / $0.50 / $20.00, Terra $2.00 / $0.20 / $12.00, Luna $0.20 / $0.02 / $1.20) and says Sol's rates are promotional through at least 2026-11-30. Costs are computed from the kit's pricing.json, so recheck current rates before drawing conclusions.
• Model names reflect the repo's defaults. The tier table says explicitly that the tier name does not guarantee capability or price, and tells you to re-measure quality if you substitute models.
• One scenario. Customer-support Q&A over a small knowledge base is friendly to caching and retrieval. Agentic coding or long documents will behave differently.
• Tier rename breaking change. The README says existing checkouts must rename the old frontier_model, mini_model and nano_model variables and redeploy fresh infrastructure rather than migrating state.
• Early repo. The page showed four commits when we looked, so expect changes.

What to copy into your own app today

1. Move dynamic data (the user's name, order details) after the stable system prompt so prefix caching can work.
2. Retrieve the two or three relevant documents instead of including everything.
3. Set an output token cap that matches the answer you want.
4. Put a cheap classifier in front and escalate only on failure.
5. Measure cost per successful answer under a quality floor, not tokens per request.
6. Compare one open-weight model on your real questions before you assume the premium tier is needed.

If you want to see how agent token costs vary by setting, our Grok 4.7 on Bedrock reasoning-effort test over 72 agent runs is a measured example, and Microsoft's own Azure cost-cut report and denial is useful context on cloud economics.

Related reading

• Context window pricing decoded
• LangChain Open SWE model router: 64% cheaper
• Grok 4.7 on Bedrock: reasoning effort and token cost
• Coinbase's fine-tuned Qwen3.5 9B fraud agent
• Claude on Azure Foundry: who bills whom
• Microsoft Azure 20% cost cuts report: claimed vs verified
• Official: microsoft/Token-Wars on GitHub

Details reflect the repository README as of October 14, 2026 and may change.