GLiNER2.5-Decide: A 340M Open-Weight Model for Fast, Rule-Bound Decisions

A 340-million-parameter model is not usually a headline in 2026, but GLiNER2.5-Decide is aimed at a job that eats a lot of large-model tokens: making small, repeatable decisions. Route this ticket, flag this message, pick the intent, judge this answer. Fastino Labs announced it on September 24, 2026 as an open-weight encoder that runs on a CPU and returns decisions with confidence scores.

For builders, the value is not the parameter count. It is replacing an LLM call with a deterministic, cheap, auditable step where a generative model is overkill.

TL;DR: what it is and what it costs

How does a "decision model" work?

Instead of prompting a language model to output a label, you define a schema. Each entry is a typed question with permitted answers and a cardinality: single answer, multiple answers, or ordered values. Schemas can include instructions, examples, answer descriptions, and cross-question rules such as implications, exclusions and cardinality constraints.

The encoder scores compatibility between the text and each candidate answer. A constrained decoder then picks the best joint assignment that satisfies the rules. That joint step is the key idea.

Take a moderation example. If independent scores say a message is "safe" but also assign a "harm type," a normal classifier can return both. With a rule such as "if safety is safe, harm type must be none," the decoder resolves the contradiction. The output carries probabilities, confidence and feasibility metadata, so downstream code can act on uncertainty instead of trusting a hard label.

Why an encoder instead of a decoder LLM?

Encoder models read the whole input at once and emit scores, so latency is predictable and there is no sampling variance. Fastino puts it this way: "A specialized, non-generative encoder can outperform much larger decoder readouts while also leading comparable encoder systems." That is a claim about this benchmark, not a general law, but it matches a long line of work on small specialized classifiers, which we covered in Jev vs. XGBoost and BERT.

What do the benchmarks show?

Fastino evaluated on Fast Decisions, an internally generated and unseen suite built from 17 datasets that test routing, triage, classification, sentiment and content understanding.

It led on 9 of 17 datasets. Standout numbers reported: 75.3% on support intent (18.6 points ahead of the next model) and 64.3% on banking intent (8.6 points ahead).

Read these with care. The suite is built by Fastino, so the task mix may favor its approach, and the comparison set is other small decision models, not frontier LLMs. A 3-point average lead over JevK5 is modest. JevK5 comes from the Jev family of "System One" models; for background see what a System One model is. Whether the gap matters depends on your datasets.

Can it really run on a CPU?

Fastino reports latency at a 64-token batch:

At 1,024 tokens, it reports 52.6 ms on A100 and 75.6 ms on V100. Two takeaways:

• GPU choice barely matters. Older, cheaper cards land in the same band as an A100, so this is not a workload that needs premium hardware.
• CPU is viable for moderate volume. About 167 ms on a many-core server is fine for background triage and gating, less so for a tight user-facing loop at high concurrency. Test with your own token lengths.

The point about air-gapped operation is real for regulated teams: no data leaves the machine, and the same weights run on a laptop-class CPU.

Where would you use it?

Fastino names tool calling, model routing, browser and computer use, and LLM-as-a-judge. In practice:

1. Gate agent actions. Before an agent runs a tool, ask typed questions: "Is this action destructive? Does it touch production?" with rules that block risky combinations.
2. Route requests. Decide which model handles a task. This pairs with the routing idea behind Cursor Router and the cost work in AT&T's 56% coding-cost cut.
3. Triage tickets and messages. Intent, urgency, sentiment and department in one pass.
4. Judge outputs cheaply. A first-pass rubric check before an expensive LLM judge runs.
5. Extract with constraints. GLiNER-family models also do span-level entity and relation extraction, so one model can label entities and classify in a single step.

A sketch of a schema

Fastino's exact API surface is documented on Hugging Face and its site, so this is an illustrative shape rather than copy-paste code:

{
  "questions": [
    {"id": "intent", "answers": ["refund", "cancel", "bug", "other"], "cardinality": "single"},
    {"id": "urgent", "answers": ["yes", "no"], "cardinality": "single"},
    {"id": "escalate", "answers": ["yes", "no"], "cardinality": "single"}
  ],
  "rules": [
    {"if": {"intent": "bug", "urgent": "yes"}, "then": {"escalate": "yes"}}
  ]
}

Check the model card for the exact schema format before wiring it in.

How does it compare with just prompting an LLM?

The honest rule is to use the encoder where the label space is fixed and volume is high, and fall back to an LLM for the long tail. A common pattern is confidence-based routing: if the decision model is confident, act; if not, escalate to a larger model.

What are the risks?

• Vendor benchmark. Fast Decisions is Fastino's own suite. Evaluate on your data before switching anything.
• Absolute accuracy is moderate. 60.1% average means many tasks are far from solved. Fine-tuning on your labels is part of the intended workflow.
• No guarantee. Fastino states it does not verify or guarantee output accuracy for any specific purpose. Do not use it as the only gate on a high-stakes decision.
• Non-generative. It will not explain itself in prose; you get scores and structured output.
• Hosted API lock-in is optional. The weights are Apache 2.0, so you can self-host.

What this means for what you build or pay

• Cost: any step where you call a frontier model just to pick a label is a candidate for replacement, with savings scaling with volume.
• Privacy: on-device or air-gapped classification removes a data-sharing question from procurement.
• Reliability: rule-bound joint decoding gives you consistent, testable outputs for agent guardrails.
• Action: pull the model from Hugging Face, run it on a labeled sample of your traffic, and compare against your current LLM classifier on accuracy, latency and cost before deciding.

How would you evaluate it in a day?

1. Pull 300 to 500 real, labeled examples from your own traffic, split into a tuning set and a held-out test set.
2. Write the schema with your real labels and two or three rules that encode business logic, such as "billing intent and urgent implies escalate."
3. Run the model zero-shot and record accuracy, per-class recall and the confidence distribution.
4. Compare against your current LLM classifier on accuracy, p50 and p95 latency, and cost per thousand decisions.
5. Set a confidence threshold below which requests escalate to a larger model, and measure how many fall through. If under a fifth of traffic escalates and accuracy on the confident slice beats your LLM, the swap pays for itself.
6. Fine-tune only if needed. Fastino positions the encoder as easy to fine-tune, so a small labeled set may lift the classes it misses.

Record the result. A benchmark average of 60.1% says little about your label space; your own held-out set is the number that matters.

Where does GLiNER fit in the family?

GLiNER models began as compact encoders for named-entity and relation extraction, and Fastino has extended them toward classification and structured records in one pass. Decide is the version built around typed questions and constraints. It is best understood as a fast reading layer in front of an agent, not a replacement for a reasoning model.

Related reading on explainx.ai

• Jev vs. XGBoost and BERT: is a System One model actually new?
• What is a System One model?
• Cursor Router: automatic model selection
• AT&T cuts AI coding costs 56% with model routing
• Structured output and tool use with JSON Schema
• Agent skills security and verification

Official sources: Fastino's launch post and the model card on Hugging Face.

Benchmark and latency figures are as reported by Fastino Labs on September 24, 2026 and have not been independently reproduced.