Model Architecturesaka CLMaka Contrastive LM

Contrastive Language Model

A Contrastive Language Model (CLM) is a System One model that trains a state encoder and an action encoder with a contrastive loss, then picks the candidate action whose embedding best matches the current state. It selects from options you supply; it does not generate text.

Ask Melo about this ← all terms

CLM-8B, released in September 2026 under Apache 2.0, is trained on 60M Q&A pairs, 30M synthetic hard negatives and 1M agentic trajectories, using a frozen LLM backbone plus a small trained head per encoder. Because state and action embeddings are cached independently, it runs up to 9x faster than Jev on some zero-shot tasks, and a fine-tuned version acts as a fast verifier for best-of-N coding agents. Not to be confused with a causal language model, which the abbreviation CLM often means in NLP code.

Where Contrastive Language Model comes up

FAQ

Is a Contrastive Language Model the same as a causal language model?: No. In most NLP code, CLM means causal language modeling, the next-token objective behind GPT-style models. A Contrastive Language Model is trained with a contrastive loss such as InfoNCE and ranks candidates by embedding similarity instead of generating tokens.
Why is a Contrastive Language Model fast?: It needs one embedding pass for the new state plus a dot product per candidate. Because the state and action encoders are independent, action embeddings can be cached and reused across states.
What can a Contrastive Language Model not do?: It cannot write open-ended text, plan long tasks by itself, or choose an action you did not list. Its scores should be checked for calibration before use as probabilities.

Contrastive Language Model

FAQ

Is a Contrastive Language Model the same as a causal language model?

No. In most NLP code, CLM means causal language modeling, the next-token objective behind GPT-style models. A Contrastive Language Model is trained with a contrastive loss such as InfoNCE and ranks candidates by embedding similarity instead of generating tokens.

Why is a Contrastive Language Model fast?

It needs one embedding pass for the new state plus a dot product per candidate. Because the state and action encoders are independent, action embeddings can be cached and reused across states.

What can a Contrastive Language Model not do?

It cannot write open-ended text, plan long tasks by itself, or choose an action you did not list. Its scores should be checked for calibration before use as probabilities.

Contrastive Language Model

Related terms

Where Contrastive Language Model comes up

FAQ

Contrastive Language Model

Related terms

Where Contrastive Language Model comes up

FAQ