Model Architecturesaka Causal Language Modelingaka Autoregressive Language Model

Causal Language Model

A causal language model predicts the next token from the tokens before it, and generates text one token at a time. GPT-style chat models are trained this way.

Ask Melo about this ← all terms

Training uses a causal attention mask so each position sees only earlier tokens, and the model learns to maximize the probability of the true next token. The abbreviation CLM often refers to this objective in NLP libraries, which is different from a Contrastive Language Model that ranks candidate actions by embedding similarity.

Where Causal Language Model comes up

Model Architecturesaka Causal Language Modelingaka Autoregressive Language Model

Causal Language Model

A causal language model predicts the next token from the tokens before it, and generates text one token at a time. GPT-style chat models are trained this way.

Ask Melo about this ← all terms

Causal Language Model

Related terms

Where Causal Language Model comes up

Causal Language Model

Related terms

Where Causal Language Model comes up