Causal masking prevents a position from reading tokens that come later in the training sequence. At inference time, each predicted token is appended to the context and used to predict the next one.
A decoder-only model predicts the next token from earlier tokens and can generate sequences autoregressively.
Causal masking prevents a position from reading tokens that come later in the training sequence. At inference time, each predicted token is appended to the context and used to predict the next one.