The candidate set changes at every generation step according to the model's distribution. Renormalizing within that set removes the low-probability tail while retaining more options when the model is uncertain.
Top-p sampling draws the next token from the smallest high-probability set whose cumulative probability reaches a chosen threshold.
The candidate set changes at every generation step according to the model's distribution. Renormalizing within that set removes the low-probability tail while retaining more options when the model is uncertain.