Perplexity JOLT: Train the Teacher to Teach, Match GRPO With 6.5x Fewer Tokens

Perplexity Research has published JOLT, a post-training method that trains a model to be both a better student and a better teacher of itself. In its October 21, 2026 write-up, A good teacher meets the student where they are, the team reports that on MATH it matched the accuracy of GRPO, the reinforcement learning method behind many reasoning models, using 6.5x fewer processed training tokens, and that an extended version, JOLT+, beat GRPO on every benchmark it tested. The Perplexity page is the source for everything below.

This is a research result on small models, not a product release. It is worth understanding because it attacks a central cost in building reasoning and agent models: sparse reward signals.

TL;DR

The problem: sparse reward in reinforcement learning

In common RL post-training, a model attempts a task, gets one outcome reward per attempt, and learns to favor actions in higher-reward attempts. Perplexity points out that a single outcome reward does not say which intermediate steps helped or hurt, which limits sample efficiency, especially on long agentic tasks where attempts are long and successes rare. GRPO, which compares each trajectory's return with the mean of its group for the same prompt, helps with variance but not with this sparsity.

A popular fix is on-policy distillation (OPD): a stronger teacher scores each token of the student's own attempts, giving dense feedback at every step. On-policy self-distillation (OPSD) removes the separate teacher: the student model, given privileged information such as a verified solution, serves as its own teacher. For related approaches on this blog, see our coverage of Perplexity's hint-guided self-distillation for tool calls, VRSD verbal-regularization self-distillation, and a paper on on-policy distillation collapse and reward hacking.

The insight: good at solving is not good at teaching

Perplexity's argument: a model that solves a task with the answer in view is not necessarily good at teaching it. Its feedback may exploit shortcuts unavailable to the student, put high probability on trajectories the student would never produce, or be unreliable after a student's mistake. Vanilla OPSD only trains the student to imitate the teacher; nothing makes the teacher's feedback useful.

How JOLT works

JOLT trains the shared model in both roles:

1. Teacher role. It receives privileged information and is trained with outcome rewards, with a KL penalty toward the student's behavior so it cannot drift to guidance the student cannot follow.
2. Student role. It follows the standard OPSD objective, minimizing reverse KL against the privileged teacher on the student's own rollouts.
3. Joint update. Both objectives update the same parameters. Each role learns from trajectories generated under its own context.

The theory piece is a "reward-consistent" teacher: one whose distillation update to the student matches the policy gradient of the student's expected task reward, up to a positive scale. The paper-style derivation shows an ideal teacher that reweights the student's actions by exponentiated advantage, and the training objective approximates it with centered returns from the teacher's own attempts. Perplexity itself notes the substitution is not exact. JOLT+ adds the student's own outcome-reward term to its update.

A funnel illustrating distillation from a larger teacher signal into a smaller student, as in JOLT self-distillation

What they tested

Baselines were GRPO, OPSD and pi-Distill, with hyperparameters tuned for each. pi-Distill also trains privileged teacher and student jointly, but its student update is off-policy.

The results

Accuracy. JOLT+ has the highest mean score in every setting. Against GRPO, the strongest baseline, the gains are 1.5 points on GSM8K, 4.2 on MATH, 3.0 on LCBv6 and 9.9 on the AppWorld state-test pass rate, plus 2.7 points on Terminal-Bench 2.1 pass@1. On Terminal-Bench, JOLT+ leads at every attempt budget, from 10.96% at pass@1 to 34.83% at pass@32.

Plain JOLT is weaker on agents. Without any student reward, JOLT beats GRPO on GSM8K, MATH and LCBv6 but stays below both GRPO and pi-Distill on AppWorld, and below GRPO on Terminal-Bench, though above the base model and both distillation baselines. OPSD scores below the base model at every k on Terminal-Bench. The honest reading is that the teacher's dense signal alone is not enough on long-horizon agent tasks, and outcome reward still matters.

Token efficiency. On MATH, JOLT levels off near 72% accuracy, OPSD plateaus near 62%, and GRPO climbs more slowly to about 70%. At a matched 63.44% accuracy, GRPO's observed checkpoint used 8.40 million tokens while JOLT's fitted curve gives 1.30 million, which is the 6.5x figure. Counting only completion tokens widens the gap to about 13x. Across target accuracies from 55% to 65%, JOLT used 9.5 to 14.4x fewer completion tokens and 4.3 to 7.4x fewer processed training tokens.

How to read the 6.5x claim

• A fitted curve, not an observed point. GRPO's number is a real checkpoint; JOLT's 1.30 million comes from a sigmoid fitted to accuracy against log token budget. That is a standard way to compare scaling, but it is an estimate.
• Token accounting matters. Perplexity counts every unpadded prompt, privileged context, optimization pass and teacher-student cross-scoring in the headline number, which is the conservative choice, and reports the larger completion-token gap separately.
• Small models. The MATH efficiency experiment uses a 0.6B model. Whether the gap holds at tens or hundreds of billions of parameters is unknown.
• Requires privileged information. JOLT needs something to put in the teacher's context: verified answers, reference solutions, prior-attempt summaries or verifier output. Tasks without such information cannot use it directly.
• Self-reported. Perplexity ran and chose the benchmarks and baselines. Independent reproductions will tell how robust the gains are.

Why it matters for builders

Post-training budgets are dominated by rollouts. A method that gets the same accuracy from fewer tokens, or more accuracy from the same tokens, directly lowers cost, which matters for labs and for teams fine-tuning open models for agents. The result also sits with other recent work from the company on cheaper, tighter model training and serving, such as its Photon retrieval engine, and with community interest in GRPO alternatives like the catalog in our FermiSense GRPO write-up.

If you want to experiment, the recipe is conceptually simple to prototype on top of an OPSD or GRPO setup: add a teacher-role rollout with privileged context, a centered-return teacher loss with a KL penalty toward the student, and for JOLT+ the student's outcome-reward term. Perplexity's page includes pseudocode for one training step; the post says the code will be open-sourced soon, but it is not released yet.

A plain-language primer for the terms

GRPO (Group Relative Policy Optimization) was introduced in the DeepSeekMath paper. For each prompt, the model samples a group of answers, scores them, and treats each answer's reward relative to the group average as its advantage. It avoids a separate value network, which is why it became a default for reasoning-model training. Its limitation, as Perplexity notes, is that the learning signal is still one number per attempt.

On-policy self-distillation is described in the OPSD paper (arXiv 2601.18734): one model plays both roles. The teacher copy sees the ground-truth solution in its context, the student sees only the problem, and training pulls the student's token distribution toward the teacher's along the student's own rollouts. This gives feedback at every token without a bigger external teacher.

Reward-consistent teacher is Perplexity's term for a teacher whose guidance, once distilled into the student, points the same way as ordinary RL on the student's reward. In everyday terms, a good tutor does not just show the answer; the tutor gives hints that would also make the student more likely to succeed alone. A tutor who solves the problem by a trick the student cannot reproduce is a bad tutor, even if they are very accurate, and the KL penalty in JOLT is what keeps the tutor within reach of the pupil.

Future direction

Perplexity ends with the idea of jointly training a task-generating teacher and a problem-solving student, using the student's learning progress to shape a curriculum of synthetic tasks. That would turn the teacher from a guide on fixed problems into a designer of what to learn next.

What to watch

• Independent reproductions on larger models and other domains.
• Released code or training recipes.
• Whether the gains survive when privileged information is noisy or unavailable.
• Whether other labs adopt reward-consistent teacher training in their agent RL stacks.

Related reading

• Perplexity hint-guided self-distillation for tool calls
• VRSD self-distillation
• On-policy distillation collapse and reward hacking
• FermiSense GRPO catalog
• Perplexity Photon
• Official: Perplexity Research, JOLT and Perplexity Research index

Details are accurate as of October 22, 2026 and come from Perplexity Research. All results are the authors' own.