Improving Nemotron and Qwen on GDP.pdf with xToken

We trained Qwen3.5-9B and Nemotron 3.5 Super VL on 538 professional document tasks using NVIDIA’s xToken, a cross-tokenizer distillation method introduced by Sreenivas et al. (2026). On held-out GDP.pdf tasks, Qwen3.5-9B’s strict pass@1 increased from 4.40% to a peak of 8.20%. Nemotron 3.5 Super VL’s rose from 6.67% to 11.83%.

The original xToken study focused on continued pre-training of a 1B base model. Later work by Liu et al. (2026) evaluated it as a baseline on math and code tasks with the post-trained Qwen3.5-2B. We build on our earlier GDPval experiments to apply xToken to instruction-tuned students on professional document tasks.

We used a modified version of the off-policy xToken recipe in NeMo RL to support long document inputs with images, cached teacher predictions, and teacher selection based on rubric performance.

In our previous GDPval experiments, we used on-policy distillation (OPD) to train smaller student models with feedback from stronger teachers. In OPD, the student generates a response to a task, and the teacher assigns a probability to each token given the task and the preceding text. Training uses these probabilities to guide the student toward choices the teacher considers more likely.

Our student and teacher shared a tokenizer, which splits text into pieces called tokens. Because both models used the same pieces, we could compare their next-token probabilities directly.

That direct comparison breaks down when the models split text differently. For example, Nemotron 3.5 Super VL tokenizes Footnote as one token, while Qwen3.8-27B splits it into Foot and note. One student token now corresponds to two teacher tokens, so their predictions no longer line up.

To make this comparison possible, we used xToken to align groups of tokens that cover the same text and connect the two vocabularies through a weighted map, W. In the Footnote example, W divides the student’s probability for Footnote between the teacher’s Foot and note entries, with the weights determining how much goes to each. Applying this map across the vocabulary lets us compare the two models’ probabilities for the same token entries.

This comparison provides the training signal: projected forward KL (P-KL) penalizes the student when its mapped prediction gives too little probability to choices the teacher considers likely. With the teacher and map held fixed, training uses that signal to update the student’s weights, bringing its mapped predictions closer to the teacher’s while leaving its tokenizer unchanged.

Supervised fine-tuning (SFT) teaches the student to reproduce what the teacher wrote, using the student’s own tokenizer. We also want the student to learn how the teacher weighed the alternatives, which is what xToken adds.

For example, suppose the teacher’s next-token probabilities are 60% for report, 30% for summary, and 10% for everything else. If it chooses report, SFT uses that token as the target.

With xToken, the student can also learn from the 30% probability assigned to summary, even though the teacher chose report.

Both students learned from three teachers: Qwen3.8-27B, DeepSeek-V4.1-Flash, and Kimi-K3. The Qwen student and teacher share the same text tokenizer; DeepSeek and Kimi provide feedback across tokenizers.

While the original xToken paper selects standard KL, P-KL, or H-KL for each teacher–student pair, we used P-KL for all three teachers and did not compare the alternative loss formulations.

Within each aligned span, we multiplied the token probabilities and normalized the result before applying the teacher’s vocabulary map. We computed a separate P-KL loss for each qualifying teacher on the same training response and averaged these losses per task. Across the training batch, we balanced this distillation loss with the loss for reproducing the response, then updated only the student.

We evaluated both students before and after training on held-out GDP.pdf tasks, which test reasoning over professional documents. We generated five answers per task and counted an answer as a strict pass only when it met every rubric criterion. Strict pass@1 avg5 is the average success rate across those attempts; strict pass@5 is the share of tasks with at least one passing answer.

Both models improved strict pass@1 and pass@5 after P-KL training with a pool of three teachers. Their average outputs also grew by roughly 74–80%, so the gains came with higher token use.

1. Sharath Turuvekere Sreenivas, Adithyakrishna Venkatesh Hanasoge, Mingyu Yang, Ali Taghibakhshi, Saurav Muralidharan, Ashwath Aithal, and Pavlo Molchanov. (2026). X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation. NVIDIA. arXiv:2605.21699.
2. Jiacheng Liu, Jingwei Song, Qituan Zhang, Siheng Chen, and Linfeng Zhang. (2026). Beyond Token Alignment: Event Completion for Cross-Tokenizer On-Policy Distillation. arXiv:2609.34738.
3. Qwen. Qwen3.5-2B model card. Hugging Face.
4. Suhaas Garre, Emily Ritchie, Sushant Mehta, and Edwin Chen. (2026). GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents. arXiv:2607.11192.
5. NeMo RL: Cross-Tokenizer Off-Policy Distillation.
6. NVIDIA. NeMo Gym: GDP.pdf benchmark, document agent, and rubric verifier.
7. AfterQuery’s earlier on-policy distillation experiments on GDPval.