cs.CL 2602.12275

On-Policy Context Distillation for Language Models

Proposes On-Policy Context Distillation (OPCD), minimizing reverse KL divergence on self-generated trajectories, boosting knowledge internalization for tasks like math reasoning and transfer.

Tianzhu Ye, Li Dong, Xun Wu et al.

2026-02-13 134 citations 39