Perplexity Trains Its Computer Agent on Real Mistakes With OPSD
Perplexity trains its GLM 5.2 computer agent using hint-guided self-distillation to correct real execution mistakes.
Stock photo for illustration only, not from the actual event
- Perplexity trained its computer agent using real execution mistakes instead of only successful sessions.
- The base model is GLM 5.2, available on Hugging Face, though training weights remain unreleased.
- The pipeline utilizes On-Policy Self-Distillation (OPSD) combined with LLM judge filtering.
Perplexity has developed a training approach for computer agents that focuses on learning from actual mistakes rather than simply imitating successful sessions. Currently, the model operates exclusively as an option inside Perplexity Computer, and the team has not released the post-trained weights or training code.
Standard rejection sampling fine-tuning (RFT) typically judges each session and imitates only the successful outcomes. However, a successful final result does not mean every intermediate step was correct. An agent can recover from a flawed tool call and still provide the right answer, meaning that imitating the entire trajectory can reinforce errors while discarding failed sessions throws away valuable data on avoidable mistakes.
Stock photo for illustration only, not from the actual event
The Perplexity team separates two key decisions: identifying which sessions contain behavior worth imitating, and identifying which turns hold mistakes that need correction. Each assistant turn receives one of three specific treatments.
- Successful sessions supply both imitation and correction targets.
- Unsuccessful sessions supply correction targets only.
The correction phase relies on On-Policy Self-Distillation (OPSD). The trainer runs the GLM 5.2 checkpoint twice on the recorded turn. The teacher pass receives a hint while the student pass does not, utilizing teacher forcing so no replacement answer is generated. The teacher's next-token probabilities act as a soft target through forward KL.
Applying OPSD allows AI agents to accurately address historical errors by isolating intermediate procedural flaws from final success. This method prevents the model from reinforcing bad habits that it managed to recover from during standard execution.
The training pipeline gathers eligible Computer sessions served by GLM 5.2 while excluding personally identifiable information and opted-out users. An LLM judge filters tasks rated 4 or 5 on a 5-point scale, requiring approval from two judges to validate a successful session.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment