OpenAI chief scientist Jakub Pachocki published an essay titled Perplexity and Intelligence on 22 November 2026, sharing the company’s internal model-prediction scores and proposing shared limits on how far frontier labs develop their models before monitoring tools improve, Explainx reported.
Key points
- OpenAI shared a log-perplexity curve spanning GPT-3.5 through more recent models, measured against a fixed set of internal code.
- Pachocki proposes limits tied to prediction scores and other measurements, with monitoring milestones required before labs move past them.
OpenAI’s fixed code snapshot
Perplexity scores a model’s success at guessing the next small unit of text, as though it were filling in a page one piece at a time without having seen it before. OpenAI’s curve uses log-perplexity measured on a fixed snapshot of its internal code, so each model faces the same prediction material. A lower score means its predictions are sharper.
On the published curve, OpenAI associates GPT-3.5 with a log-perplexity of about 0.99 and GPT-4 with about 0.91. Pachocki associates those stages with fluent conversation and productive chain-of-thought reasoning, respectively, and says the capabilities were not predictable from the thresholds in advance.
Pachocki’s 0.76 mathematics account
An internal model that reached about 0.76 log-perplexity began resolving longstanding open mathematics problems, according to Pachocki’s account reported by OfficeChai. Pachocki warns that research into pretraining algorithms could see an avalanche of advances as soon as 2027, without a matching acceleration in alignment work.
Pachocki’s proposed perplexity limits
Pachocki proposes that frontier labs publish comparable measures of pretraining progress where possible, agree on evaluations and safety testing, and set initial limits tied to perplexity and companion measurements. Sam Altman, OpenAI’s chief executive, said he hoped the field could agree to “common limits”.
The proposal faces measurement problems Pachocki acknowledges: scores from different labs are difficult to compare without an agreed dataset, perplexity is not a score for tasks people perform, and reasoning techniques can add capability without lowering a base model’s loss. He proposes that labs hold to an agreed limit until monitoring reaches a concrete milestone, giving a successor to chain-of-thought monitors as an example.