OpenAI does not publish a training manual for AGI. What it does publish — research papers, system cards, safety frameworks, the occasional offhand comment from a researcher on a podcast — is enough to sketch the pipeline. That sketch is worth having, because most public argument about how OpenAI is building AGI treats “training” as a single thing. It isn’t. It is at least four distinct processes stacked on top of each other, and each one solves a different problem.
Here is how each layer works, what it costs, and where it still comes apart.
Pretraining: the expensive layer nobody sees
Everything begins with next-token prediction over a very large corpus. GPT-3 in 2020 used 300 billion tokens and 175 billion parameters. By GPT-4, widely circulated leaks put the token count near 13 trillion, and Epoch AI’s estimate of training compute sits around 2×10²⁵ FLOPs. OpenAI has confirmed none of those figures. The direction of travel is not in dispute.
The unglamorous work is the data
Raw web crawl is mostly duplicates, boilerplate and spam. Most of the pretraining effort goes into cleaning it: quality classifiers, near-duplicate removal, boilerplate stripping, and deliberate upweighting of sources that teach reasoning. Code earns its place twice over, since it carries explicit reasoning steps that ordinary prose tends to leave implicit.
- Web text — broad coverage, heavily filtered and deduplicated.
- Code — structured logic, step-by-step problem solving, syntax that punishes sloppiness.
- Books, papers and reference works — long-form coherence across thousands of tokens.
- Synthetic data — model-generated text, used increasingly to fill gaps and sharpen narrow skills.
Scaling laws and the compute bill
OpenAI’s early bet rested on a simple empirical finding: loss falls predictably as you add parameters, data and compute. That gave the team a planning tool and a multi-billion-dollar capital requirement in the same breath, since a frontier run means tens of thousands of GPUs and months of wall-clock time. Whether the curve keeps paying out is the biggest fork in the road, and the assumptions behind the 2027 and 2100 AGI forecasts come down to it.
Post-training: RLHF and the reward model
A pretrained model is a text predictor, not an assistant. Ask it a question and it may well continue with more questions. Post-training fixes the shape of the response.
OpenAI’s InstructGPT paper from January 2022 laid out the template still in use. Human contractors write demonstrations of good answers, then rank several model outputs from best to worst. Those rankings train a reward model, which scores new outputs, and the language model is then tuned with reinforcement learning to maximise that score. The headline result was striking: a 1.3B-parameter InstructGPT model was preferred by labelers over the 175B GPT-3 it was derived from. Preference tuning bought more perceived quality than a hundredfold parameter increase.
Rules sitting alongside the reward model
Pure human preference has a known failure mode. Raters reward confidence and flattery, so the model learns to be agreeable rather than correct. The GPT-4 system card described a partial fix: rule-based reward models. A separate classifier checks outputs against a short written list of behaviours and penalises violations, with a small bias term added so the model does not learn to game the classifier itself. Crude, but it moved the needle where human preference data was thin.
Teaching models to reason: the RL shift
From GPT-4 onward the emphasis moved from producing a better final answer to producing better intermediate work. OpenAI’s process supervision research, published as Let’s Verify Step by Step in 2023, rewarded each step of a solution rather than only the endpoint. On a challenging maths subset it lifted solve rates from roughly 42% to 78% compared with outcome-only supervision.
The o1 release in September 2024 made the strategy explicit: large-scale reinforcement learning applied to chain-of-thought, so the model learns which lines of reasoning lead somewhere and which waste time. Two consequences follow. Accuracy now scales with thinking time at inference, not just with parameter count, which opens a second axis of compute to spend. And the reasoning traces themselves become training data, raising an awkward question about how you supervise a model whose internal work is longer and stranger than anything a human rater can check in one sitting.
Scalable oversight: training something smarter than the trainer
This is the crux of the AGI-specific problem. Fine-tune a model on labels from humans and you cap it near human performance on that task. OpenAI’s Superalignment team, announced in July 2023 with a pledge of 20% of company compute, existed to attack exactly this. It was disbanded in May 2024 after several senior researchers left.
Weak-to-strong generalisation
The team’s flagship experiment trained a strong model using labels generated by a weak one: a GPT-2-era supervisor teaching a GPT-4-era student. Naive fine-tuning recovered about 60% of the gap between weak and strong performance on benchmark tasks. An auxiliary confidence loss pushed that to roughly 80%. Not nothing, not solved. The technical picture is worth reading in the primary sources, and this review of OpenAI’s AGI research papers separates the load-bearing work from the press releases.
Deliberative alignment
Published in December 2024, this approach trains a reasoning model to recall and reason over an explicit safety specification before it answers, rather than relying on reflexes baked in during RLHF. Models trained this way showed large drops in harmful compliance with little loss in general capability. It is the most concrete example so far of using reasoning itself as a safety mechanism rather than a coat of paint on top of one.
Evals, red teaming and the deployment gate
None of the above decides whether a model ships. That is the job of the evaluation stack: benchmark suites, domain-expert red teams, third-party access programmes, and the Preparedness Framework, which scores models against risk categories including cyber offence, bioweapons, persuasion and model autonomy. The thresholds matter less than the escape hatches, and the clauses and loopholes in OpenAI’s AGI policy are worth understanding in detail before you trust a headline score.
Separating signal from noise in this space is genuinely hard, since every capability demo doubles as a safety claim. Reading AGI safety news critically, rather than as a press-release feed, is the only way to judge whether the gate is real or decorative.
Where the methods still come apart
Three problems sit unresolved underneath the whole stack.
Safety training is thin relative to capability training. Pretraining runs for months over trillions of tokens. Alignment tuning is a comparatively small pass at the end. Strip the post-training layer off and you get the base model’s raw behaviour back, which tells you where the dispositions actually live.
Interpretability lags behind measurement. We can observe what a model does far better than why. Without that, evaluation becomes a search over inputs for bad behaviour, and the space of inputs is infinite.
Concentration risk. Only a handful of labs can afford frontier runs, which narrows the set of people able to check the work. It is part of why senior researchers keep leaving to build alternatives with different training philosophies, and the new labs spinning out of the ChatGPT team make a useful counterweight to OpenAI’s own account of itself.
The honest reading is that the capability side of OpenAI’s AGI training methods is well understood and broadly repeatable, while the oversight side remains a set of promising prototypes. Whether those prototypes scale faster than the models they are meant to supervise is the open question, and no published result answers it yet.

