Ask a large language model how many times the letter “r” appears in “strawberry” and there’s a decent chance it says two. Then ask it to explain the Krebs cycle and it produces something a first-year biology student would envy. That gap, fluent on the hard stuff and clumsy on the trivial, is the clearest window into what these systems are and aren’t.
Large language models went from research curiosity to infrastructure in about six years. They draft contracts, write commit messages, summarise depositions, and quietly power half the autocomplete you touch. Understanding them well enough to use them without being fooled has become a genuinely practical skill.
What a Large Language Model Is, Mechanically
Strip away the marketing and a large language model is a function that takes a sequence of text and returns a probability distribution over what token comes next. A token is roughly a word fragment; “unbelievable” might be three of them. The model repeats that calculation billions of times to produce a paragraph.
The “large” is doing real work. GPT-3 shipped with 175 billion parameters in 2020. Llama 3.1 405B pushed past 400 billion. Modern flagship models train on somewhere between 10 and 20 trillion tokens of text and code. Parameters are the learned weights; tokens are the fuel.
The transformer, briefly
Almost everything in this class of model descends from the transformer architecture, introduced in the 2017 paper “Attention Is All You Need.” The key trick is self-attention: every token gets to look at every other token and decide how much each one matters for the current prediction. That’s why a model can keep the subject of a sentence straight across four clauses, and it’s why context length matters so much.
How a Model Gets From Raw Web Text to Chat Partner
Training happens in stages, and each stage leaves a fingerprint on the behaviour you see.
Pretraining is the expensive part. Trillions of tokens stream through the network, and the model picks up grammar, facts, code patterns, translation, and a startling amount of world structure as a side effect of getting better at prediction. This stage eats the overwhelming majority of the compute budget.
Supervised fine-tuning comes next. Curated instruction-and-response pairs teach the model the format of being helpful rather than merely continuing text.
Preference tuning, whether RLHF or its simpler cousin DPO, uses human or AI rankings to shape tone, refusal behaviour, and safety. This is where a raw text predictor becomes something that argues politely and declines certain requests. The people who design that pipeline have outsized influence on what the final model will do; the story of Ilya Sutskever and the scaling culture at OpenAI shows how those choices actually get made.
The Scale Debate Is Still Unsettled
For a while the story was simple: more parameters, more data, better results. The Chinchilla paper in 2022 complicated it, showing that many models were badly under-trained relative to their size and that a smaller model fed more tokens could match a bigger one at lower inference cost. That insight reshaped how everyone budgets compute.
Then came the “emergent abilities” argument, the claim that certain skills appear abruptly past some threshold. A 2023 paper by Schaeffer and colleagues argued much of that was an artefact of measurement: use a smoother metric and the jumps flatten into curves. Emerging or not, capability kept climbing. What has changed is that benchmark scores now saturate within months of release, so lab claims deserve more skepticism than a leaderboard implies. A closer look at what GPT-class models actually deliver in 2025 is worth reading before you take any demo at face value.
Where Large Language Models Genuinely Deliver
- Rewriting and restructuring. Turn a rambling thread into a memo, a memo into bullets, bullets into a customer email. The model isn’t inventing facts, it’s rearranging ones you supplied.
- Code assistance. Autocomplete, test generation, and explaining unfamiliar codebases are consistently strong. Accuracy drops sharply on large refactors that span many files.
- Classification at volume. Sorting 50,000 support tickets into twelve categories is a task where a fine-tuned smaller model often beats a giant general one on both cost and accuracy.
- First drafts of anything structured. Outlines, job descriptions, SQL queries, regex. A starting point you edit beats a blank page.
Where they struggle: reliable arithmetic over many steps, precise citations without retrieval, up-to-date facts, and multi-step planning with real-world consequences. Also, notoriously, anything involving counting letters.
Hallucination Isn’t a Bug You Can Patch
A next-token predictor has no internal representation of “I don’t know this.” It has a distribution over plausible continuations, and a confident-sounding falsehood is often the most probable one. That’s why hallucination persists even in models that ace exams.
Retrieval helps enormously. Put the relevant documents in the prompt and ask the model to answer only from those, and factual error rates drop sharply. It doesn’t eliminate the problem, because the model can still misread what you handed it. Treat generated facts as claims to verify, not conclusions.
Memory, Context, and the Illusion of Continuity
Context windows have grown from 4,096 tokens to a million and beyond. Bigger windows are useful, but “lost in the middle” effects persist: models attend well to the start and end of a long prompt and skim the rest. Stuffing 400 pages into context and hoping is not a strategy.
There’s also a real gap between context and memory. Nothing persists between sessions unless you build it. Most products that feel like they remember you are stitching a database onto a stateless API call.
Using Them in Production Without Regret
The distance between a demo and a deployed system is mostly engineering discipline. Build an evaluation set of 50 to 200 real examples with known good answers before you tune a single prompt. Log prompts and outputs. Version prompts like code. Pin model versions, because a silent provider update can shift your accuracy by double digits overnight.
For teams getting started, a hands-on walkthrough of the OpenAI API covers the mechanics from that first call through to retries and rate limits. Once you layer tool use and multi-step loops on top, you’re in agent territory, and the current state of AI programming tools and agents maps that landscape.
What Reasoning Models Changed
The newest shift is spending compute at inference time rather than only at training time. Reasoning models generate long internal chains of thought before answering, trading latency and cost for accuracy on maths, logic, and multi-step code problems. The gains are real on hard benchmarks and largely invisible on easy ones. You don’t need a reasoning model to reformat a date.
This is where the terminology gets slippery. Systems that plan, check their own work, and revise feel qualitatively different from a text predictor. Whether that adds up to something worth calling general intelligence is a separate argument, and the two very different meanings of “general AI” explains why so many conversations about it talk past each other. The move toward machines that reason rather than retrieve traces where that line is genuinely being drawn.
The Quiet Shift Toward Smaller Models
While headlines chase trillion-parameter flagships, much of the practical progress is happening between 7 and 30 billion parameters. Quantised open-weight models now run on a single consumer GPU or even a laptop, and distilling a large model’s outputs into a small one routinely produces something that hits 90% of the quality at 2% of the cost for a narrow task.
The pricing math is brutal in a useful way. A frontier API call might cost a few cents per thousand tokens. A locally hosted small model costs nothing per call once the hardware is paid for. For a classification job running ten million times a month, that difference decides whether the product exists at all.
So the practical question with large language models is rarely “which one is best.” It’s which task, what accuracy threshold, how much latency the user will tolerate, and what happens when the model is wrong. Answer those four and the model choice tends to make itself.