The two-second magic trick
You type “explain compound interest like I’m 12” into a chatbot. Two seconds later, a tidy answer appears. No search engine went looking for a page. Nothing was lifted word-for-word from a website. So what actually happened in those two seconds?
A very large pile of arithmetic. That’s the honest answer, and everything else is detail. A modern language model is a mathematical function with hundreds of billions of adjustable numbers inside it. Your prompt goes in, the numbers get multiplied and added together in a fixed order, and a prediction comes out. Run that loop a few hundred times, once per word, and you have a paragraph.
The surprising part isn’t that it works. It’s that this approach produces anything useful at all. For decades, researchers assumed you’d have to hand-code rules about grammar, facts and logic. The breakthrough was abandoning that idea and letting statistics do the heavy lifting.
What we actually mean by “AI” in 2025
Nearly everything you touch today is narrow AI: a system trained to do one job extremely well. Translating text. Flagging a suspicious transaction. Ranking your music queue. None of it is conscious, and none of it wants anything. If you want the wider picture of how these systems are categorised, a plain-English guide to what artificial intelligence actually is covers that ground properly.
The systems grabbing headlines are built on neural networks, a design loosely inspired by brain tissue. Layers of simple units pass signals to one another, with connection strengths that can be turned up or down. Stack enough layers and the network can represent wildly complicated relationships between inputs and outputs.
The part that matters most: training
A freshly built network is useless. Its internal numbers start as random values, so its answers are gibberish. Training is the slow process of nudging those numbers until the output stops being nonsense.
Guess badly, then measure the damage
Show the model a sentence with the final word hidden. It produces a probability for every word in its vocabulary. Early on, that guess is essentially random. A loss function then scores the gap between the guess and the truth: wrong by a mile, big number; wrong by a hair, small number.
Turn the dials, repeat a few billion times
Backpropagation works out which internal numbers contributed to the error and which direction they should move. Gradient descent takes a small step that way. Then you do it again. GPT-3 has 175 billion parameters, and every single one gets touched by this process.
Do that across a few hundred billion words of text from books, Wikipedia, code repositories and the open web, and something strange emerges. To predict the next word well, the model has to internalise grammar, facts, tone and a rough sense of how arguments are structured. Nobody programmed those rules in.
Why a “thinking” machine is really a next-word guesser
Before anything else, your prompt gets chopped into tokens. Roughly 0.75 words per token in English, so a word like “unbelievable” might split into three pieces. Tokens are the unit the model actually sees.
Then comes the transformer architecture, introduced in a 2017 paper bluntly titled “Attention Is All You Need.” Attention lets the model weigh every token against every other token in its window, which is how it works out whether “it” refers to the router or the warranty.
- Input: your prompt becomes a sequence of tokens.
- Attention: each token gets scored for relevance against all the others.
- Prediction: the network outputs a probability for every possible next token.
- Sampling: one token is chosen, usually not always the top-ranked one, which is why answers vary between runs.
- Loop: the chosen token is appended and the whole process runs again.
That loop explains two everyday quirks. Longer answers take longer, because the model is generating one token at a time. And it has no memory of your last conversation unless that conversation is handed back to it as context.
Where hallucination comes from
The model isn’t checking facts. It’s choosing the next token that best fits the pattern. If a plausible citation or case name fits, it will produce one, complete with a realistic-looking volume number. Fluent and true are two separate objectives, and only one of them is being optimised.
Numbers, dates, legal references and specific quotes are the highest-risk zones. Anything the model can’t pattern-match from training data gets filled in with something statistically reasonable. Fine-tuning and human feedback later steer the raw engine toward being helpful, which is why a base model and the chat product built on top of it feel so different.
The bill nobody sees
Training is brutally expensive. Estimates put GPT-3’s run in the low millions of dollars, consuming roughly 1,300 megawatt-hours of electricity. Frontier models today cost considerably more and require thousands of specialised processors running in parallel for weeks. Answering your question is cheap per query, but it happens billions of times a day.
What this means when you sit down to use it
Once you accept that the model is a pattern completer, the practical advice writes itself: give it a pattern worth completing. Examples, structure and constraints beat vague requests every time. A structured approach such as a five-step workflow for turning rough notes into finished work outperforms ad-hoc prompting because it shows the model the shape of the answer you want.
The same principle decides where automation is worth the money. Systems that classify, summarise, extract or route information are where AI reliably pays for itself; open-ended judgement calls are where it wobbles. It’s worth reading where AI automation pays off and where it breaks before committing a budget.
If you’d rather see the machinery up close, build something small. An afternoon project like a working action-item extractor you can build in one sitting will teach you more about tokens, context windows and prompt structure than a month of reading.
And when you’re facing fifty pages of meeting transcripts, a repeatable workflow for using an AI reader will save you far more time than pasting walls of text into a chat box.
Some limits are permanent features
- No memory between sessions unless something stores it and hands it back.
- Knowledge frozen at a training cutoff, though search tools can patch around that.
- No independent way to verify its own claims.
- Finite context windows, so long documents get truncated or summarised.
- Confidence that reads the same whether it’s right or wrong.
Why the next few years will feel different
Three shifts are already underway. Models are being handed tools: a calculator, a search index, a code interpreter. That stops them guessing at arithmetic and starts them running it. Retrieval systems now fetch relevant documents and drop them into the context window, which cuts hallucination sharply on factual questions. And smaller models tuned for one job are quietly replacing giant general ones in production, because a 7-billion-parameter model that’s excellent at one task is cheaper and faster than a frontier model that’s decent at everything.
None of that changes the core mechanic. Tokens in, probabilities out, weights adjusted by gradient descent. The cleverness lives in what gets wrapped around that loop: memory, tools, guardrails, and the human deciding what the output is actually for.