Three weeks ago I started logging every prompt I sent to Claude Pro in a spreadsheet. Not for science, mostly because a client asked me to justify the subscription on their invoice and I didn’t want to answer with a shrug. By the end I had 214 entries, an accuracy score for each one, and a much clearer picture of where this model is genuinely reliable and where it quietly invents things.
What follows is the honest version, including the two hours a fabricated citation cost me and the handful of tasks where Claude Pro has been noticeably better than the tools I was paying for before.
What “accuracy” actually means with Claude Pro
Accuracy gets thrown around as if it’s a single dial. In practice I ended up scoring three separate behaviours, because they behave nothing alike:
- Factual accuracy — whether a claim survives a two-minute check against a primary source
- Instruction accuracy — whether it did what I asked, including word counts, formatting rules and banned words
- Consistency — whether the same prompt two days later produces the same substance
Claude Pro scores very differently on all three. That gap is basically the whole review in miniature.
How I actually tested it
I split 214 prompts across six categories over 19 working days: 41 client emails and follow-ups, 33 code and script fixes, 28 spreadsheet formulas, 36 long-document summaries, 44 factual lookups, and 32 tone rewrites. Each response got a score from 1 to 5, with a one-line note on what went wrong when it did. I spot-checked five facts per summary and every number in every formula.
I ran a control batch of the same prompts through the free tier to see whether Pro’s higher limits changed answer quality. Worth knowing up front: they mostly don’t. Paying mainly buys you more room before you hit a wall rather than a smarter model, which shapes how you should read every accuracy figure below.
Where Claude Pro really does hold up
The average score across all 214 prompts landed at 3.9 out of 5. That number hides a lot, because the results clustered hard at the ends.
Long documents
Of the 36 summaries of PDFs between 20 and 60 pages, 33 were accurate on the specifics I checked. It handled tables, footnotes and legal sub-clauses without dropping the thread, and when I asked it to quote a clause verbatim it pulled the exact wording 9 times out of 10. That is the strongest part of the product by some distance.
Fussy formatting rules
I gave it a 90-word product description brief with no exclamation marks, a banned-words list and a required call to action at the end. It met every constraint on 19 of 20 attempts. Small thing, but it saves a surprising amount of editing and it is where cheaper tools fall over.
Code and spreadsheet formulas
About seven in ten formulas worked first time with no edits. The three failures were all cases where I’d described the sheet layout badly rather than cases where Claude misunderstood it. When I rewrote the prompt with actual column letters, it fixed each one on the first retry.
Admitting uncertainty
This one surprised me. Asked about a niche UK planning regulation from 2019, it said plainly that it wasn’t confident and suggested where to verify rather than guessing. It did that maybe 14 times across the three weeks. That restraint is worth more than raw speed.
Where accuracy drops off, sometimes badly
Anything recent or genuinely obscure
Every factual miss in my log came from one of two places: events after its knowledge cut-off, or narrow local details. It knew the wrong council name for a Birmingham licensing query and confidently repeated a 2022 fee that had changed since.
Numbers with moving parts
Straight arithmetic was fine. Multi-step estimates, like projecting a monthly cost across tiered usage bands, were wrong roughly one time in four, usually by a small margin that would slip past a casual reader. Always rebuild the maths yourself.
Citations and quotes
This is the real weakness. I asked for academic sources supporting a claim about remote-work productivity. It produced four papers with authors, journals and years. Two existed. Two were plausible-sounding composites of real researchers. I spent two hours chasing one before the DOI failed. The lesson is not that Claude lies, it’s that its citation confidence runs well ahead of its citation accuracy.
Claude Pro accuracy against the alternatives you’d pay for
Set against the field, Claude Pro sits near the top for document work and instruction-following, and mid-pack on live facts where anything with search built in will beat it. If you’re weighing up several subscriptions at once, this comparison of paid AI tools on accuracy alone is a fairer benchmark than any single-model review, including mine.
One pattern from that wider field holds here too: accuracy falls off a cliff not when the model is weak, but when the task is vague. My 1s and 2s were almost all prompts where I’d left the goal implicit.
Does the accuracy survive contact with business work?
Mostly, yes, with a caveat about plumbing. For drafting, summarising and rewriting, the error rate is low enough that a human review pass catches nearly everything. For anything customer-facing, the runs where I fed it a messy shared inbox and asked for replies needed more checking, which is why a grounded look at AI tools in customer support matters more than the model’s headline score.
The other thing I’d flag for teams: feed Claude Pro clean context and accuracy jumps. Feed it scattered Google Drive folders and shared drives and it will confidently summarise the wrong version of a document. If your data is genuinely messy, the fix is on the retrieval side, and building agents that read your company’s messy data properly will do more for accuracy than upgrading your plan ever will.
Five habits that close the accuracy gap
By week three my average score had crept from 3.6 to 4.2 without the model changing at all. These are the changes I made:
- State the audience, the length and one hard constraint in the prompt, every time
- Paste the source text instead of asking it to recall, even for things it should know
- Ask for a confidence line: “flag anything you’re under 80% sure about”
- Run every number, citation and date through a separate check before it leaves my desk
- Reuse good prompts as templates rather than retyping from memory
The unglamorous takeaway from 214 logged prompts is that Claude Pro is accurate enough to trust with a first draft and nowhere near accurate enough to trust with a final one. It summarises a 40-page contract beautifully, writes clean formulas, and holds a formatting brief like a pro. It will also hand you a convincing citation for a paper that doesn’t exist, so keep the verification step where it belongs: between the model’s output and your name on the work. If you’re costing that time against the fee, a value-for-money breakdown of business AI tools is the more useful exercise than another benchmark chart.