How Many Guesses Does It Take to Crack an AI’s “Random” Number?

Open a chatbot, ask it for a random number between 1 and 100, and write down what you get. Do it a few more times in fresh chats.

A fair generator would land on each number about once in a hundred tries. Language models don’t, and plenty of people have already shown that. The write-ups I came across mostly count how often each number turns up. I wanted the security version of the question: if a model picks the number that protects something, how many guesses does an attacker need?

This article puts a number on it. For each model I read the probability assigned to every answer from 1 to 100 straight from its logits, so there is no sampling noise and no histogram of a thousand runs. From those 100 probabilities I compute bits of randomness and the expected number of guesses to crack the pick. I do it for base models and their instruction-tuned versions side by side, to see whether the habit comes from pretraining or from what happens after it.

The short version: with a plain-text prompt, a 1.5-billion-parameter instruction-tuned model I tested answers 47 about three times in four, and with a reworded prompt it still picks 47 more than a third of the time.

What is already known, and what I wanted to add

The favorite-number habit is well documented. The most thorough sampling study I found tested six models in seven languages, at six temperatures and three ranges (1 to 5, 1 to 10, 1 to 100), 75,600 calls in all, and reports that the models keep repeating a few values [1]. For 1 to 100 the favorites it lists are 37, 47 and 73, with 42 as the odd one out because of an obvious literary explanation [1]. Its reading is that the models inherit human biases, and it cites a 200,000-person poll in which people also lean on numbers containing a 7 [1]. A separate study of binary coin-flip sequences found that GPT-4 and Llama 3 exhibit and exaggerate nearly every human bias the authors tested, while GPT-3.5 behaved more randomly [2].

The comparison between a base model and its tuned successor has also been made informally, for older models. In 2022 janus plotted the probability of every number from 0 to 100 for OpenAI’s base davinci model and for text-davinci-002 [3]. The base model’s distribution was fairly uniform with a slight preference for 42, while the tuned model had a much more pronounced favorite, 97. The post was later corrected: text-davinci-002 was probably not trained with RLHF but with a similar method that OpenAI had not described [3].

Why would tuning matter at all? The explanation “it inherited this from human text” predicts that base models, which have only seen that text, should show the habit too. Post-training could change the picture: a controlled study of the fine-tuning stages found that RLHF significantly reduces output diversity compared with supervised fine-tuning [4].

Three things seemed worth adding.

The first is access. Those OpenAI models were closed, so nobody could rerun the experiment or inspect the recipe, and the correction shows that even the recipe was misunderstood at the time. With open checkpoints anyone can repeat the readout, and OLMo 2 publishes every post-training stage, so the pick can be followed through each one.

The second is robustness. A favorite number is only interesting if it survives a change of prompt. I read the distributions for five open model families under two prompt formats, a plain-text prompt shared by base and tuned models and each model’s own chat template, and I reran two families with a reworded prompt.

The third is risk. A histogram of favorites says little about how guessable a secret is. Security people have a precise vocabulary for that, and I wanted to use it. The next section sets it up.

Three ways to measure how random a pick is

A fair pick from 1 to 100 gives every number probability 0.01. A real model assigns unequal probabilities to the hundred numbers, and there are three reasonable ways to say how far that is from fair. They answer different questions and can disagree, so I report all three.

Shannon entropy is the average surprise of the answer, in bits:

A fair pick gives 6.64 bits, which is the base-2 logarithm of 100. Raising 2 to the power of the entropy turns it into a head count: the model behaves, on average, like a fair pick among that many numbers.

Min-entropy looks only at the favorite:

It measures protection against an attacker who gets one guess and spends it on the model’s favorite. A fair pick again gives 6.64 bits.

Guessing entropy, introduced by Massey in 1994 [5], is the expected number of guesses when the attacker tries numbers from most likely to least likely:

A fair pick gives 50.5 guesses.

A toy case shows why one number is not enough. Suppose a model answers 73 half the time and otherwise picks uniformly among the other 99 numbers. Its Shannon entropy is 4.31 bits, which sounds respectable and equals about 20 fair choices. Its min-entropy is exactly 1 bit, because a single guess of 73 succeeds half the time. Its guessing entropy is 26.0 guesses, roughly half the fair 50.5. One distribution, three different stories.

Shannon entropy does bound guessing from below. Massey showed that when the entropy H is at least 2 bits, the expected number of guesses satisfies [5]:

For the toy case that floor is 5.97 guesses, far under the true 26.0, so entropy alone would have been a poor guide to the risk.

All three measures assume the attacker knows the distribution and guesses in the best order. That is generous to the attacker, but anyone can load the same open model and read the same probabilities, as the next section does.

Reading the answer straight from the logits

Sampling a model many times only estimates its answer distribution. A language model hands you that distribution directly, because at every step it produces a probability for each possible next token. The probability of a whole answer is the product of the probabilities of its tokens.

For the answer 73, the model has to produce the tokens for 7 and 3 (or a single token for 73, depending on the tokenizer) and then stop. The stop matters. Without it, the probability of 7 would also count every answer that starts with a 7. So for each candidate n from 1 to 100:

Here “stop” means an end-of-turn token, a newline or a period. I score all 100 candidates and rescale them to sum to 1. The total before rescaling, which I call coverage, says how often the model answers with a bare number at all. If coverage is low, everything below describes what the model picks when it does answer with a number, and not its behavior in general.

The core of the scoring code is short. This is a trimmed version of what I ran, where logprob_fn returns the model’s next-token log-probabilities after each position:

import numpy as npdef score_candidates(logprob_fn, prompt_ids, candidates, stop_ids):    # log P(answer, then a stop token | prompt) for every candidate answer    n = len(prompt_ids)    out = []    for cand in candidates:                    # token ids of '1', '2', ... '100'        ids = np.array([prompt_ids + cand])        lp = logprob_fn(ids, n - 1)[0]         # log-probs after positions n-1, n, ...        answer_lp = sum(lp[k, tok] for k, tok in enumerate(cand))        stop_lp = np.logaddexp.reduce(lp[len(cand), stop_ids])        out.append(answer_lp + stop_lp)    return np.array(out)logp = score_candidates(logprob_fn, prompt_ids, candidates, stop_ids)p = np.exp(logp - np.logaddexp.reduce(logp))          # rescaled to sum to 1coverage = np.exp(np.logaddexp.reduce(logp))          # mass on a bare numberps = np.sort(p)[::-1]shannon = -(p[p > 0] * np.log2(p[p > 0])).sum()       # bitsmin_entropy = -np.log2(ps[0])                         # bitsguesses = (np.arange(1, 101) * ps).sum()              # expected guesses

I ran two prompt modes. In the raw mode the prompt is plain text, “Pick a random number between 1 and 100. Reply with just the number.”, identical for base and instruction-tuned models, which keeps that comparison fair. In the chat mode the same sentence goes through the instruction-tuned model’s own chat template, which is how people actually use it. Base models have no chat format, so they only get the raw mode.

The models are five families, each as a base model and its instruction-tuned version: Qwen2.5 at 0.5B and 1.5B parameters, SmolLM2 at 360M and 1.7B, and OLMo 2 at 1B. OLMo 2 publishes every post-training stage, so for it I also scored the SFT and DPO checkpoints, which gives base, SFT, DPO and the final Instruct model [6]. Qwen2.5’s report describes its post-training as supervised fine-tuning followed by offline DPO and online GRPO [7]; I did not look up SmolLM2’s recipe. Everything ran in float32 on a Colab GPU, 19 model and prompt combinations in all. They are small because the experiment has to run in a notebook. The exact checkpoints are Qwen/Qwen2.5-0.5B and Qwen/Qwen2.5-1.5B with their -Instruct versions, HuggingFaceTB/SmolLM2-360M and HuggingFaceTB/SmolLM2-1.7B with their -Instruct versions, and allenai/OLMo-2-0425-1B with its -SFT, -DPO and -Instruct versions.

One rule runs through everything below. A row is only as trustworthy as its coverage, so I base every claim on rows where at least half of the probability sits on a bare number, and I say so wherever a row falls short.

Before trusting any real model, I tested the scoring code on a toy model whose answer distribution I chose myself, with two kinds of tokenizer: one digit per token, and runs of up to three digits per token. The code recovered the distribution to within floating-point rounding. The real models cover both tokenizer cases: Qwen2.5 and SmolLM2 split 73 into the tokens 7 and 3, while OLMo 2 has a single token for 73.

How random is the pick?

A fair pick scores 6.64 bits and takes 50.5 guesses on average. Table 1 shows where the models landed, for the rows where at least half of the probability sits on a bare number. “Raw” is the plain-text prompt and “chat” is the model’s own chat template.

Table 1. Randomness of the 1 to 100 pick, read from the logits. Rows with under 50% coverage are left out; the OLMo SFT and DPO checkpoints are in Table 2.

The extreme case is Qwen2.5-1.5B-Instruct with the raw prompt. It puts 74% of its probability on 47 and 94% on its top three numbers. An attacker who guesses 47 first is right three times in four and needs 1.9 guesses on average, against 50.5 for a fair pick, and the model behaves like a fair pick among two or three numbers. Through its chat template the same model is less extreme but still narrow: 4.12 bits, 73 as the favorite at 29%, and 7.9 guesses.

The other instruction-tuned rows are milder but point the same way. The final OLMo 2 1B Instruct sits at 4.82 bits and 11.6 to 12.1 guesses with either prompt format, and the Qwen2.5-0.5B-Instruct rows are at 5.40 and 4.84 bits. The exception is SmolLM2-1.7B-Instruct through its chat template, at 6.26 bits and 30.7 guesses, close to the base models and with no favorite above 3.4%. Its raw-prompt row puts only 11% of its probability on a bare number, so I cannot say whether post-training narrowed it.

Across the 13 rows with good coverage (Table 1 and Table 2), 42 is the top pick in six, 50 and 73 in two each, and 25, 47 and 99 in one each. Of the three favorites the sampling study reports for 1 to 100 [1], 47 and 73 show up as top picks here and 37 never does, although it sits among the bright squares in Figure 2.

Figure 2 shows where the probability sits for the model with the most extreme pick. Each square is one number, from 1 at the top left to 100 at the bottom right, colored by how many times likelier it is than in a fair pick, on one scale for every panel.

Six of the 19 rows fall short of the half-coverage rule and are left out of Table 1: Qwen2.5-0.5B base (49.9%), the SmolLM2-360M base, raw and chat rows (28%, 1.6% and 34%), and the SmolLM2-1.7B base and raw rows (38% and 11%). For those rows the numbers describe only the cases where the model answers with a bare number, and for the SmolLM2 raw rows that is a sliver. No claim in this article depends on them.

Where the narrowing happens

An instruction-tuned model is a base model plus post-training. With the same raw prompt, any difference between the two therefore comes from post-training, and any bias the base model already has was there before it.

The base models are not fair. Qwen2.5-1.5B’s base row already needs only 26.2 guesses on average and puts 10% on 42, while OLMo 2’s base is the flattest I measured, at 6.57 bits and 41.5 guesses. Qwen2.5-0.5B’s base row, at 6.11 bits, misses the coverage rule by a fraction of a point.

Instruction tuning then narrows the pick with the raw prompt, in every family I can count. Qwen2.5-1.5B falls from 5.94 to 1.33 bits, a drop of 4.61, and OLMo 2 from 6.57 to 4.82, a drop of 1.75. Qwen2.5-0.5B falls from 6.11 to 5.40, a drop of 0.71, though its base row sits just under the coverage rule. The SmolLM2 pairs point the same way but have too little coverage to count. It is also the direction janus saw for OpenAI’s models in 2022 [3].

OLMo 2 1B lets me ask which step does it, because every stage is published as its own checkpoint: the base model, after SFT, after DPO, and the final Instruct model, which per its model card adds a last reinforcement-learning stage (RLVR) [6]. Table 2 and Figure 3 follow the pick through them.

Table 2. OLMo 2 1B after each published post-training stage. Every row has at least 71% coverage. The base model has no chat format.

With the raw prompt, entropy falls by 0.76 bits at SFT, 0.62 at DPO and 0.37 at the final stage, and the expected guesses drop from 41.5 to 22.2, then 14.4, then 11.6. SFT alone accounts for more than 40% of the total drop in bits. Through the chat template the first post-training stage already reaches 5.02 bits, and the later stages add little (4.94, then 4.82). In both formats no later stage undoes the narrowing, which fits the report that preference-based tuning reduces output diversity compared with SFT [4].

The favorites change along the way. The base model has no number above 2.1%. After SFT, 50 holds 9% and 47 holds 6%. After DPO, 42 and 47 share the top at 11% and 10%. The final model puts 14% on 50 and 12% on 47.

The prompt format matters too, and not in a consistent direction. Moving the instruction-tuned models from the raw prompt to their chat template raises entropy by 2.79 bits for Qwen2.5-1.5B (1.33 to 4.12), lowers it by 0.56 bits for Qwen2.5-0.5B (5.40 to 4.84), and leaves OLMo 2 at 4.82 either way. For these three well-covered families the favorite changes every time: 47 to 73, 25 to 99, and 50 to 42.

How much the wording matters

I reran Qwen2.5-0.5B and Qwen2.5-1.5B with a different sentence: “Give me a random integer from 1 to 100. Just the number.” The base-to-instruct narrowing survives. Qwen2.5-1.5B goes from 6.19 bits (base) to 3.44 (instruct, raw prompt) and 2.67 (chat template). Qwen2.5-0.5B goes from 6.07 to 5.77 and 2.71, but under this wording its base and chat rows have only 38% and 45% coverage, so lean on the 1.5B figures.

The details move much more. Qwen2.5-1.5B-Instruct’s raw-prompt entropy rises from 1.33 to 3.44 bits, with 47 still on top but at 35.5% instead of 74%. Qwen2.5-0.5B-Instruct’s chat entropy falls from 4.84 to 2.71 bits as its favorite changes from 99 to 10, which then holds 63% (on that 45% coverage). Across the six reworded rows the entropy shifted by 1.06 bits on average and by 2.13 at most, in both directions. A favorite number belongs to a model and a wording together. The narrowing is the robust part.

Base models look alike, post-trained ones do not

The sampling study noticed that a model keeps some values of its own across prompt languages, which the author calls fingerprint values [1]. With full distributions I can ask how similar two models’ picks are, using the Jensen-Shannon distance between their 100-number distributions: 0 for identical, 1 for no overlap. Figure 4 clusters the well-covered rows by it.

The clusters follow training stage and prompt format more than family. The two well-covered base models, OLMo 2’s and Qwen2.5-1.5B’s, are 0.36 apart. Instruction-tuned rows from different families are much further apart, at an average distance of 0.74 with the raw prompt and 0.74 with the chat template. Inside OLMo 2 the post-trained checkpoints group by prompt format first: the raw-prompt rows cluster together, and so do the chat rows. Qwen2.5-1.5B’s instruction-tuned rows and Qwen2.5-0.5B’s chat row each stand far from everything else.

One reading is that pretraining on shared human text gives different models similar habits, and post-training pushes each family toward favorites of its own. The evidence is thin, because only one base pair qualifies. Including all 19 rows, the base models average 0.31 apart against 0.69 and 0.77 for the instruction-tuned ones, which points the same way but leans on the low-coverage rows.

What to do instead

In the chat-template rows with good coverage, the instruction-tuned models gave between 4.12 and 6.26 bits and between 7.9 and 30.7 expected guesses, against 6.64 bits and 50.5 guesses for a fair pick. With the raw prompt, Qwen2.5-1.5B-Instruct needed only 1.9. In plain words, the Qwen2.5-1.5B chat model’s 7.9 guesses is what a fair pick among about 15 numbers would take. Whenever a number from a model has to be unpredictable, that gap is the risk: a PIN, a test key, a lottery pick, a sample drawn for a statistical test.

Temperature is not a reliable fix. In the sampling study, raising it helped some models and did nothing for others [1]. Rewording the prompt is not one either: in my check it moved the entropy by 1.06 bits on average and by up to 2.13, in both directions, and it changed favorites.

The fix is to stop asking the model to be the source of randomness. Let a proper generator do the drawing, for example Python’s secrets module when the number protects something, and let the model write the code or call the tool. Language models are good at deciding that a random number is needed and where it goes. Producing the number is the part they should hand off.

What this does not show

• The models are small. They run in a notebook, so they are far smaller than the chat products most people use. The sampling study left out models below about 5 billion parameters because its first tests with 2B and 3B models showed trouble understanding the task consistently [1]. I only tested open models and make no claim about hosted assistants.

• The numbers are conditional on answering with a number. Coverage says how often that happens, and six of the 19 rows fall under half. The SmolLM2 results are mostly out of the analysis for that reason, which leaves Qwen2.5-1.5B and OLMo 2 as the clean base-to-instruct pairs, with Qwen2.5-0.5B just short.

• Wording and prompt format matter a lot. The headline numbers come from one sentence. A reworded sentence on the two Qwen2.5 families moved entropy by up to 2.1 bits and changed favorites. I did not test other ranges or other languages, which change favorites too [1].

• Five families, one run each. The readout is exact for a given model and prompt, so there is no sampling error, but five families are too few to say how general the narrowing is. For Qwen2.5 and SmolLM2 I only scored the released instruction-tuned model, so only OLMo 2 shows the stages.

• The attacker is idealized. The guessing numbers assume someone who knows the model’s exact distribution and guesses in the best order. Real attackers are usually worse informed, though anyone can load the same open model.

• Only standard tokenizations are scored. Each answer is scored as the tokenizer would normally split it. Unusual splits of the same digits get no credit, which should matter little for the models here but is not zero.

  ···

References

[1] J. Coronado-Blázquez, Deterministic or probabilistic? The psychology of LLMs as random number generators (2025), arXiv:2502.19965

[2] K. Van Koevering, J. Kleinberg, How Random is Random? Evaluating the Randomness and Humaness of LLMs’ Coin Flips (2024), arXiv:2406.00092

[3] janus, Mysteries of mode collapse (2022), LessWrong

[4] R. Kirk, I. Mediratta, C. Nalmpantis, J. Luketina, E. Hambro, E. Grefenstette, R. Raileanu, Understanding the Effects of RLHF on LLM Generalisation and Diversity (2024), ICLR 2024, arXiv:2310.06452

[5] O. Rioul, Variations on a Theme by Massey (2022), IEEE Transactions on Information Theory 68(5), 2813-2828

[6] Allen Institute for AI, Model Card for OLMo 2 1B (2025), Hugging Face, allenai/OLMo-2-0425-1B

[7] A. Yang et al., Qwen2.5 Technical Report (2024), arXiv:2412.15115