AI Determinism: Why It Gives Different Answers

Ask an AI tool the same question twice and it will probably answer in two different ways. Try the same thing with your accounting spreadsheet: 2 + 2 always gives 4, today, tomorrow and a year from now. That difference has a name, determinism, and it decides which parts of your business generative AI fits into and which ones will land you in trouble.

This is not a bug someone will fix in the next release. It is how the technology works under the hood. And understanding it, even if you are not technical, saves you from putting AI where it should not be.

What does it mean for software to be “deterministic”?

A deterministic system always produces the exact same output for the same input. You feed in the same data, you get the same result, every time, no surprises.

It is what you expect from almost all the software your company uses. The calculator, the payroll, the program that issues invoices, the VAT calculation. If your invoicing program returned a different total every time you opened it with the same line items, you would throw it out that same morning. That reliability is the foundation on which you build auditable processes: if something goes wrong, you reproduce the input, you look at the output and you find the error.

Traditional software runs on fixed rules written by a person. “If the amount exceeds one thousand euros, apply this discount.” The rule always holds the same way because someone programmed it that way. That is the logic you have used for decades, and the one I cover in more detail in how AI differs from traditional software.

Generative AI does not play by those rules.

Why does AI give different answers to the same question?

Generative AI does not follow fixed rules: it predicts. An LLM (a language model, the engine behind tools like ChatGPT or Claude) generates text by guessing which word is most likely to come next, then the next one, and so on until it completes the answer. Technically it does not work with whole words but with tokens, which are chunks of a word, but the idea for you is the same: it keeps picking the next piece based on probabilities.

The important detail is in each step: there is no single “correct” word, there are several candidates with different probabilities. The model does not always pick the most likely one. It leaves itself a deliberate margin of choice, controlled by a setting called temperature. With high temperature, the model takes more risks and produces more varied and creative answers. With low temperature, it sticks to what is most likely and sounds flatter.

That variation is there on purpose. It is what makes AI write an email with nuance instead of spitting out the same robotic sentence every time. The price of that richness is that you lose repeatability. If you want to understand the full mechanism underneath, I break it down in how generative AI works under the hood.

	Traditional software	Generative AI
How it decides	Fixed rules written by a person	Predicts the next word by probability
Same input	Same output, always	Can give different outputs
Auditable?	Yes, you reproduce the calculation	Hard: the output changes
Fits in	Invoicing, payroll, legal calculations	Drafts, summaries, support, ideas

Can you make AI repeatable?

You can get fairly close, but you cannot take repeatability for granted. By setting the temperature to its lowest value, the model almost always picks the most likely word and the answers look very similar to each other. For many uses, that is enough.

The problem is everything else moving around it. The provider updates the model to a new version and the same question starts answering differently. You change a comma in your instruction and the answer drifts. The system carries the context of earlier messages and that influences it too. None of those pieces is fully under your control.

So the useful question is not “can I make it deterministic?”. The useful question is “does my process withstand the answer changing now and then?”. And that answer depends on the process, not on the AI.

Why does it matter to you as a decision-maker?

It matters because your processes are not all the same: some demand the exact same result and others live comfortably with variation. Putting AI in the wrong group is where companies get hurt.

Think of two columns. On the left, what demands determinism: the invoice you send to a client, your employees’ payroll, the accounting entry, a legal deadline calculation, a bank account number. There, a different result means an error that ends up at the tax office, in a complaint or in a wrong payment, not a creative variant.

On the right, what tolerates variation: the first draft of a reply to a customer, the summary of a long meeting, three headline ideas for a campaign, a translation that someone reviews afterwards. If AI drafts it in two different ways, neither of them is “wrong”. It works as a starting point.

Processes that demand determinism	Processes that tolerate variation
Issuing invoices and calculating taxes	Drafting an email
Payroll and payments	Summarizing a long document
Accounting entries	Generating ideas for a campaign
Legal calculations and deadlines	First draft of customer support

The decision rule is short: if the process needs the answer to always be exactly the same and auditable, generative AI should not have the final say. If the process allows a draft that a person reviews or that does not matter if it varies, AI genuinely saves you time. That criterion, applied case by case to your company, is exactly what we work on in the AI without hype course.

One new concept every week

Subscribe

The mistakes I see decision-makers make

Almost every stumble here comes from treating AI as if it were traditional software. Four that keep coming up:

Putting AI in charge of calculating the invoice and “skimming the review”. A quick review easily misses a changed number, because the text looks fine. The form is flawless even when the figure is wrong. If the calculation has to be exact, a fixed rule does the calculation, and AI at most helps you write the message that goes with it.

Believing that if it worked in the demo it will work the same in production. The demo went well because you asked the question once and it came out nicely. In production that same question runs hundreds of times with different data, and that is where the answers you never saw show up. A test with a single case tells you nothing about repeatability.

Confusing “it gets things wrong sometimes” with “it gives different answers”. These are two different problems. That the answer changes is the non-determinism this article is about. That the answer sounds convincing but is false is another thing, hallucination, which I cover separately in what an AI hallucination is. You can have a stable, wrong answer, or a correct answer that changes every time. Confusing them leads to “fixing” the problem you did not have.

Asking to “remove all the randomness” and feeling safe. Lowering the variation helps, but it does not turn AI into a calculator. You still have version changes, sensitivity to how you ask and context in the mix. Treating it as if it were 100% reliable after that adjustment trades a visible risk for a hidden one.

Frequently asked questions

Is non-determinism the same as an AI hallucination?

No. Non-determinism is the AI giving you different answers to the same question. A hallucination is it giving you an answer that sounds believable but is incorrect. You can have one without the other: an answer that changes every time but is always correct, or a stable answer that always repeats the same error.

Does setting the temperature to zero fix determinism in AI?

It helps a lot, but it does not guarantee it fully. With the temperature at its minimum the model almost always picks the most likely word, so the answers look fairly alike. Even so, a change in the model version by the provider, or a shift in how you phrase the question, can alter the result.

Can I use AI for invoicing if a human reviews it afterwards?

For the calculation itself, better not to. Human review catches gross errors, but a changed amount in a well-written text slips through easily. The safe combination is the usual one: a deterministic system does the number and AI, if anything, writes the text that goes with that number.

Why did the same question give me one answer yesterday and another today?

For several reasons at once. AI adds deliberate variation to each answer, the provider may have updated the model between yesterday and today, and even a small change in how you wrote the question has an effect. None of that means something has broken. It is the normal behavior of this technology.

Will non-determinism be fixed in the future?

Non-determinism is part of the design of how these models generate text; no one has it queued up to fix. The variation is exactly what lets them write with nuance instead of repeating identical sentences. The sensible thing is not to wait for it to disappear, but to choose carefully in which processes you take advantage of it and in which ones you keep it out.