☀️ AI Morning Minute: Non-determinism
Ask an AI the same question twice. Get two different answers. That’s not a bug. It’s how these systems are built.
Most software is predictable. Run the same calculation twice and you get the same result. AI language models don’t work that way. Ask the same question in the same session and you’ll often get a somewhat different response each time: different wording, different examples, sometimes different conclusions.
This property has a name: non-determinism. It’s one of the most misunderstood things about how AI works, and it matters the moment you start building with it.
What it means
Non-determinism in AI means that a model doesn’t always produce the same output for the same input. The reason is a setting called temperature, which controls how much randomness the model injects when choosing its next word. At temperature zero, the model always picks the most statistically likely next word, producing consistent, predictable output. At higher temperatures, it samples from a range of likely options, introducing variation. Most consumer AI products run at a temperature between 0.7 and 1.0, which is why responses feel natural and varied rather than robotic. You can’t guarantee identical output twice, even with an identical prompt.
Why it matters
Non-determinism is a feature in creative work and a liability in production systems. It’s what makes AI writing feel human rather than templated. But if you’re building a system that needs to pass the same test every time, return consistent structured data, or behave predictably in a regulated environment, non-determinism is a real engineering problem you have to design around.
It makes testing AI systems genuinely hard. Traditional software testing assumes the same input produces the same output. With non-deterministic models, a test that passes today might fail tomorrow with the same inputs. Teams building production AI systems often run hundreds of sample outputs to characterize behavior statistically rather than test for exact matches.
Temperature isn’t the only source of non-determinism. Model updates, infrastructure changes, and even parallel processing at the hardware level can all introduce variation. Setting temperature to zero reduces randomness but doesn’t eliminate it entirely. True determinism in a large language model is harder to guarantee than it sounds.
Simple example
A law firm builds an AI tool to draft standard contract clauses. In testing, the output looks great. In production, they notice the same clause prompt generates slightly different language each time: same meaning, different phrasing. For a creative brief that’s fine.
For a legal document where consistency matters, it’s a problem. They switch to temperature zero and add a review step for any output that will go into a signed document. Non-determinism managed, not eliminated.