Temperature: start low when you must cite

Temperature is a dial on how freely the model picks the next word. Low is more repeatable. High is more mixed. It is not a creativity switch you turn for fun, and it is not a fix for a bad prompt.

This post is which number to start with, when to move it, and how to prove the move.

What it does

At low temperature, the model keeps picking the likely next token. Same input, same day, you often get the same shape.

At high temperature, it samples more of the long tail. You get variety. You also get extra clauses, new dates, and a different heading each run.

For a tool that must cite a file, variety is a bug.

If you need the same five cases to pass twice, start low.

A starting number

For schema, flags, search queries, and cited bullets: start at 0 or 0.2.

For a title you will still review: 0.3 is plenty.

For a story you asked to be a story: higher, and do not present it as from a document.

Do not start at 0.9 on a notes helper. You will spend the week chasing a Thursday that only appears on some runs.

If the product already fails the bait question at 0, raising temperature will not save it.

Walk through once

Fix temperature at 0. Run the grounded question three times. The bullets should stay close. The cite title should stay the same.

Run the bait question three times. You want three refuses, not two refuses and one Saturday.

Now set 0.8. Run both again, three times each.

If the grounded answer grows a fourth bullet, you paid for noise.

If the bait invents a day on one of three runs, you made the leak intermittent. Intermittent is worse. You will think you fixed it.

Write the six results in a grid. Keep the number that keeps the grid boring.

What good looks like

The step has one number, written next to the model name.

You can rerun the bait three times this afternoon and get three refuses.

A teammate can run the same grid next month and not need you to “feel” the setting.

Production and eval use the same number. A pretty demo at 0.9 does not ship if eval is at 0.

Intermittent is the thing to fear

A bait that fails one time in five is worse than a bait that fails five times in five.

When it always fails, you will fix it. When it sometimes passes, you will ship, then get a Saturday in the wild, then blame the user.

High temperature manufactures intermittent. That is why cited steps stay low even if a stakeholder wants “more life” in the bullets.

If you want life, write two templates in code and pick one. Do not ask the sampler to decorate a rule.

Put the number in the same place you put the model name. If a library defaults to 0.7 and you never set it, you are running 0.7. Print it in the trace. A “we thought it was 0” week is how Saturday comes back.

If a step is a yes/no or a schema, you can often set temperature to 0 and leave it. You will not miss a 0.15. You will miss a forgotten default.

The messy column is not “more creative.” It is a checker you cannot trust.

What temperature will not do

It will not add the file. That is RAG.

It will not add a grounded field. That is schema.

It will not stop a planted line in the PDF. That is injection handling.

It will not make a small model into a large one.

People move the dial because it is one number and it feels like control. The control you need is usually a field and a refuse.

Common mistakes

A different temperature per mood. The eval will not match production.

High temperature plus “be accurate.” You asked for two opposite things.

Tuning it before you have five cases. You will pick the run you liked.

Hiding a flake by rerunning. Log the first run.

Copying a number from a chat screenshot. Re-run your cases.

Practise

1. Set temperature to 0.
2. Run one grounded and one bait, three times each.
3. Set 0.8 and repeat.
4. Keep the setting where the bait never slips.
5. Write the number next to the step on your model sheet.

Leave the knob left for anything that must cite a file.

Checklist

• You know low means repeatable, high means mixed.
• Cited work starts at 0 or 0.2.
• You compared three runs, not one.
• A bait that fails once at high is a reason to stay low.
• The number sits on the step sheet.

The next step is three boring runs at 0, not a hunt for a magic decimal.