Tokens and cost: a retry is a second ticket

Every word you send, and every word you get back, is tokens. Tokens are how the bill is counted. A loop that “tries again” is a loop that spends again.

This post is how to see the count, why agents get expensive, and when a smaller model is enough.

What you pay for

You pay for input and output.

Input is the rules, the question, the chunks, the chat you left on the desk. Output is the answer.

A long paste is a large input. A chatty answer is a large output. A graph that calls the model five times is five bills, even if four of them were repairs.

You do not need the exact price list to start. You need a log:

• which step called the model
• how big the prompt was (words is fine)
• how big the reply was
• whether the step passed or failed

If you cannot fill that log, you cannot say why last week cost more.

A retry is not free. It is a second ticket.

Why agents surprise people

A single “explain this paragraph” is one call.

An agent that retrieves, drafts, checks, and repairs twice is five calls. Each call may include the same chunks again. You paid for those chunks five times.

That is why a repair cap is not only a quality rule. It is a money rule.

Empty retrieve should stop. If you draft anyway, you pay for a draft you will throw away.

A crew with three roles is three calls per turn. A reviewer that always says “looks good” is a paid stamp.

A size you can feel

Count words in the prompt you send. Two hundred words is small. Two thousand is getting heavy. Twenty thousand is a stuffing problem, not a model problem.

If the answer is a number or three bullets, ask for that. Do not ask for an essay and then throw it away.

If a small model can fill a schema, use it for that step. Keep the larger model for the one step that needs judgement — or do not, if the schema is the whole job.

Try the small one on your five eval cases. If they still pass, stay small.

If two of five calls are repairs, fix the check. Do not buy a bigger model first.

What good looks like

A cheap run is a short desk and few calls.

The known case: one retrieve, one draft, maybe one check. Two or three calls.

The unknown case: retrieve, then stop. One call, or two if the query rewrite is separate. No draft.

The cap case: retrieve, draft, check, repair, check, stop. Not eight.

If your log does not look like that, you are paying for hope. Cut the extra call before you change models.

How to cut the bill without guessing

1. Print the prompt. Cut what the step does not need.
2. Cap repairs at two.
3. Stop on empty retrieve.
4. Ask for fields, not a page.
5. Do not send the whole chat into every node.

Then look at the log again. If one step is still huge, that step is stuffing. Go back to the context-window rules: facts on the desk, not the file.

Do not turn off eval to save money. A cheap agent that emails the wrong person is not cheap.

Common mistakes

Retry on any error with no cap. That is a runaway bill.

The same 20-page paste in every node. Search once. Pass the three chunks.

A large model for grounded: true|false. That is a yes/no. Use a small model, or your own code.

No log. Then “it got expensive” is a mood.

Streaming a novel the user did not ask for. Output tokens are paid too.

Walk through once

Take the notes helper with five eval cases.

Run them once. For each case write: calls, words in, words out. Add a total.

Look at the unknown case. If it still drafted a paragraph, you paid for a draft you will hide. Stop before draft when retrieve is empty. Run again. The total should drop.

Look at the cap case. If you see four repairs, you paid four times to fail. Set the cap to two. Run again.

If the known case still passes after both cuts, you did not lose the product. You lost waste.

That sheet — five rows, before and after — is the cost review. Do it when someone says “try a bigger model.” Often the sheet says no.

Practise

1. Run your notes helper on the five eval cases.
2. Write down calls, words in, words out.
3. Remove one leftover chat from the prompt. Run again.
4. Cap repairs at two if you have not.
5. Try a smaller model on the schema step. Keep it if the cases still pass.

The second column should be shorter. If it is not, you did not cut.

Checklist

• You can list the model calls for one job.
• A repair has a cap.
• Empty work does not draft.
• You tried a smaller model on a schema step.
• Eval still runs after you cut.

The next step is a shorter desk and a cap, not a higher limit.