Module 1 lab and quiz: the cost is zero, find out why
The official Monitor course puts a lab at the end of every module, hands on in the LangSmith org they provision for you. That is a strong hint about how this domain is examined: it is not a recall subject, it is a “go and look, then reason about what you saw” subject.
What this lab is really training
Three habits, all of which are worth more than any individual number in this module.
Reading the stem for what it has already given you. Every cost question hands you one or two of the three conditions in its setup. Spotting which ones narrows four options to two before you have thought about the content at all.
Distinguishing an absence from a zero. An empty cost cell is not a cost of zero. Free runs do not exist; unpriced runs do. Any tooling you build on top of LangSmith cost data has to treat null and zero as different, or your monthly report will confidently under count.
Checking totals for completeness before believing them. A rollup over incompletely priced children is the single easiest way to be wrong about money while looking rigorous.
The offline version
You are studying on bad wifi and you may not have an org in front of you. The reasoning half of this lab works on paper.
Sketch a five run trace: a root chain, two tool runs, two llm runs. Give each llm run token counts. Give one of them a model name your pricing map knows and the other a self hosted name it does not. Now write the cost figure that appears on each of the five rows, including the root.
Four of those five numbers are easy. The interesting one is the root, and the interesting question is whether anything on the screen would tell you it is short.
Try it yourself
Diagnose from what is visible
Two runs from the same project, same day, same code path. Here is what each one's metadata pane shows.
// Run A: cost column is populated
{ "usage_metadata": { "input_tokens": 1200, "output_tokens": 340 },
"ls_provider": "openai", "ls_model_name": "gpt-4o-mini" }
// Run B: cost column is empty
{ "usage_metadata": { "input_tokens": 900, "output_tokens": 210 },
"ls_provider": "openai", "ls_model_nme": "gpt-4o-mini" }Show answer
Correct answer: B — Condition two, run B's model identity key is misspelled so nothing matches
ls_model_nme is a typo, so condition two fails and no lookup ever happens. The third option is the tempting one because a pricing map miss produces an identical empty cell and is a real, common cause, so it is a sound guess in the absence of evidence. Here there is evidence: both runs name the same model on the same day, so a dated pricing gap would have hit both. Condition one is visibly satisfied, and there is no delayed pricing behaviour to appeal to.
The greedy walk with different numbers
100 input tokens. 40 of them are cached reads priced at 0.5e-6. The general input rate is 3e-6.
A: 100 * 3e-6
B: 40 * 0.5e-6 + 100 * 3e-6
C: 100 * 0.5e-6
D: 40 * 0.5e-6 + 60 * 3e-6Show answer
Correct answer: D — Line D
The specific type is priced first and subtracted, leaving 60 tokens at the general rate, which is line D. Line B is the same double counting trap as before with the numbers changed, and it is worth noticing that it is nearly right: it gets the specific rate and the specific count correct and only fails on the subtraction. That is exactly how the mistake survives a code review, because the wrong line contains all the right ingredients.
The whole cost module in one card
Consolidation card. If you can produce this from memory the module is done, and if you cannot, the gap it exposes tells you which lesson to reread.
State the three conditions for a cost to be computed, the rule for pricing a run with mixed token types, and the two properties of a pricing map entry that explain why an older run can be unpriced while a newer one is not.
Reveal answer
Conditions: token counts in usage_metadata, ls_provider and ls_model_name in metadata, and a matching active entry in the model pricing map. All three, as an AND, failing silently to an empty cell. Mixed token types are priced greedily from most specific to least specific, each type charged at its own rate and subtracted from the remaining total, so every token is billed exactly once. Pricing entries are matched by regex against the model name and carry activation dates, so an entry can match a name and still not apply to a run that predates it.
Which single change makes the cost appear
A self hosted Llama deployment reports tokens correctly and carries ls_provider and ls_model_name, both spelled right. The cost column is empty for every run, old and new.
Show answer
Correct answer: A — Add a custom pricing entry matching that model name, active from a date at or before the runs
Conditions one and two are established by the stem, so the only remaining failure is condition three, and the fix is a custom pricing entry in LangSmith configuration. The renaming option is the genuinely tempting one, because it would technically produce a number, and people do it. It would be a false number, priced at somebody else's rates, which is worse than no number because it is confidently wrong. There is no plan gate on cost tracking; the only plan gated feature in this domain is Insights.
A total that is too low
A trace's headline cost looks suspiciously cheap after a refactor that moved one step onto a new internal model.
Show answer
Correct answer: C — The new internal model is unpriced, so its runs contribute nothing to the rollup and the total is short
A rollup sums the runs that have a price. An unpriced run contributes zero rather than raising an error, so the total is short with no visible symptom. The first option is the one you want to be true and it is exactly the reasoning that lets a bad number ship: the refactor might well be cheaper, but you cannot conclude that from a total whose completeness you have not checked. Tool runs are not priced because they make no model call, which is different from being excluded by design.
Lab: run the cost triage on your own project
The official course does this hands on in a provisioned org. You can do the reasoning half offline in a scratch file and the clicking half in any org you have, in either order.
- Pick one project and open the runs table. Sort by cost descending. Write down the single most expensive run and the model it used.
- Open that run's trace. Find where inside the tree the money actually goes, and write down whether the expensive step is the one you would have guessed.
- Now filter the same table for runs where cost is empty. If there are none, say so; that is a passing result and worth recording.
- For any empty ones, walk the three conditions in order and name which one fails. Write the name of the condition, not just "no cost".
- List the metadata fields present on a typical run. For each of tenant, environment, app version and feature, note whether you could group cost by it today.
- Write one sentence naming the field you most wish had been attached three months ago.