Conversation state, system prompts, and the prefill rule

The API has no memory. None. Every call is the first call as far as the server is concerned, and the entire conversation is something you carry in and out yourself.

Stateless is a design decision, not an omission

There is no session, no conversation_id, no thread that lives on Anthropic's side. To continue a conversation you resend it:

{
  "model": "claude-sonnet-5",
  "max_tokens": 1024,
  "system": "You are a terse code reviewer.",
  "messages": [
    { "role": "user", "content": "Review this diff." },
    { "role": "assistant", "content": "Two problems. First..." },
    { "role": "user", "content": "Fix the second one." }
  ]
}

Two consequences fall straight out of this, and both are examinable.

Input tokens grow with the conversation. Turn twenty pays for turns one through nineteen all over again. That is not waste caused by a bug, it is how the protocol works, and it is precisely the bill that prompt caching exists to cut.

Your app owns persistence. If you want conversations to survive a process restart, that is your database, your schema, your problem. The Agent SDK and Managed Agents give you harnesses that do this for you, which is one of the real reasons to pick them over a hand-rolled loop.

The system prompt is configuration, not conversation

Because system is a top-level parameter it sits outside the alternation entirely. It does not count as a turn, it does not need a matching reply, and it is byte-identical on every request in a well-built app.

That last property is why it is the natural front of your cached prefix. Hold that thought until module four.

system also accepts an array of content blocks rather than a bare string, which is how you attach cache control to part of it while leaving the rest uncached.

Prefill: alive in the middle, dead at the end

Assistant prefill was the trick where you end your messages array with a partial assistant message and let the model continue it, forcing a format:

"messages": [
  { "role": "user", "content": "List three risks as JSON." },
  { "role": "assistant", "content": "[" }
]

The replacement for last-turn prefill is structured outputs, which is a real feature with a real schema rather than a hack that leans on the model's completion behaviour. That is module two, lesson three.

Why this changed at all

Prefill worked by exploiting the fact that the model just continues text. Newer models do more between turns, including thinking, and injecting a half-finished assistant turn in front of that is ambiguous at best. Rather than let it silently degrade, the API rejects it loudly.

Loud rejection is a gift. A 400 you can read beats a subtly worse answer you cannot detect, and that framing shows up more than once in this course.