Multimodal messages
Text is one content block type among several. Once you see message content as a list of typed blocks rather than a string, multimodal stops being a separate feature and becomes the same feature with a different type key.
The four message classes
HumanMessage is user input. AIMessage is the model’s response, including its tool calls and metadata. SystemMessage tells the model how to behave. ToolMessage carries the output of a tool call.
That last one is the pairing you met in the tools lesson: every tool call in an AIMessage needs a matching ToolMessage result, keyed by tool_call_id.
Content is a list of blocks
from langchain.messages import HumanMessage
message = HumanMessage(content=[
{"type": "text", "text": "Describe this image"},
{"type": "image", "url": "https://example.com/image.jpg"},
])The block types are text, image, audio, video and file. PDFs are file, which is worth pinning now, because document is the word almost everyone reaches for and it is not the key.
One sourcing pattern, repeated
Every non-text block carries its payload one of three ways, and it is the same three every time.
{"type": "image", "url": "https://example.com/path/to/image.jpg"}
{"type": "image", "base64": "AAAAIGZ0eXBt...", "mime_type": "image/jpeg"}
{"type": "image", "file_id": "file-abc123"}Why it is a list rather than a field
There is a design decision here worth thirty seconds, because it explains a shape you will keep meeting.
A message could have been given a text field and an optional images field. Instead content is an ordered list of typed blocks, which means position carries meaning. “Here is a picture, now here is my question about it” and “here is my question, now here is the picture” are different messages, and a model reads them differently.
The same structure explains AIMessage, which carries text content, tool calls and metadata together. The tool calls are not stored in some parallel channel; they are part of what the model produced, in order, alongside whatever it said. Once you have seen content as an ordered list, the tools lesson’s insistence that every tool call needs a matching ToolMessage stops being a rule to memorise and starts being an obvious consequence.
The honest caveat
Not every model accepts every type. The documentation is explicit that you check the provider’s reference for supported formats and size limits, and it does not publish a matrix.
So the practical rule when something rejects a well-formed block: suspect provider support before you suspect your block. The block schema is small enough to eyeball in five seconds; the support surface is the part that varies underneath you.
Try it yourself
Fill the blank
One key is missing and the block is not valid without it.
{"type": "image", "base64": "AAAAIGZ0eXBt...", "___": "image/jpeg"}Show answer
Correct answer: C — mime_type
The key is mime_type. media_type is the genuinely tempting distractor because it is the key Anthropic's own raw API uses for the same job, so anyone who has written against that API directly will reach for it by muscle memory. In LangChain content blocks the key is mime_type, and it is required alongside base64 because the bytes alone do not say what they are.
Three ways to supply the same file
The block types vary, but the sourcing pattern underneath them does not, which is the thing that makes this small enough to memorise.
What are the three ways a non-text content block can carry its payload, and which of them needs an extra key?
Reveal answer
A url, a base64 string, or a provider file_id. The base64 form additionally requires mime_type, because raw bytes carry no type information; url and file_id do not, because the type is resolved on the other side. The same three forms apply across image, audio, video and file blocks, so learning it once covers all four.
The block type for a PDF
You are passing an invoice PDF to a model that supports document input. Which block type carries it?
Show answer
Correct answer: A — {"type": "file", ...}
The block type is file, and PDF is the documented example of it. document is the trap: it is the word most people reach for, and it is the term several provider APIs use natively, but the LangChain content block is named file. The last option describes a real workaround from before native document support and is now unnecessary work.
Where support is decided
Your image block is correctly formed and the model returns an error about unsupported content. What is the most likely cause?
Show answer
Correct answer: B — The provider or model does not accept that content type or format
Support varies by provider and by model, and the docs are explicit that you check the provider reference for accepted formats and size limits. The third option is the interesting distractor because it is half true: content is a list of blocks, and a single bare dict is a real mistake, but a correctly formed block by definition already satisfies that.
Name the four message classes
Write the four message classes down with one clause each on what each represents.
You listed HumanMessage, AIMessage, SystemMessage and ToolMessage, and you attached tool outputs to ToolMessage rather than to AIMessage.