Tracing LangChain code, and wrapping raw SDK calls

Every agent you have is judged by vibes: "vault-tidy seems fine", "chronicler's classification looks right". This module turns that into a number. You trace the module 1 classifier and module 2 agent, build a dataset from real outputs, run an eval, and score vault-tidy's frontmatter fixes on a real sample. Quality stops being a feeling.

Tracing is free instrumentation. Set two env vars and every LangChain / LangGraph call in the process reports to LangSmith: inputs, outputs, latency, token counts, the full call tree.

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=ls-…

No code change for anything built on LangChain. You get run trees in the LangSmith UI immediately.

To trace non-LangChain code, wrap it with traceable.

import { traceable } from "langsmith/traceable";

const classifyDay = traceable(
  async (messages) => { /* … */ },
  { name: "classify-day" },
);

Code that calls the Anthropic SDK directly, like Ark's SdkSession, is not LangChain, so tracing does not see it automatically. Wrap the client.

import { wrapAnthropic } from "langsmith/wrappers/anthropic";
import Anthropic from "@anthropic-ai/sdk";

const client = wrapAnthropic(new Anthropic());
// every client.messages.create(...) now traces