# AI Observability: Monitor & Trace Agents and LLMs > Monitor and trace AI agents, LLM calls, and tool executions in production. Catch errors, latency, token costs, and budget overruns with agent tracing and full trace context **URL:** https://sentry.io/solutions/ai-observability/ --- Agents, LLMs, vector stores, custom logic: visibility can’t stop at the model call. Trace every agent run end to end and get the context you need to debug failures, optimize performance, and keep AI features reliable. ## Supported SDKs - Vercel AI — [Docs](https://docs.sentry.io/platforms/javascript/guides/node/agent-tracing/vercelai/) - OpenAI — [Docs](https://docs.sentry.io/platforms/python/agent-tracing/openai/) - OpenAI Agents — [Docs](https://docs.sentry.io/platforms/python/agent-tracing/openai-agents/) - Anthropic — [Docs](https://docs.sentry.io/platforms/python/agent-tracing/anthropic/) - Google Gen AI — [Docs](https://docs.sentry.io/platforms/python/agent-tracing/google-genai/) - LangChain — [Docs](https://docs.sentry.io/platforms/python/agent-tracing/langchain/) - LangGraph — [Docs](https://docs.sentry.io/platforms/python/agent-tracing/langgraph/) - Mastra — [Docs](https://docs.sentry.io/platforms/javascript/guides/node/agent-tracing/mastra/) - Pydantic AI — [Docs](https://docs.sentry.io/platforms/python/agent-tracing/pydantic-ai/) - LiteLLM — [Docs](https://docs.sentry.io/platforms/python/agent-tracing/litellm/) - Hugging Face — [Docs](https://docs.sentry.io/platforms/python/agent-tracing/huggingface_hub/) - Workers AI — [Docs](https://docs.sentry.io/platforms/javascript/guides/cloudflare/agent-tracing/workers-ai/) - Cloudflare Agents — [Docs](https://docs.sentry.io/platforms/javascript/guides/cloudflare/agent-tracing/agents-sdk/) - Eve — [Docs](https://docs.sentry.io/platforms/javascript/guides/eve/) - Flue — [Docs](https://docs.sentry.io/platforms/javascript/guides/node/agent-tracing/flue/) - Laravel AI — [Docs](https://docs.sentry.io/platforms/php/guides/laravel/agent-tracing/) ## Details ### Full LLM Observability #### See everything on one pane of glass Track all agent runs, error rates, LLM calls, tokens used, and tool executions. Monitor traffic patterns and duration metrics across your AI-powered features. ### Track Model Costs & Tokens #### Monitor spending across models. Compare costs across different models. See token usage breakdown by model, track input vs output tokens, and identify expensive operations. ### Agent tracing, end to end #### Trace every agent run, end to end. AI agents fail across a chain of steps, not a single line of code. [Agent tracing](https://sentry.io/product/tracing/ai-agent/) stitches every LLM call, tool execution, and handoff into one connected trace, with full prompt and response context, token counts, cost, and timing on every span. Follow the trace to the exact tool or step that went wrong, and apply agent tracing best practices to keep runs reliable as they scale. ## Cookbook recipes ### Monitor your AI agent costs and calls in Next.js Add Sentry's Vercel AI integration to your Next.js app to track LLM token usage, per-model costs, tool calls, and agent traces with full-stack trace context. ### Monitor your MCP server with Sentry Add monitoring to your own Node.js or Python MCP server with one line of code. Track clients, transports, tool performance, error rates, and trace each request. ### Monitor Claude Code usage and costs with Sentry Track Claude Code token usage, API costs, and tool activity with Sentry AI Observability. Set up OpenTelemetry-based observability for your Claude Code sessions in minutes. ### Monitor your OpenCode sessions with Sentry Add Sentry AI observability to OpenCode in minutes. Track tool calls, token usage, model costs, and session durations across every coding session. ### Monitor your pi coding agent sessions with Sentry Add Sentry AI observability to pi in minutes. Track tool calls, LLM calls, token usage, and model costs across every pi coding session. ### Use production AI agent conversations in your evals Export real AI agent conversations from Sentry with the CLI or MCP server and turn them into eval datasets in Braintrust, Langfuse, promptfoo, or Phoenix. ### Send Vercel AI SDK telemetry to Sentry via OpenTelemetry Keep your existing Vercel AI SDK and OpenTelemetry setup, and route LLM spans to Sentry's AI Agents Insights without ripping out @vercel/otel. ### Automate daily AI agent error triage with Claude Routines and Sentry Set up a Claude Routine that uses Sentry Agent Tracing via MCP to automatically triage your AI agent's overnight errors, sample conversations, and file tickets ### Install the Sentry plugin in your coding agent Install the Sentry plugin in Claude Code, Cursor, Codex, or Grok with one command, then use prompts for setup, debugging, monitoring, and code review. ### Instrument Flue agents with Sentry Add Sentry to a Flue agent. Trace model calls and tools, correlate logs, and capture terminal agent failures as Sentry Issues. ### Instrument Eve agents with Sentry Instrument an Eve agent with Sentry and inspect its conversations, model calls, tool executions, tokens, and latency. ### Monitor AI agent spend with dashboards and alerts Build a Sentry dashboard with the CLI for AI agent cost, tokens, top spenders, and models, then add static and anomaly detectors that alert when hourly spend spikes. ## AI observability FAQ **What is AI observability?** AI observability is the practice of understanding what your AI features are actually doing in production, across LLM calls, agent runs, tool executions, and the application code around them. Instead of treating the model as a black box, it connects prompts, responses, token usage, latency, errors, and cost to the rest of your application's [traces](https://sentry.io/product/tracing/) and logs, so you can debug failures down to the specific call that caused them. **What is LLM observability, and how is it different from AI observability?** LLM observability focuses specifically on the behavior of large language model calls: prompts, completions, token counts, model latency, and cost. AI observability is broader: it covers LLM calls plus the agents, tools, retrieval steps, and custom logic that wrap them. In practice you want both, which is why Sentry tracks individual LLM calls and stitches them into full [agent traces](https://sentry.io/product/tracing/) in one place. **What's the difference between AI monitoring and AI observability?** AI monitoring tells you that something is wrong: an error spiked, a model got slow, costs jumped. AI observability helps you understand why, by giving you the connected traces, prompts, responses, and code context behind the signal. Sentry does both: real-time alerts when something breaks, plus the [debuggability](https://blog.sentry.io/monitoring-observability-and-debuggability-explained/) to trace it to a root cause. **What is agent tracing?** Agent tracing follows a single AI agent run from start to finish as a connected trace, capturing every LLM call, tool execution, and handoff as a span with its inputs, outputs, token counts, cost, and timing. Because AI agents make decisions across many steps, a flat log isn't enough; agent tracing shows you the full execution path so you can see exactly which step failed or stalled. [See how agent tracing works in Sentry](https://docs.sentry.io/product/agents/). **How do I get observability for AI agents?** Add the Sentry SDK and the integration for your agent framework: OpenAI Agents, Vercel AI SDK, LangChain, LangGraph, Anthropic, Google Gen AI, and more. Sentry auto-instruments agent runs and captures `invoke_agent`, `execute_tool`, and handoff spans, so you get full AI agent observability and tracing without rewriting your agent logic. **Can I monitor and trace MCP servers?** Yes. Sentry gives you [observability for Model Context Protocol (MCP) servers](https://sentry.io/resources/mcp-observability/): tracking tool calls, errors, latency, and usage, and connecting them into the same traces as the agents and LLMs that call them. See the [guide to monitoring an MCP server](https://sentry.io/cookbook/monitor-mcp-server/) to get started. **Is AI observability included in Sentry's free plan?** Yes. AI agent and LLM observability is available on every Sentry plan, including the free Developer plan: full tracing, token and cost tracking, and tool-execution monitoring out of the box. [See pricing details](https://sentry.io/pricing/). ## Customer Story > Sentry played a significant role in helping us develop [Claude] Sonnet [Read the full story](https://sentry.io/customers/anthropic/) - **10-15%** increase in developer productivity - **600+** engineers rely on Sentry to ship code - **20-30%** faster incident resolution --- *Source: [sentry.io/solutions/ai-observability/](https://sentry.io/solutions/ai-observability/)*