# Monitor LLM spend with Sentry dashboards and alerts

> Build a Sentry dashboard with the CLI for AI agent cost, tokens, top spenders, and models, then add static and anomaly detectors that alert when hourly spend spikes.

**URL:** https://sentry.io/cookbook/monitor-ai-agent-spend-with-dashboards-and-alerts/

---

Build a shared Sentry dashboard for AI agent cost, tokens, and top spenders, then get alerted when hourly spend breaches your budget or spikes. The dashboard the script builds is literally 'LLM Spend per User'.

**Time:** 20 minutes | **Difficulty:** Intermediate

## What You'll Learn

- Checked that your AI spans contain the fields used by the dashboard

- Created eight spend and usage widgets with the Sentry CLI

- Avoided double-counting cost from parent agent spans

- Created static and anomaly spend detectors

**Topics:** AI Observability, Agent Tracing, Dashboards, Alerts, CLI, API

## Before you start

**Telemetry**

- A Sentry project receiving AI agent spans with standard `gen_ai.*` attributes

- At least a few model calls with token data

**Accounts & access**

- Permission to create dashboards and detectors in your Sentry organization

- A Sentry auth token for the CLI

**Tools**

- The [Sentry CLI](https://docs.sentry.io/cli/installation/) installed

- Bash, `curl`, and Python 3

## Steps

### 1. Check the fields that drive the dashboard

Open [Trace Explorer](https://sentry.io/orgredirect/organizations/:orgslug/explore/traces/) and inspect a model-call span. Confirm that it contains `gen_ai.operation.type:ai_client`, `gen_ai.usage.total_tokens`, and `gen_ai.response.model`. Add `user.id` and `user.username` in your instrumentation if you want per-user widgets, and set `gen_ai.conversation.id` if you want the most expensive conversations table. The dashboard filters to `ai_client` spans because one turn can carry cost on both the model call and its parent agent span. Summing both would count the same cost twice.

[Sentry AI agent monitoring](https://docs.sentry.io/product/agents/)

### 2. Install and authenticate the Sentry CLI

Install the current Sentry CLI, then authenticate it. `sentry login` stores the token in the CLI configuration so the dashboard script can call your organization APIs.

```bash
curl -sL https://sentry.io/get-cli/ | sh
sentry login
```

[Sentry CLI authentication](https://docs.sentry.io/cli/configuration/)

### 3. Create the dashboard and detectors

Download the reusable script from Sentry's agent tracing examples, make it executable, and pass your organization and project slugs. The script creates one dashboard and two detectors. It stops before making changes when items with the same names already exist, and it removes anything from the current run if a later API call fails.

```bash
curl -O https://raw.githubusercontent.com/getsentry/sentry-agent-tracing-examples/main/dashboards/llm-spend-per-user.sh
chmod +x llm-spend-per-user.sh
./llm-spend-per-user.sh <org-slug> <project-slug>
```

[Dashboard script source](https://github.com/getsentry/sentry-agent-tracing-examples/tree/main/dashboards)

### 4. Review the spend dashboard

Open the URL that the script prints, or select **LLM Spend per User** in [Dashboards](https://sentry.io/orgredirect/organizations/:orgslug/dashboards/). If the user or conversation widgets are empty, add `user.id`, `user.username`, or `gen_ai.conversation.id` to your model-call spans.

[Sentry Dashboards](https://docs.sentry.io/product/dashboards/)

### 5. Tune the spend monitors

Open [Monitors](https://sentry.io/orgredirect/organizations/:orgslug/monitors/) and adjust `LLM spend rate high` and `LLM spend anomaly` for your traffic and budget. To route notifications during creation, set `WORKFLOW_ID` and, optionally, `OWNER=user:` or `OWNER=team:` before you run the script. [Watch the alert workflow in the demo video](https://www.youtube.com/watch?v=2FKg1rJpZk4&t=451s).

[Sentry Monitors and Alerts](https://docs.sentry.io/product/monitors-and-alerts/)

## Summary

**Agent spend is visible before it becomes a surprise.**

Your team can now see who and what drives AI cost, then respond when hourly spend crosses a limit or leaves its normal range.

## Pro Tips

- Start with the included thresholds in a test project, then replace them with limits that match your model mix and traffic.

- Set `user.id` to a stable internal identifier and `user.username` to a useful display value. Do not use email addresses unless your data policy allows them.

- Use `TITLE` to give a second dashboard a distinct name when you need separate views for environments or teams.

- Add an alert workflow so a detector routes to the team that can reduce spend or stop a faulty agent.

## Common Pitfalls

- Removing the `gen_ai.operation.type:ai_client` filter from cost or token widgets. Parent agent spans can repeat the model-call values and double the totals.

- Assuming Sentry receives a dollar value from the SDK. Sentry derives `gen_ai.cost.total_tokens` from token counts and its model price list.

- Running the script again without deleting or renaming the existing dashboard and detectors. The duplicate-name guard stops the run on purpose.

- Treating the starting thresholds as production defaults. A suitable hourly limit depends on your traffic, models, and budget.

## FAQ

**Does the script work only with Flue or Eve?**

No. It works with any Sentry project that receives the standard GenAI span attributes used by the queries.

**Why does the cost field end with `total_tokens`?**

That is the current Sentry field name for derived model cost. Its value is in US dollars, not tokens.

**Why are the per-user widgets empty?**

Your model-call spans need `user.id`. The top-spenders and conversation tables also use `user.username` and `gen_ai.conversation.id`.

**Can the alerts notify Slack or assign an owner?**

Yes. Connect an existing alert workflow with `WORKFLOW_ID`, and set `OWNER` to a Sentry user or team when you create the detectors.

## Next Steps

- [Instrument a Flue agent](/cookbook/instrument-flue-agents-with-sentry/): Send Flue conversations, tools, logs, and terminal failures to Sentry.

- [Instrument an Eve agent](/cookbook/send-eve-agent-traces-to-sentry/): Send Eve conversations and tool calls to Sentry over OTLP.

- [Browse the agent examples](https://github.com/getsentry/sentry-agent-tracing-examples): See complete Flue, Eve, and dashboard examples in one repository.

---

*Source: [sentry.io/cookbook/monitor-ai-agent-spend-with-dashboards-and-alerts/](https://sentry.io/cookbook/monitor-ai-agent-spend-with-dashboards-and-alerts/)*