LLM Monitoring

Sentry LLM monitoring helps you understand your LLM calls.

此功能当前处于 Beta 阶段。Beta 功能仍在开发中,可能存在 bug。我们意识到其中的讽刺意味。

This view is only available on Team and Business plans. Team plans can access the last 7 days of data, while Business plans can access the last 90 days. To enable extended data access, upgrade to the Business or Enterprise plan.

Sentry's LLM Monitoring tools help you understand what's going on with your AI pipelines. They automatically collect information about prompts, tokens, and models from providers like OpenAI and Anthropic.

  • Users are reporting issues with an LLM workflow, and you want to investigate responses from the relevant large language models.
  • You'd like to receive an alert if a specific pipeline costs more than $100 in a single day.
  • Users report that LLM workflows are taking longer than usual, and you want to understand what steps in a workflow are slowest.

To use LLM Monitoring, you must have an existing Sentry account and project set up. If you don't have one, create an account here.

LLM Monitoring User Interface

Learn how to set up Sentry's LLM Monitoring.