Edit

Share via


Monitor Azure AI Foundry Agent Service

This article describes:

  • The types of monitoring data you can collect for this service.
  • Ways to analyze that data.

Note

If you're already familiar with this service and/or Azure Monitor and just want to know how to analyze monitoring data, see the Analyze section near the end of this article.

When you have critical applications and business processes that rely on Azure resources, you need to monitor and get alerts for your system. The Azure Monitor service collects and aggregates metrics and logs from every component of your system. Azure Monitor provides you with a view of availability, performance, and resilience, and notifies you of issues. You can use the Azure portal, PowerShell, Azure CLI, REST API, or client libraries to set up and view monitoring data.

Monitoring is available for agents in a standard agent setup.

Important

Monitoring support is currently limited to Azure AI Foundry hubs. Azure AI Foundry projects are not supported.

Dashboards

Azure AI Foundry Agent Service provides out-of-box dashboards. There are two key dashboards to monitor your resource:

  • The metrics dashboard in the AI Foundry resource view
  • The dashboard in the overview pane within the Azure portal

To access the monitoring dashboards, sign in to the Azure portal and then select Monitoring in the left navigation menu, then click Metrics.

Screenshot that shows out-of-box dashboards for a resource in the Azure portal.

Azure monitor platform metrics

Azure Monitor provides platform metrics for most services. These metrics are:

  • Individually defined for each namespace.
  • Stored in the Azure Monitor time-series metrics database.
  • Lightweight and capable of supporting near real-time alerting.
  • Used to track the performance of a resource over time.
  • Collection: Azure Monitor collects platform metrics automatically. No configuration is required.

For a list of all metrics it's possible to gather for all resources in Azure Monitor, see Supported metrics in Azure Monitor.

Azure AI Foundry Agent Service metrics

Azure AI Foundry Agent Service has commonality with a subset of Azure AI services. Here's a list of currently available metrics on Azure Monitor:

Metric Name in REST API Unit Aggregation Dimensions Time Grains DS Export
Agents Agents Count Average, Maximum, Minimum, Total (Sum) EventType PT1M No
IndexedFiles IndexedFiles Count Average, Maximum, Minimum, Total (Sum) ErrorCode, Status, VectorStoreId PT1M No
Messages Messages Count Average, Maximum, Minimum, Total (Sum) EventType, ThreadId PT1M No
Runs Runs Count Average, Maximum, Minimum, Total (Sum) AgentId, RunStatus, StatusCode, StreamType PT1M No
Threads Threads Count Average, Maximum, Minimum, Total (Sum) EventType PT1M No
Tokens Tokens Count Average, Maximum, Minimum, Total (Sum) AgentId, TokenType PT1M No
ToolCalls ToolCalls Count Average, Maximum, Minimum, Total (Sum) AgentId, ToolName PT1M No

Analyze monitoring data

There are many tools for analyzing monitoring data.

Azure Monitor tools

Azure Monitor supports the metrics explorer, a tool in the Azure portal that allows you to view and analyze metrics for Azure resources. For more information, see Analyze metrics with Azure Monitor metrics explorer.

Azure Monitor export tools

You can get data out of Azure Monitor into other tools by using the REST API for metrics to extract metric data from the Azure Monitor metrics database. The API supports filter expressions to refine the data retrieved. For more information, see Azure Monitor REST API reference.

To get started with the REST API for Azure Monitor, see Azure monitoring REST API walkthrough.

Alerts

Azure Monitor alerts proactively notify you when specific conditions are found in your monitoring data. Alerts allow you to identify and address issues in your system before your customers notice them. For more information, see Azure Monitor alerts.

There are many sources of common alerts for Azure resources. The Azure Monitor Baseline Alerts (AMBA) site provides a semi-automated method of implementing important platform metric alerts, dashboards, and guidelines. The site applies to a continually expanding subset of Azure services, including all services that are part of the Azure Landing Zone (ALZ).

The common alert schema standardizes the consumption of Azure Monitor alert notifications. For more information, see Common alert schema.

Metric alerts evaluate resource metrics at regular intervals. Metric alerts can also apply multiple conditions and dynamic thresholds.

Every organization's alerting needs vary and can change over time. Generally, all alerts should be actionable and have a specific intended response if the alert occurs. If an alert doesn't require an immediate response, the condition can be captured in a report rather than an alert. Some use cases might require alerting anytime certain error conditions exist. In other cases, you might need alerts for errors that exceed a certain threshold for a designated time period.

Depending on what type of application you're developing with your use of Azure AI Foundry Agent Service, Azure Monitor Application Insights might offer more monitoring benefits at the application layer.

Azure AI Foundry Agent Service alert rules

You can set alerts for any metric listed in the monitoring data reference.

Advisor recommendations

For some services, if critical conditions or imminent changes occur during resource operations, an alert displays on the service Overview page in the portal. You can find more information and recommended fixes for the alert in Advisor recommendations under Monitoring in the left menu. During normal operations, no advisor recommendations display.

For more information on Azure Advisor, see Azure Advisor overview.