Run AI Red Teaming Agent in the cloud (preview)

2025-06-03

Important

Items marked (preview) in this article are currently in public preview. This preview is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Though the AI Red Teaming Agent (preview) can be run locally during prototyping and development to help identify safety risks, running them in the cloud allows for pre-deployment AI red teaming runs on larger combinations of attack strategies and risk categories for a fuller analysis.

Prerequisites

Note

You must use a Foundry project for this feature. A hub based project isn't supported. See How do I know which type of project I have? and Create a Foundry project.

If this is your first time running evaluations or AI red teaming runs on your Azure AI Foundry project, you might need to do a few additional setup steps.

Create and connect your storage account to your Azure AI Foundry project at the resource level. This bicep template provisions and connects a storage account to your Foundry project with key authentication.
Make sure the connected storage account has access to all projects.
If you connected your storage account with Microsoft Entra ID, make sure to give MSI (Microsoft Identity) permissions for Storage Blob Data Owner to both your account and Foundry project resource in Azure portal.

Getting started

First, install Azure AI Foundry SDK's project client which runs the AI Red Teaming Agent in the cloud

uv install azure-ai-projects azure-identity

Note

For more detailed information, see the REST API Reference Documentation.

Then, set your environment variables for your Azure AI Foundry resources

import os

endpoint = os.environ["PROJECT_ENDPOINT"] # Sample : https://<account_name>.services.ai.azure.com/api/projects/<project_name>
model_endpoint = os.environ["MODEL_ENDPOINT"] # Sample : https://<account_name>.services.ai.azure.com
model_api_key= os.environ["MODEL_API_KEY"]
model_deployment_name = os.environ["MODEL_DEPLOYMENT_NAME"] # Sample : gpt-4o-mini

Supported targets

Running the AI Red Teaming Agent in the cloud currently only supports Azure OpenAI model deployments in your Azure AI Foundry project as a target.

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import (
    RedTeam,
    AzureOpenAIModelConfiguration,
    AttackStrategy,
    RiskCategory,
)

with AIProjectClient(
  endpoint=endpoint,
  credential=DefaultAzureCredential(exclude_interactive_browser_credential=False),
) as project_client:

# Create target configuration for testing an Azure OpenAI model
target_config = AzureOpenAIModelConfiguration(model_deployment_name=model_deployment_name)

# Instantiate the AI Red Teaming Agent
red_team_agent = RedTeam(
    attack_strategies=[AttackStrategy.BASE64],
    risk_categories=[RiskCategory.VIOLENCE],
    display_name="red-team-cloud-run", 
    target=target_config,
)

# Create and run the red teaming scan
red_team_response = project_client.red_teams.create(red_team=red_team_agent, headers={"model-endpoint": model_endpoint, "api-key": model_api_key,})

curl --request POST \  --url https://{{account}}.services.ai.azure.com/api/projects/{{project}}/redteams/runs:run \  --header 'content-type: application/json' \  --header 'authorization: Bearer {{ai_token}}'  --data '{  "scanName": "sample_scan_magic_1",  "riskCategories": [    "Violence"  ],  "attackStrategy": [    "Flip"  ],  "numTurns": 1,  "target": {    "type": "AzureOpenAIModel",    "modelDeploymentName": "{{connectionName}}/{{deploymentName}}"  }}'

Replace {{account}}, {{project}} with Foundry Project account name and project name.
Replace {{ai_token}} with Bearer token with audience "https://ai.azure.com"
Replace "{{connectionName}}" with the Azure OpenAI model connection name connected to the Foundry project account.
Replace "{{deploymentName}}" with the Azure OpenAI deployment name of the AOAI connection account.

Get an AI red teaming run

Python
cURL

# Use the name returned by the create operation for the get call
get_red_team_response = project_client.red_teams.get(name=red_team_response.name)
print(f"Red Team scan status: {get_red_team_response.status}")

curl --request GET \  --header 'authorization: Bearer {{ai_token}}'  --url https://{{account}}.services.ai.azure.com/api/projects/{{project}}/redteams/runs/{{scan_id}}

Replace "{{scan_id}" with the ID returned by the POST API.

List all AI red teaming runs

Python
cURL

for scan in project_client.red_teams.list():
  print(f"Found scan: {scan.name}, Status: {scan.status}")

curl --request GET \  --header 'authorization: Bearer {{ai_token}}'  --url https://{{account}}.services.ai.azure.com/api/projects/{{project}}/redteams/runs

Once your AI red teaming run is finished running, you can view your results in your Azure AI Foundry project.

Try out an example workflow in our GitHub samples.

Share via