Configure a federation policy

Databricks OAuth token federation enables you to securely access Databricks APIs using tokens from your identity provider (IdP). To enable OAuth token federation, you must configure a federation policy, either as Databricks account-wide or for workloads.

This article describes how to create and configure an OAuth token federation policy.

Account-wide token federation

Account admins can configure OAuth token federation in the Azure Databricks account using an account federation policy. An account federation policy enables all users and service principals in your Azure Databricks account to access Databricks APIs using tokens from your identity provider. An account federation policy specifies:

• The identity provider or issuer from which Azure Databricks will accept tokens.
• The criteria for mapping a token to the corresponding Azure Databricks user or service principal.

For example, given the following account federation policy:

issuer: "https://idp.mycompany.com/oidc"
audiences: ["databricks"]
subject_claim: "sub"

This JWT body can be used to authenticate to Databricks as username@mycompany.com:

{
  "iss": "https://idp.mycompany.com/oidc",
  "aud": "databricks",
  "sub": "username@mycompany.com"
}

Configure an account federation policy

To configure an account federation policy, you must specify:

• The required token issuer, specified in the iss claim of your tokens. The issuer is an HTTPS URL that identifies your identity provider.

• The allowed token audiences, specified in the aud claim of your tokens. This identifier represents the recipient of the token. As long as the audience in the token matches at least one audience in the policy, the token is considered a match. If unspecified, the default value is your Azure Databricks account ID.

• The subject claim. This indicates which token claim contains the Azure Databricks username of the user the token was issued for. If unspecified, the default value is sub.

• Optionally, the public keys (or the URL of the public keys) used to validate the signature of your tokens, in JSON Web Key Sets (JWKS) format. If unspecified (recommended), Databricks automatically fetches the public keys from your issuer's well known endpoint. Databricks strongly recommends relying on your issuer's well known endpoint for discovering public keys.

  Note

  If you do not specify a JWKS or a jwks_uri in your federation policy, your identity provider must serve OpenID Provider Metadata at {issuer-url}/.well-known/openid-configuration. The OpenID Provider Metadata must include a jwks_uri that specifies the ___location of the public keys used to verify token signatures.

Account admins can configure an account federation policy using the Databricks CLI (version 0.239.0 and above) or the Databricks API. You can specify up to five account federation policies in your Azure Databricks account.

Databricks CLI

1. Install or update to the newest version of the Databricks CLI.

2. As an account admin, authenticate to your Databricks account using the CLI. Specifying the ACCOUNT_CONSOLE_URL (e.g.https://accounts.cloud.databricks.com) and your Databricks ACCOUNT_ID:

   databricks auth login --host ${ACCOUNT_CONSOLE_URL} --account-id ${ACCOUNT_ID}
   

3. Create the account federation policy. For example:

   databricks account federation-policy create --json \
   '{
     "oidc_policy": {
       "issuer": "https://idp.mycompany.com/oidc",
       "audiences": [
         "databricks"
       ],
       "subject_claim": "sub"
     }
   }'
   

Databricks Account API

The following is an example Databricks REST API call to create an account federation policy:

curl --request POST \
  --header "Authorization: Bearer $TOKEN" \
  "https://accounts.cloud.databricks.com/api/2.0/accounts/${ACCOUNT_ID}/federationPolicies" \
  --data '{
    "oidc_policy": {
      "issuer": "https://idp.mycompany.com/oidc",
      "audiences": [
        "databricks"
      ],
      "subject_claim": "sub"
    }
  }'

Example account federation policies

The following table provides example account federation policies and the matching JWT body.

Workload identity federation

Workload identity federation allows your automated workloads running outside of Azure Databricks to access Databricks APIs without the need for Databricks secrets. Account admins can configure workload identity federation using a service principal federation policy.

A service principal federation policy is associated with a service principal in your Azure Databricks account, and specifies:

• The identity provider (or issuer) from which the service principal can authenticate.
• The workload identity (or subject) that is permitted to authenticate as the Databricks service principal.

For example, given the following service principal federation policy for a Github Actions workload:

issuer: "https://token.actions.githubusercontent.com"
audiences: ["https://github.com/my-github-org"]
subject: "repo:my-github-org/my-repo:environment:prod"

This JWT body can be used to authenticate to Azure Databricks:

{
  "iss": "https://token.actions.githubusercontent.com",
  "aud": "https://github.com/my-github-org",
  "sub": "repo:my-github-org/my-repo:environment:prod"
}

Configure a service principal federation policy

To configure a service principal federation policy, you must specify:

• The required token issuer, specified in the iss claim of workload identity tokens. The issuer is an HTTPS URL that identifies the workload identity provider.

• The required token subject, specified in the sub claim of workload identity tokens. The subject uniquely identifies the workload in the workload runtime environment.

• The allowed token audiences, specified in the aud claim of workload identity tokens. The audience represents the recipient of the token. As long as the audience in the token matches at least one audience in the policy, the token is considered a match. If unspecified, the default value is your Azure Databricks account ID.

• Optionally, the public keys (or the URL of the public keys) used to validate the signature of the workload identity tokens, in JSON Web Key Sets (JWKS) format. If unspecified (recommended), Azure Databricks automatically fetches the public keys from the issuer's well known endpoint. Databricks strongly recommends relying on the issuer's well known endpoint for discovering public keys.

  Note

  If you do not specify a JWKS or a jwks_uri in your federation policy, your identity provider must serve OpenID Provider Metadata at {issuer-url}/.well-known/openid-configuration. The OpenID Provider Metadata must include a jwks_uri that specifies the ___location of the public keys used to verify token signatures.

• Optionally, the subject claim. This indicates which token claim contains the workload identity (or subject) of the token. If it is unspecified, the default value is sub. Databricks strongly recommends using the default sub claim for workload identity federation. A claim other than sub should only be used in cases where the sub claim is not an appropriate or stable subject identifier, which is uncommon. See Example service principal federation policies below for details.

Account admins can configure a service principal federation policy using the Databricks CLI (version 0.239.0 and above) or the Databricks API. You can create up to five service principal federation policies per Databricks service principal.

Databricks CLI

1. Install or update to the newest version of the Databricks CLI.

2. As an account admin, authenticate to your Databricks account using the CLI. Specifying the ACCOUNT_CONSOLE_URL (for example, https://accounts.cloud.databricks.com) and your Databricks ACCOUNT_ID:

   databricks auth login --host ${ACCOUNT_CONSOLE_URL} --account-id ${ACCOUNT_ID}
   

3. Get the numeric ID of the service principal that will have the federation policy applied to it. (For example, 3659993829438643.)

   If you know the service principal application ID (typically a GUID value, such as bc3cfe6c-469e-4130-b425-5384c4aa30bb) in advance, you can then determine the service principal numeric ID using the Databricks CLI:

   databricks account service-principals list --filter 'applicationId eq "<service-principal-application-id>"'
   

4. Create the service principal federation policy. Here is an example of creating a federation policy for a GitHub Action:

   databricks account service-principal-federation-policy create ${SERVICE_PRINCIPAL_NUMERIC_ID} --json \
   '{
     "oidc_policy": {
       "issuer": "https://token.actions.githubusercontent.com",
       "audiences": [
         "https://github.com/my-github-org"
       ],
       "subject": "repo:my-github-org/my-repo:environment:prod"
     }
   }'
   

Databricks Account API

1. Get the numeric ID of the service principal that will have the federation policy applied to it, for example, 3659993829438643. If you know the service principal application ID (typically a GUID value, such as bc3cfe6c-469e-4130-b425-5384c4aa30bb) in advance, you can then determine the service principal numeric ID using the Databricks service principal REST API:

   curl --get \
     --header "Authorization: Bearer $TOKEN" \
     "https://accounts.cloud.databricks.com/api/2.0/accounts/${ACCOUNT_ID}/scim/v2/ServicePrincipals" \
     --data-urlencode 'filter=applicationId eq "<service-principal-application-id>"'
   

   The service principal numeric ID is returned in the id field of the response.

2. Create the service principal federation policy. Here is an example of creating a federation policy for a GitHub Action:

   curl --request POST \
     --header "Authorization: Bearer $TOKEN" \
     "https://accounts.cloud.databricks.com/api/2.0/accounts/${ACCOUNT_ID}/servicePrincipals/${SERVICE_PRINCIPAL_NUMERIC_ID}/federationPolicies" \
     --data '{
       "oidc_policy": {
         "issuer": "https://token.actions.githubusercontent.com",
         "audiences": [
           "https://github.com/my-github-org"
         ],
         "subject": "repo:my-github-org/my-repo:environment:prod"
       }
     }'
   

Example Databricks service principal federation policies

The following table provides example service principal federation policies and the matching JWT body.

For complete configuration steps for enabling workload identity federation for some of these common identity providers, see Enable workload identity federation in CI/CD.

Next steps

After you have configured a federation policy for your account:

• You might need to configure your identity provider to generate tokens for your users to exchange with Databricks. See the documentation for your identity provider for instructions. For complete configuration steps for enabling workload identity federation for some common identity providers, see Enable workload identity federation in CI/CD.
• You can use a JWT from your identity provider to access the Databricks API. First exchange a JWT token from your identity provider for a Databricks OAuth token, and then use the Databricks OAuth token in the Bearer: field of the API call to gain access and complete the call. Tokens must be valid JWTs that are signed using the RS256 or ES256 algorithms. For details about implementing the token exchange, see Use an identity provider token to authenticate Databricks REST APIs.