Enable workload identity federation for GitLab CI/CD

Databricks OAuth token federation, also known as OpenID Connect (OIDC), allows your automated workloads running outside of Databricks to securely access Databricks without the need for Databricks secrets. See Authenticate access to Azure Databricks using OAuth token federation.

To enable workload identity federation for GitLab CI/CD:

1. Create a federation policy
2. Configure the GitLab YAML file

After you enable workload identity federation, the Databricks SDKs and the Databricks CLI automatically fetch workload identity tokens from GitLab CI/CD and exchange them for Databricks OAuth tokens.

Create a federation policy

First, use the Databricks CLI to create a workload identity federation policy. For GitLab CI/CD set the following values:

• issuer: the GitLab organization URL
• audiences: the GitLab's organization URL
• subject: a concatenation of values taken from the job context

For example, given a GitLab URL https://gitlab.com/example-group with a Databricks service principal numeric ID of 5581763342009999, create a federation policy using the following Databricks CLI command:

databricks account service-principal-federation-policy create 5581763342009999 --json '{
  "oidc_policy": {
	"issuer": "https://gitlab.com/example-group",
	"audiences": [
  	  "https://gitlab.com/example-group"
	],
	"subject": "project_path:my-group/my-project:..."
  }
}'

Configure the GitLab YAML file

Next, modify the GitLab configuration file. In addition to setting the following workspace environment variables, store the token in the DATABRICKS_OIDC_TOKEN Databricks environment variable. Alternatively, use a custom environment variable and set DATABRICKS_OIDC_TOKEN_ENV.

• DATABRICKS_AUTH_TYPE: env-oidc
• DATABRICKS_HOST: your Databricks workspace URL
• DATABRICKS_CLIENT_ID: the service principal (application) ID

stages:
  - my_script_using_wif

variables:
  DATABRICKS_AUTH_TYPE: env-oidc
  DATABRICKS_HOST: https://my-workspace.cloud.databricks.com/
  DATABRICKS_CLIENT_ID: a1b2c3d4-ee42-1eet-1337-f00b44r

my_script_using_wif:
  id_tokens:
    DATABRICKS_OIDC_TOKEN:
      aud: https://gitlab.com/example-group
  stage: explore
  image: ubuntu:latest
  before_script:
    - curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sh
  script:
    - databricks current-user me