Share via

Use a Microsoft Entra service principal for automation with Azure Databricks Git folders

You can use a Microsoft Entra ID to authenticate access to Azure Databricks Git folders from your Azure DevOps automation. In this topic, you configure a new Azure Databricks service principal to provide authorization for your Azure Databricks application through Microsoft Entra.

Requirements

To complete these steps, you must have the following configured for your Azure Databricks account:

You must also have the following:

Configure a Microsoft Entra service principal

  1. Log in to your Azure Databricks workspace, navigate to your profile icon in the upper right, and select Settings from the dropdown.

  2. Select Identity and access under Settings in the left sidebar and then click the Manage button for Service principals.

    The Identity and access screen if Databricks workspace settings

    1. Choose Add service principal or an existing service principal that you want to reconfigure. If you have an existing service principal, you can skip the next step.

  3. If you are creating a new Microsoft Entra ID managed service principal in your Azure Databricks workspace, select the Microsoft Entra ID managed radio button on the Add service principal pane. Provide the Microsoft Entra application ID you previously created or copied in the Microsoft Entra application ID text box and a Service principal name. Check the boxes for the appropriate entitlements that your Azure DevOps automation will require, including "Workspace access".

    • If your service principal will be used to run Lakeflow Jobs that access artifacts from your Git folders, select Unrestricted cluster creation when configuring the Git integration for your Azure Databricks service principal.

    When you've finished configuring your Azure Databricks service principal user, click Add. Your new service principal will be listed under Service principals.

    The list of service principals available for your workspace in the Service principals pane

  4. In the Service principals pane, find and select the name of the service principal for your Microsoft Entra application.

  5. Open the Git integration tab on the pane for your service principal and select “Azure DevOps Services (Azure Active Directory)” from the Git provider dropdown. In a grey box at the bottom of the Service principal details pane, you will see a list of inputs for creating a federated credential. Copy this information, as you will use it in the next step. Do not check the I've done the steps above box or click Save.

    The Service principal details pane for Git integration. Copy down the details from the grey callout box.

  6. Open a new browser window or tab, and go to your Microsoft Entra portal for your Azure subscription. Find your Azure Application, and select Manage and then Certificates & secrets. Select the Federated credentials tab, and under it select Add credential.

    The Certificates and secrets pane in the Microsoft Entra portal

  7. (Microsoft Entra portal) Refer to the details from the callout box from when you configured your Azure Databricks service principal's Git integration in a previous step, and use that information to populate the Issuer, Type, and Value fields under Connect your account.

    The service principal details pane for Git integration. Copy the details from the grey callout box.

  8. Now, return to the browser window or tab with your incomplete service principal Git integration configuration. Check the box for I’ve done the steps above box, then click Save to save your new credential configuration.

You can now use this service principal with Azure Databricks and Azure DevOps to run jobs that access Azure Databricks Git folders. When sharing this service principal, give Service Principal User access to any Azure Databricks workspace users who run Git jobs or have automation code that will access the Repos API with your service principal.

Troubleshooting

  • Your Azure Databricks service principal must have an access level of Basic or above in the Azure DevOps organization of the target repository. Configure this through your Azure DevOps subscription under Organization Settings > Users > Add Users. You can copy and paste your service principal’s application (client) ID into the Users or Service Principals search box to find and select your service principal account. For more information, see Change access levels in the Azure DevOps documentation.
  • Your service principal must be added to your Microsoft Azure workspace and other users on your Azure account must have permissions to use it. See Service principals.

Additional resources