← Back to list

How to Create a New Catalog in Databricks Unity Catalog on Azure (Storage Credential + Access…

If you’ve ever tried to stand up a new Unity Catalog catalog on Azure and hit a wall of unfamiliar terms — access connector, managed…

Vipul Malhotra · 2026-09-14 18:38 · 0 claps · 5.2 min read paywalled
#azure-databricks #databricks-unity-catalog
Open on Medium ↗
Wiki topics: RAG · RAG & Retrieval ☁️ · DevOps & Cloud 🔧 · Data Engineering 🎮 · Gaming

How to Create a New Catalog in Databricks Unity Catalog on Azure (Storage Credential + Access Connector, Step by Step)

If you’ve ever tried to stand up a new Unity Catalog catalog on Azure and hit a wall of unfamiliar terms — access connector, managed identity, storage credential, external location — you’re not alone. The Azure path to Unity Catalog has more moving parts than AWS or GCP, mostly because Azure doesn’t have a direct equivalent to an IAM role you can just attach to a bucket. Instead, Databricks uses a first-party Azure resource called an Access Connector for Azure Databricks to bridge the gap.

This guide walks through the entire chain, in order: storage account → access connector → storage credential → external location → catalog. By the end you’ll have a brand-new catalog backed by your own Azure Data Lake Storage (ADLS Gen2) account, governed entirely through Unity Catalog.

What You’ll Need Before You Start

  • An Azure Databricks workspace that’s already enabled for Unity Catalog, with a metastore attached. For all the latest databricks workspace, this is automatically enabled
  • Account admin or metastore admin privileges in Databricks (or CREATE STORAGE CREDENTIAL / CREATE EXTERNAL LOCATION privileges granted to you).
  • Contributor or Owner on an Azure resource group, so you can create the access connector.
  • Owner or User Access Administrator on the target storage account, so you can assign it a role.
  • An ADLS Gen2 storage account with hierarchical namespace enabled (standard blob storage won’t work with Unity Catalog).

Databricks workspace

Databricks workspace

Step 1: Create Your ADLS Gen2 Storage Account and Container

If you don’t already have one, create a storage account in the Azure Portal:

  1. Go to Storage accounts → Create.
  2. On the Advanced tab, enable Hierarchical namespace — this is what turns a regular blob account into ADLS Gen2.
  3. Pick the same region your metastore and workspace live in. Cross-region traffic adds latency and, in some setups, egress cost.
  4. Once the account is created, go to Containers and create one, e.g. unity-catalog-data.

Keep the storage account name and container name handy — you’ll build a path like abfss://unity-catalog-data@<storageaccount>.dfs.core.windows.net/ from them shortly.

Azure storage with container

Azure storage with container

Step 2: Create an Access Connector for Azure Databricks

This is the “Azure connector” piece, and it’s the key difference from AWS/GCP setups. The Access Connector is a lightweight Azure resource whose only job is to carry a managed identity that Databricks can use to authenticate against your storage.

  1. In the Azure Portal, click Create a resource and search for Access Connector for Azure Databricks.
  2. Choose the same region as your storage account.
  3. On the Managed Identity tab, leave the system-assigned identity On (simplest option), or attach a user-assigned managed identity if your org standardizes on those.
  4. Click Review + create, then Create.
  5. Once deployed, open the resource and copy its Resource ID — it looks like:
/subscriptions/455f49a6-9e15-430a-8b2a-6d1b50c26b70/resourceGroups/databricks-rg/providers/Microsoft.Databricks/accessConnectors/azureconnector

You’ll paste this into the storage credential in Step 4.

Step 3: Grant the Connector’s Managed Identity Access to Storage

The connector exists, but it can’t touch your data yet — you need to explicitly grant it a role on the storage account.

  1. Open your storage account → Access Control (IAM)Add role assignment.
  2. Select the Storage Blob Data Contributor role.
  3. Under Assign access to, choose Managed identity, then find the access connector you just created (search by its name).
  4. Save. Role assignments can take a minute or two to propagate — don’t panic if the very next step fails immediately.

Adding connector access to storage account

Adding connector access to storage account

Step 4: Create a Storage Credential in Unity Catalog

The storage credential is the Unity Catalog object that formally links your metastore to the access connector.

Using Catalog Explorer:

  1. In your Databricks workspace, go to Catalog → External Data → Credentials → Create credential.
  2. Set Credential type to Azure Managed Identity.
  3. Name it something clear, like azure-uc-storage-cred.
  4. Paste in the Access Connector resource ID from Step 2.
  5. Click Create.

Creating new connection in databricks catalog

Creating new connection in databricks catalog

New connection created

New connection created

Step 5: Create an External Location

The storage credential says how to authenticate; the external location says where — a specific ABFSS path plus the credential to use there.

In Catalog Explorer: go to Catalog → External Data → External Locations → Create location, then fill in:

  • URL: abfss://unity-catalog-data@<storageaccount>.dfs.core.windows.net/
  • Storage credential: the one you created in Step 4

Creating new external location in databricks catalog, using the connection created earlier

Creating new external location in databricks catalog, using the connection created earlier

New external location created

New external location created

Step 6: Create the New Catalog

Now for the actual point of this exercise. You can bind the catalog directly to the external location’s path as its managed storage root.

In Catalog Explorer: click Create Catalog, give it a name, and under Storage location point it at the external location path (or a subpath of it, like .../marketing).

Creating a new catalog

Creating a new catalog

Step 7: Verify It Actually Works

Don’t just trust the green checkmarks — create something:

CREATE SCHEMA marketing_catalog.campaigns;
CREATE TABLE marketing_catalog.campaigns.test_table (id INT, name STRING);
INSERT INTO marketing_catalog.campaigns.test_table VALUES (1, 'hello unity catalog');
SELECT * FROM marketing_catalog.campaigns.test_table;

Table created and data added

Table created and data added

New catalog with schema and tables created by us

New catalog with schema and tables created by us

Common Pitfalls

  • “Standard” storage accounts silently fail. Unity Catalog requires hierarchical namespace (ADLS Gen2). Double-check this at creation time — it can’t be toggled on later without recreating the account.
  • Role assignment lag. Give the IAM role assignment a minute or two before testing; it isn’t always instant.
  • Region mismatches. Keep the storage account, access connector, workspace, and metastore in the same Azure region to avoid latency and, in some configurations, outright connectivity issues.
  • Path collisions. Unity Catalog won’t let two managed locations overlap (e.g., one catalog’s path being a parent of another’s). Plan your folder structure before you start creating catalogs.
  • Wrong privilege level. Creating storage credentials and external locations requires metastore-level privileges — a workspace admin alone isn’t always enough.

Wrapping Up

The Azure-specific piece that trips people up is almost always the access connector: it’s easy to forget it’s a separate Azure resource that needs its own role assignment before Unity Catalog will accept it as a storage credential. Once you’ve done it once, though, the pattern is reusable — the same access connector and storage credential can back external locations for any number of catalogs, so most of this setup is a one-time cost per storage account, not a per-catalog one.


메타데이터
post_id
ce99bdab7fd9
slug
how-to-create-a-new-catalog-in-databricks-unity-catalog-on-azure-storage-credential-access-ce99bdab7fd9
url
https://medium.com/@vipulm124/how-to-create-a-new-catalog-in-databricks-unity-catalog-on-azure-storage-credential-access-ce99bdab7fd9
canonical_url
https://medium.com/@vipulm124/how-to-create-a-new-catalog-in-databricks-unity-catalog-on-azure-storage-credential-access-ce99bdab7fd9
author_url
https://medium.com/@vipulm124
status
ok
fetched_at
2026-09-15 22:18:39