Debugging Microsoft Entra Cloud Sync: What a Broken Hybrid Identity Deployment Taught Me

When people talk about hybrid identity, they usually show the ideal scenario. Install the provisioning agent, sign in, create a sync configuration, and a few minutes later your Active Directory users appear in Microsoft Entra ID.

That wasn’t my experience.

Instead, I spent the day troubleshooting authentication failures, DNS issues, Kerberos configuration, provisioning errors, and a corrupted service principal. At several points I thought I had found the root cause, only to discover another layer underneath.

As frustrating as it was, the experience turned out to be far more valuable than a perfect deployment. Every issue forced me to understand another piece of how Microsoft Entra Cloud Sync actually works instead of simply following a tutorial.

By the end of the day I had a much clearer understanding of the relationship between Active Directory Domain Services, Microsoft Entra ID, the Cloud Sync provisioning agent, service principals, group Managed Service Accounts (gMSAs), and the permissions required to tie everything together.

This article walks through that journey, the mistakes I made, how I investigated them, and what I learned along the way.

The Goal

The objective for this project was to build a small hybrid identity environment inside Azure.

I deployed a Windows Server 2022 virtual machine, promoted it to a domain controller, and configured Active Directory Domain Services as a new forest. From there, I wanted to synchronize my on-premises identities with my Microsoft Entra ID tenant using Microsoft Entra Cloud Sync.

Rather than stopping once the synchronization worked, I also wanted to verify how password hash synchronization behaved, observe the synchronization cycle, and understand what was happening behind the scenes instead of treating Cloud Sync as a black box.

Why I Chose Cloud Sync Instead of Microsoft Entra Connect Sync

Microsoft currently offers two primary ways to synchronize on-premises Active Directory with Microsoft Entra ID.

Microsoft Entra Connect Sync is the traditional solution. It installs a full synchronization engine on a Windows Server that handles directory synchronization locally. While it’s a mature and capable solution, it also requires more infrastructure and additional planning if you want high availability.

Cloud Sync takes a different approach.

Instead of running the synchronization engine on your server, you install a lightweight provisioning agent that maintains only outbound communication with Microsoft’s cloud service. The synchronization logic itself runs in Microsoft Entra, which simplifies deployment and eliminates the need for inbound firewall rules or staging servers.

Since Microsoft is actively investing in Cloud Sync, I decided to build my lab around the newer architecture.

That decision ended up shaping almost every troubleshooting step that followed.

Building the Environment

The lab started with a Windows Server 2022 Datacenter Azure Edition virtual machine. Before promoting it to a domain controller, I assigned it a static private IP address. Domain controllers rely heavily on DNS, and changing their IP address later can introduce unnecessary headaches, so it’s much easier to get this right from the beginning.

With networking configured, I installed the Active Directory Domain Services role and promoted the server to a new forest using the domain corp.contoso.local.

Once the server restarted, I created a simple directory structure with a few Organizational Units and test users. There was nothing complicated here. The goal was simply to have enough objects to verify that synchronization was working correctly later on.

I also verified that each user had the correct User Principal Name (UPN) configured. Hybrid identity depends on consistent identities between Active Directory and Microsoft Entra ID, so it’s worth checking this early instead of troubleshooting unexpected sign-in issues later.

The First Roadblock: DNS

The first issue appeared long before I installed the Cloud Sync agent.

When a server becomes a domain controller, it should use itself as its primary DNS server because Active Directory stores its DNS records locally. That’s standard practice and easy enough to configure.

The part I initially overlooked was outbound name resolution.

Although the server could resolve internal Active Directory records without any problems, it also needed to reach Microsoft Entra endpoints over the internet. Without an external DNS forwarder, those lookups failed, which meant the provisioning agent couldn’t complete its connectivity checks.

The solution was to configure a DNS forwarder that pointed to Azure’s built-in DNS service at 168.63.129.16. After adding the forwarder, external name resolution immediately started working.

To make sure everything was configured correctly, I verified external name resolution using nslookup and confirmed internet connectivity before moving on to the Cloud Sync installation.

It was a small configuration change, but it served as a reminder that hybrid identity depends on both worlds working together. The server wasn’t just talking to Active Directory anymore. It also needed reliable communication with Microsoft Entra services, and DNS was the bridge between the two.

The First Real Problem: The Sign-in Window Wasn’t Actually Using My Browser

With Active Directory running and DNS working correctly, it was finally time to install the Microsoft Entra Cloud Sync provisioning agent.

I expected this to be the easy part.

The installation completed without any issues, but the moment I tried to sign in to my Microsoft Entra tenant, the wizard displayed an unexpected message telling me that my browser was outdated and unsupported.

At first, that didn’t make any sense.

I was using Microsoft Edge, it was fully up to date, and I had no issues signing in anywhere else. I assumed there was something wrong with my browser configuration, so I started working through the usual troubleshooting steps.

I installed the WebView2 Runtime, added Microsoft’s sign-in pages to the Trusted Sites zone, disabled Internet Explorer Enhanced Security Configuration, and even experimented with Internet Explorer compatibility settings. Nothing changed. Every sign-in attempt failed with the same message.

After digging through the provisioning agent logs, the real problem finally became clear.

The setup wizard wasn’t using Microsoft Edge at all.

Instead, it was launching a legacy embedded Internet Explorer rendering engine to handle authentication. Modern Microsoft Entra sign-in pages, especially those protected with multi-factor authentication, simply don’t play nicely with that older authentication flow.

The logs confirmed exactly what was happening. Every attempt ended with an MsalClientException reporting authentication_canceled, along with entries showing that the installer was relying on the legacy embedded browser.

Once I understood the actual problem, the solution was surprisingly simple.

Instead of fighting the installer, I bypassed the graphical sign-in process completely and switched to Microsoft’s supported PowerShell installation method.

The provisioning agent was still the same. The only difference was how authentication was performed.

Rather than opening a browser window, PowerShell authenticated directly using credentials, avoiding the embedded browser entirely.

Start-Process 'ProvisioningAgentSetup.exe' /quiet -NoNewWindow -Wait
Import-Module "C:\Program Files\Microsoft Azure AD Connect Provisioning Agent\Microsoft.CloudSync.PowerShell.dll"
Connect-AADCloudSyncAzureAD -Credential $hybridAdminCreds
Add-AADCloudSyncGMSA -Credential $domainAdminCreds
Add-AADCloudSyncADDomain -DomainName corp.contoso.local -Credential $domainAdminCreds

To my surprise, everything worked on the very first attempt.

Sometimes the quickest solution isn’t fixing the broken component. It’s recognizing when there’s an alternative path that avoids the problem entirely.

A few moments later, the provisioning agent appeared in my Microsoft Entra tenant with a healthy status, confirming that the registration had succeeded.

With the agent connected, I created my first Cloud Sync configuration.

For this lab, I enabled Password Hash Synchronization and synchronized every user in the domain. In a production environment, I’d almost certainly scope synchronization to specific Organizational Units or security groups, but syncing everything kept the lab simple and made validation easier.

Confident that the difficult part was finally over, I clicked Start Provisioning and waited for the first synchronization cycle.

That’s when the next problem appeared.

The Second Problem: Everything Looked Healthy, but Nothing Was Syncing

With the provisioning agent successfully registered, I expected my test users to appear in Microsoft Entra ID within a few minutes.

Instead, nothing happened.

Provisioning on Demand kept reporting an InvalidCredentials error, which immediately made me suspect that I had misconfigured the group Managed Service Account (gMSA). Since the provisioning agent relies on the gMSA to communicate with Active Directory, it seemed like the most likely explanation.

My first step was to validate the service account itself.

Using Test-ADServiceAccount, I confirmed that the gMSA was installed correctly and functioning as expected. That ruled out what I initially thought was the problem.

At this point, I had learned an important lesson about troubleshooting: just because an error message points in one direction doesn’t mean that’s where the real problem is.

Instead of focusing on the Provisioning on Demand test, I opened the Cloud Sync configuration overview to see what the service itself was reporting.

That’s when I noticed something I had completely overlooked.

The Last cycle start time still showed Not started.

The synchronization engine had never actually run.

I had created the configuration, configured the synchronization scope, and saved everything correctly, but I hadn’t fully started the provisioning cycle. Those are two separate actions, and it’s surprisingly easy to assume that saving the configuration automatically begins synchronization.

Once I started the provisioning service, the configuration moved into its first synchronization cycle.

For a brief moment, it looked like everything was finally working.

Then the configuration entered quarantine with a new error:

AzureDirectoryServiceAuthorizationFailed

The gMSA had never been the real problem.

It had simply been the first clue in a much larger authentication issue that was happening between Microsoft Entra ID and the provisioning service itself.

The investigation wasn’t over yet.

The Root Cause: A Corrupted Service Principal

The new error gave me a much more useful starting point.

Instead of pointing toward Active Directory, it pointed toward Microsoft Entra ID itself.

The synchronization job had entered quarantine with the error:

AzureDirectoryServiceAuthorizationFailed

The message suggested that my tenant wasn’t correctly configured for directory synchronization, but that didn’t line up with everything I had already verified. I was signed in as a Global Administrator, the provisioning agent was registered successfully, and the gMSA had passed every validation check.

Something else was wrong.

I opened Enterprise Applications in Microsoft Entra ID and searched for the Cloud Sync application that had been created during the provisioning process.

That’s when I noticed something unusual.

The application had no admin-consented permissions.

Normally, granting admin consent is straightforward for a Global Administrator, but in this case the button was disabled. Even trying to grant consent through a direct admin consent URL resulted in another error:

AADSTS1003031: Misconfigured required resource access in client application registration

That error completely changed the direction of my investigation.

At this point, I stopped looking at my server and started questioning the objects that had been created inside Microsoft Entra ID.

My theory was that the repeated failed authentication attempts during the provisioning agent installation had left behind a partially configured service principal. After abandoning the installer and switching to the PowerShell registration method, the tenant was now trying to use an application object that had never been created correctly in the first place.

I couldn’t prove that theory with absolute certainty, but it matched every symptom I was seeing.

Rather than trying to repair an object that appeared to be corrupted, I decided to rebuild the environment from scratch.

I deleted the Cloud Sync configuration, removed the associated Enterprise Application, uninstalled the provisioning agent, and then installed everything again using the PowerShell method from the beginning.

The new provisioning agent registered successfully, a fresh service principal was created, and the environment looked significantly healthier than before.

It was one of those moments where starting over wasn’t admitting defeat. It was simply the fastest way to eliminate every questionable variable and return to a known-good state.

Even after rebuilding the environment, the authorization error persisted for a while.

After digging through Microsoft’s documentation, I found that backend propagation for repaired synchronization configurations can take time. In other words, not every issue is immediately under your control, even when everything is configured correctly.

That was an important lesson in itself.

When troubleshooting cloud services, there comes a point where continuing to change your own configuration only introduces more variables. Sometimes the best decision is to verify everything you can control, document the current state, and give the platform time to finish its work.

Lessons Learned

This project ended up teaching me far more than I expected.

I started the day thinking I was learning how to configure Microsoft Entra Cloud Sync.

Instead, I learned how to troubleshoot it.

I saw how important DNS is in a hybrid identity environment, discovered that the provisioning agent’s embedded browser can become a problem in modern authentication scenarios, gained a much better understanding of how group Managed Service Accounts fit into the synchronization process, and learned that service principals are just as important to investigate as the servers running the workloads.

More importantly, I was reminded that error messages don’t always identify the real problem.

The InvalidCredentials error wasn't caused by invalid credentials.

The synchronization issue wasn’t caused by Active Directory.

The real problem was an incomplete identity object inside Microsoft Entra ID, something I wouldn’t have found if I had continued focusing only on the domain controller.

That’s probably the biggest takeaway from this project.

Effective troubleshooting isn’t about trying random fixes until something works. It’s about following the evidence, validating assumptions one layer at a time, and being willing to change direction when the facts point somewhere new.

Hybrid identity involves Active Directory, DNS, Kerberos, Microsoft Entra ID, service principals, provisioning agents, and cloud services all working together. When one layer breaks, understanding how those pieces connect is far more valuable than memorizing a list of troubleshooting commands.

That understanding is what I took away from this project, and it’s something no step-by-step tutorial could have taught me.