This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .
Case Background Information:
My customer needed a way to manage their on-premises Windows and Linux servers as well as some non-Azure servers in the Azure Portal. This customer needed to be able to monitor server performance, update servers, manage compliance and many other Azure Management capabilities all in one place; not on premises but manage them all in the cloud. This customer selected to use the Azure Arc capabilities in Azure for these requirements.
In a nutshell, Azure Arc is a centralized way to manage your existing non-Azure and/or on-premises resources in Azure Resource Manager. If you want an easy way to manage Windows servers, Linux servers, Kubernetes clusters, VMware servers, AWS servers, GCP servers, Azure Arc can provide the way. In this article, we are going to specifically discuss Azure Arc-enabled servers and a specific troubleshooting case with the Azure Arc Connected Machine Agent.
What the Eyewitnesses Reported:
They needed to onboard their server to Azure Arc. They navigated to Azure Arc in the Azure Portal. From the Overview, they selected Add (Add your infrastructure for free)
They then clicked on Add (Servers)
Then they clicked on Generate Script (Add a single server)
This customer then filled out what they needed for the Add a server with Azure Arc wizard. They put the Resource Group as the one that they are using for Azure Arc enabled servers. They selected the Region where the servers reside. They selected Windows but Linux is also an option. For connectivity method, they selected public endpoint but are looking into private endpoint. They do want to enable Automanage especially since the Microsoft Antimalware for Azure is free. However, for now, the customer left blank because they just want to get the server connected.
They then downloaded the script.
After that, they copied it to the server that they were onboarding to Azure Arc. They ensured that the requirements were met; the customer was a local admin on the server, the server had access to port 443 and final the set of outbound URLs for Azure Arc were also accessible.
They ran the script above in PowerShell as an administrator.
Azure Arc Agent Gone Rogue:
And then….the on-premises Windows server showed up in Azure Arc/Infrastructure/Machines but with an Arc agent status as DISCONNECTED. The customer tried all the troubleshooting actions listed here:
Troubleshoot Azure Connected Machine agent connection issues - Azure Arc | Microsoft Learn
Tried uninstalling it using actions from here:
No matter what the customer tried, nothing would change the status to CONNECTED.
Scorched Earth for the WIN:
In the session we had with the customer, we first verified that the Azure Arc agent status on the server in question and if the Azure Arc endpoints were available:
- In PowerShell or a Command Terminal as an admin, type azcmagent show
- This command will display the agent status
- Check to see if the Agent Service, Extension Service and GC Service are running
- Next, type azcmagent check
- This command will run network connectivity checks for the Azure Arc endpoints
Next, we decided to uninstall and reinstall the Azure Arc client using the steps in the URLs above. However, after many attempts, the server was still showing up as Disconnected.
We then hit upon a scorched earth approach; meaning delete any local files from Azure Arc on the server as well as stopping any associated processes.
Uninstall Azure Connected Machine Agent (Programs and Features)
Use with caution (some files may not exist)
Files (File Explorer) – delete these if they exist
C:\Packages\
C:\Resources\
C:\Program Files\AzureConnectedMachine\
C:\ProgramData\AzureConnectedMachineAgent\
C:\ProgramData\GuestConfig\
Processes (Task Manager) – stop these if they exist
AMAExHealthMonitor.exe
MonAgentManager.exe
MonAgentLauncher.exe
MonAgentHost.exe
Microsoft.Online.Reporting.MonitoringAgent.XXX
Azure Arc Connected Machine Agent Tamed:
After these items have been completed, simply try to connect to Azure Arc via azcmagent command
azcmagent connect –resource-group “RG-Azure-Arc-ConnectMe” –tenant-id “insert yours here” –location “West US 3” –subscription-id” insert yours here”
Check in 30 minutes and Voila! Azure Arc machines show as Connected in the Azure Portal.
Azure Arc Connected Machine Agent Case Appendix – Dig Deeper:
Dig into the Azure Arc logs as well for more detailed information on what is going on with the agent – Check out the information on how to enable agent verbose logging here: Troubleshoot Azure Connected Machine agent connection issues - Azure Arc | Microsoft Learn
For an excellent in-depth reference on Azure Arc architecture, security considerations and more, please check out this blog post by Sonia Cuff The care and feeding of Azure Arc for Servers - Microsoft Community Hub