Standalone Service Fabric Integration with Azure Log Analytics

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.

The Azure Log Analytics agent, previously referred to as the Microsoft Monitoring Agent (MMA) or OMS agent. These agents attach to an Azure Monitor and store collected log data from different sources in your Log Analytics workspace, as well as any unique logs or metrics as defined in a monitoring solution. In three simple steps, in this article we will configure Service Fabric Log Analytics workspace to retrieve performance counter for Service Fabric (performance /infrastructure monitoring) and events for system, application and Service Fabric (cluster/platform monitoring). Further we will create alert in Azure Log Analytics when node gets down.

 

In other option, you can achieved this via EventFlow (https://github.com/Azure/diagnostics-EventFlow). Write a small app that listens for Service Fabric ETW events and forwards them to app insights / event hub / more options. For Application logs you have to write you own code and generate logs. These can also be pushed to Log Analytics using custom logs. https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-sources-custom-logs .

 

1) Create the Service Fabric Log Analytics workspace

https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-diagnostics-oms-setup#deploy-a-log-analytics-workspace-by-using-azure-marketplace

blogimage1.png

 

2) Install the agent on-premises Virtual Machines (If you have a 3 Node/ 5 Node cluster then this agent needs to be installed on each Virtual Machine).

 

Download:

Windows 64-bit agent - https://go.microsoft.com/fwlink/?LinkId=828603 

blogimage2.png

Once the agent is downloaded, you can go to install the agent using the setup wizard

blogimage3.png

blogimage4.png

Check if connected:

blogimage5.png

Note that even after installing and connecting the agent to the Log Analytics workspace, to get certain performance counter/events related to Service Fabric, you would need to manually add some additional performance counters/events in the Log analytics workspace. For this refer to step 3.

 

Check Service Fabric performance counters on-premise virtual machine. List of performance counter https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-diagnostics-event-generation-perf

blogimage6.png

Check Service Fabric events on-premise virtual machine. List of events https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-diagnostics-event-generation-operational

blogimage7.png

 

3) Config the Azure Log Analytics workspace on the Azure Portal

 

You would need to go to Portal > Log Analytics  Worksapce > “Workspace” > Advanced Settings > Data > Windows Performance data.  After that in the “Collect the following performance counters > Search bar, you would need to add the below counters” or any other counter you wish to monitor. They may not be shown in the auto-complete, but you can add them and save the changes.

 

Service Fabric Naming Gateway(*)\Client Connections

Service Fabric Reconfiguration Agent(*)\# FTs

Service Fabric Native Store\”CounterName”

 

blogimage8.png

 

blogimage9.png

 

In addition, you can also send event viewer traces (ETW) for application and system events and performance counter refer to below screens. More details at : https://docs.microsoft.com/en-us/azure/azure-monitor/platform/agent-data-sources#configuring-data-sources.

 

Further you can add Service Fabric events for Monitoring.

Add Service Fabric Events:

Microsoft-ServiceFabric/Admin

Microsoft-ServiceFabric/Operational

 

blogimage10.png

blogimage11.png

Validation:

Wait for sometime around 15-20 minutes and run the below query in the Log Search : Perf | where ObjectName contains “Service Fabric Reconfiguration Agent” and check for other counter.

 

Performance Counter (performance /infrastructure monitoring):

blogimage14.pngblogimage15.png

Events (cluster/platform monitoring):

blogimage13.png

Configure an Alert:

Click "New Alert Rule" at the top and now anytime an event arrives based on this query, you will receive an alert in your chosen method of communication.

This will create an alert when a node goes down:

Event

| where EventLevel == 1 and ParameterXml contains "NodeDown" and EventLog == "Microsoft-ServiceFabric/Operational"

blogimage12.png

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.