This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.
Brief Intro to Windows Logs
Logs are constantly recording what is going on in a machine. Logs can provide help in tracking security related events, can be used for auditing, troubleshooting and more. In Windows, logs that are saved contain information about applications and the operating system itself. Moreover, these logs are structured and human-readable. For viewing the logs, Windows uses its Windows Event Viewer.
There are five main categories under Windows logs:
- System - Logs created by the operating system
- Application- Logged by an application hosted locally
- Setup - Logs created in the process of installing or changing the Windows installation
- Security - Logs related to logins, privileges, and other similar events
- Forwarded Events - Events forwarded by other computers
How can a SOC team ingest and analyze Windows Logs with Microsoft Sentinel? What are the main options to ingest Windows Logs into a Log Analytics Workspace and use Microsoft Sentinel as a SIEM to manage security incidents from events recorded on these logs?
Read on to find out!
There are three main options to ingest Windows Logs into a Log Analytics Workspace for Monitoring:
In Sentinel, the Windows Security Events via AMA Content Hub solution pulls the Windows Log, Application, Security, Setup and System shown on the Event Viewer, Snippet above to be ingested into a Sentinel Log Analytics Workspace.
Use this article as a guide on how to enable this Content Hub Solution:
Connect using the Windows Security Events via AMA Connector - Training | Microsoft Learn
The SecurityEvent table is generated by solutions that fall into Security and Audit (Defender for Cloud) and SecurityInsights (Microsoft Sentinel). \
This table, SecurityEvent, collects using log level and only collects logs that are security related. It will not collect performance and some process logs. Microsoft Sentinel data collection rule for Windows Security Events collects log levels. This data connector collects event IDs pre-identified in log level. You can choose ALL, Common, Minimal and Custom. Custom allows you to filter Events further using XPath queries to pull for specific logs.
The Snippet below shows the subset of Event IDs that are collected by the Windows Security Events via AMA Data Connector. Notice, it doesn’t collect Forwarded Events when that machine is configured as a Windows Event Forwarder.
Please see the schema of the SecurityEvent table here.
The data connector shown in the image above - Windows Security Events via AMA, along with the data connector Security Events via Legacy Agent data connector, along with Analytic Rules and Workbooks related to these sources, are all distributed using the Windows Security Events Content Hub Solution for Sentinel.
There is a second type of connector for a very specific use case, the Windows Event Forwarder. This data connector can be used when there is a constraint related to accessing all the servers to be monitored, for example, the servers are part of a managed cloud, not accessible by the SOC team, or there are other constraints related to the deployment of the Azure Monitoring Agent at scale for a large set of Windows Servers. In this case you can request to your managed service provider to create a Windows Event Forwarder with a Windows Event Collector (WEC) and install the AMA Agent only on the Collector that has the Forwarded Events logs (see first image on this article, the Forwarded Events show on a Windows Server OS that is configured as a Windows Event Fowarding Log Collector). This blog post offers more details on how to configure a Windows Event Collector.
The WindowsEvent table shows up in the Log Analytics Workspace when you deploy the Azure Arc and Azure Monitoring Agent on your Windows Event Forwarder server to accommodate disconnected machines.
- This LAW table has a similar logging format as the SecurityEvent table except the options for filtering the types of events collected from the Forwarder are All Events or Custom only.
- The main trade off is this table, though it will have the same information as SecurityEvent, it will not unpack all the data into separate columns like SecurityEvent does; there isn’t a 1-2-1 parity between the WindowsEvent and SecurityEvent table. For that, as a Security Analyst, you will need to parse more the events recorded under Windows Events using KQL.
- System administrators need to configure the Windows Event Forwarder and the Collector. Configure Windows Event Forwarding in Advanced Threat Analytics | Microsoft Learn
- See the WindowsEvent table schema documented here
We recommend using ASIM - Advanced Security Information Model (ASIM) security content | Microsoft Learn to parse and normalize these logs. To normalize the logs collected via a Windows Event Forwarder Log Collector.
It is a common misconception that by installing the Windows Security Events via AMA Connector you will get the Forwarded Events logs collected at a Windows Event Collector ingested into a LAW without the Windows Forwarded Events data connector.
Be aware of this gotcha.
In the case of Windows Forwarded Events Content Hub solution, it only, by default, deploys 2 Analytic rules; and this make it difficult for SOC Analysts to conduct an accurate crosswalk using the MITRE Att&ck Framework dashboard.
Using Azure Monitor and a Data Collection Rule to collect Windows Logs:
One additional way to collect logs from logs from a Windows Server or set of servers, without using Sentinel, is by using the Events table and Azure Monitor Data Collection Rules. In this case the Event table is populated using the LogManagement configuration whether from Microsoft Monitoring Agent natively ingesting these events or from creating a Data Collection Rule (DCR) from Azure Monitor, as opposed of the two Security Solutions previously discussed that require Microsoft Sentinel.
- This configuration does not parse out Security data and may/will have non security related data like performance ingested in it by default.
- These logs are not considered high density or high fidelity to detect security events, do threat hunting etcetera.
- This is NOT recommended to use when sending logs to Security Solutions in Azure. The recommended path is to use the DCR built into Sentinel so that the Security logs are properly parsed.
Wrapping up:
On this article we covered three options to collect Windows Events into a Log Analytics Workspace, what options are considered better for collecting Windows Security Events: Security Events via AMA and Windows Event Forwarder. How the use of the Windows Event Forwarder requires the installation of the AMA agent on the Windows Event Forwarder Collector and how this option ingests these logs in the WindowsEvent table. This last content hub solution offers less reactive monitoring out of the box in Sentinel with fewer Analytic Rule templates thus forcing the SOC Analysts to parse the data collected further to pin-point security events related to security incidents. We discussed a common gotcha, that by installing the Security Events via AMA data connector in Sentinel the Forwarded Events in a properly configured Windows Forwarder Event Collector were ingested into the SecurityEvent LAW table. We finally mentioned a solution to ingest Windows Events into a LAW using Azure Monitor, the AMA agent, and a Data Collection Rule, ingesting these logs into the Event table. This last solution will result in the collection of low density and low fidelity events from the security perspective, as it may also contain OS performance events, making this last solution far from ideal to monitor and detect security incidents.
Co-authors: @Andre_Murrell , @Simona_Kovatcheva , @lizetpena