This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.
Intro: What is CEF collection
Common Event Format (CEF) is an industry standard format on top of Syslog messages, used by many security vendors to allow event interoperability among different platforms. Azure Sentinel provides the ability to ingest data from an external solution. If your appliance or system enables you to save logs as Syslog Common Event Format (CEF), the integration with Azure Sentinel enables you to easily run analytics, and queries across the data.
The number of systems supporting Syslog or CEF is in the hundreds, please make sure to check out the Azure Sentinel: Syslog and CEF source configuration grand list.
In this blogpost, we will provide details on how CEF collection works and the best practices you should consider when configuring common event format collection in Azure Sentinel.
How does it work?
CEF Collection in Azure Sentinel works by selecting or creating a Linux machine that Azure Sentinel will use as the proxy between your security solution and Azure Sentinel. The Linux machine can be on your on-prem environment, Azure or other clouds. In addition, the Azure Sentinel syslog agent, based on the Microsoft monitoring agent is required to be installed on the Linux machine to collect data and send it securely to your Azure Sentinel workspace.
The following flow chart details the high-level steps to configure CEF collection in Azure Sentinel:
Note: For additional information on how CEF collection works, please visit:
https://docs.microsoft.com/en-us/azure/sentinel/connect-common-event-format
For data security in Log analytics see https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-security
CEF Best Practices:
1. Secure your machine
Make sure to configure the machine's security according to your organization's security policy. For example, you can configure your network to align with your corporate network security policy and change the ports and protocols in the daemon to align with your requirements. You can use the following instructions to improve your machine security configuration: Secure VM in azure, Best practices for Network security.
Note: To use TLS communication between the security solution and the Syslog machine, you will need to configure the Syslog daemon (rsyslog or syslog-ng) to communicate in TLS: Encrypting Syslog Traffic with TLS - rsyslog, Encrypting log messages with TLS – syslog-ng.
2. Properly Secure an Azure VM with Public IP
Common Event Format uses UDP 514 therefore messages will be sent in clear text. If we use an unencrypted connection and a third party intercepts the connection, they can see all the information exchanged in plain text. TLS can protect this communication traffic. Additionally, a bad actor can potentially intercept and send messages to your Azure VMs to create false records. Make sure to protect the Azure VM with either a Network Security Group or NVA. On the NSG, you should configure a list of source IP addresses you want to allow to communicate to the VM.
3. Deploy CEF collector machine close to source
Let’s review the common scenarios:
On-prem source – We recommend you deploy the CEF collector on-prem. Syslog CEF is default sent over UDP port 514 in plain text. Once it reaches the CEF collector, it is sent to Azure Sentinel using HTTPs. Deploying on-prem ensure your data is not sent in the clear outside your network.
If on-prem deployment is not an option at all, we recommend you deploy in a cloud VM and use TCP over TLS or ensure there is a secure path (VPN) between the source and the collector VM.
Cloud source – We recommend you deploy the CEF collector in the same virtual network as the source. You can also use VNet peering to have multiple VNets connected to the CEF collector. This again prevents your traffic from being sent over the internet in the clear.
4. Do not configure the source to send to 2 CEF
To avoid duplicate records/messages in Azure Sentinel do not configure your third-party security appliance to send data to 2 Common Event Format collectors. Additionally, sending duplicate data will increase your consumption bill therefore ensure that your security appliances are set to send data to a single CEF collector. You may consider using a load balancer or rsyslog cluster (https://www.rsyslog.com/load-balancing-for-rsyslog/) as an aleternative.
5. TCP is the default protocol, always use
Syslog CEF can support UDP or TCP. However, TCP is now the default protocol and should always be used unless the source device does not support TCP. The advantage is the reliability of the TCP protocol.
Querying the Data
Once you satisfy the configuration steps in the Common Event Format data connector, you can then query the data. CEF logs will reside within the CommonSecurityLog table, hence by querying for CommonSecurityLog you will have visibility to your CEF data.
Now that we can see the data in Azure Sentinel, we now can build workbooks, analytic rules, hunting queries, or associate it with other data for correlation.
Conclusion
In this blog, you learned how Common Event Format collection works and the best practices to consider when configuring CEF collection in Azure Sentinel. To learn more about Azure Sentinel, see the following articles:
Learn how to get visibility into your data, and potential threats.
Get started detecting threats with Azure Sentinel.