Azure Sentinel To-Go! A Linux 🐧 Lab with AUOMS Set Up to Learn About the OMI Vulnerability 💥

This post has been republished via RSS; it originally appeared at: Azure Sentinel articles.

main_image.PNG

 

Last week, on September 14th, 2021, Microsoft released fixes for three Elevation of Privilege (EoP) vulnerabilities CVE-2021-38645CVE-2021-38649CVE-2021-38648and one unauthenticated Remote Code Execution (RCE) vulnerability CVE-2021-38647 .  

These vulnerabilities affect the Open Management Infrastructure (OMI), an open-source project to further the development of a production quality implementation of the DMTF CIM/WBEM standards. The OMI Common Information Model Object Manager (CIMOM) is also designed to be portable and highly modular. It is written in C and the code is available in GitHub.

 

Great Resources to Read First 

The following resources have already been shared by Microsoft to provide guidance on updating vulnerable extensions for Cloud and On-Premises deployments, and indicators to detect the exploitation of the vulnerability: 

 

What is this about? 

In this post, I will show you how to automatically deploy a research lab environment with Azure Sentinel , a few Linux virtual machines and the Microsoft Audit Collection Tool (AUOMS) set up to understand the underlying behavior of the exploitation of the OMI vulnerability.

 

This is an extension of the amazing work shared by MSTIC through the following resources: 

 

Before going through a few concepts and the deployment process, remember that this vulnerability is actively being exploited. Therefore, make sure you do not expose your lab environment to the Internet.

 

Microsoft Audit Collection Tool (AUOMS) 

AUOMS is a Microsoft audit collection tool that can collect events generated by the Linux kernel’s audit subsystem, kaudit, and the optional user-space daemon, auditd. This allows, for example, the collection of syscalls events such as process creations, file access, and other valuable telemetry for research.

 

AUOMS is part of the installation of the Log Analytics Agent for Linux, also known as the  Operations Management Suite (OMS) Agent for Linux, which allows the streaming of events from Linux-based, syslog supporting devices into Azure Sentinel. However, AUOMS is not set up by default as shown below: 

 

auoms_not_enabled.png

 

My colleague Kevin Sheldrake documented everything that is required to set it up in this blog post Hunting Threats on Linux with Azure Sentinel.

 

The question is “How do we automate the whole setup?

 

Enter Azure Sentinel To-go! 

Azure Sentinel2Go is an open-source project developed to expedite the deployment of an Azure Sentinel lab along with other Azure resources and a data ingestion pipeline to consume pre-recorded datasets for research purposes.

 

Azure Sentinel + Linux Environment 

Currently, we have a Linux environment ready to go and deploy everything needed for a small research lab with AUOMS configured and sending logs to Azure Sentinel:

 

Azure-Sentinel2Go/grocery-list/Linux at master · OTRF/Azure-Sentinel2Go (github.com) 

 

We were able to use Azure Resource Manager (ARM) templates and a few bash scripts to automate the whole setup. These are all the resources used for each component of the lab: 

 

What about the OMI Vulnerability? 

As we know, older versions of the OMI agent (< 1.6.8.1) are vulnerable. Therefore, we created the following script to install version 1.6.8.0, and open port 5986.

 

Blacksmith/Install-OMI.sh at master · OTRF/Blacksmith (github.com)

 

We added that script to the Linux lab templates, and we now have a demo environment that you can also use to learn more about the exploitation of the OMI vulnerability. 

 

Azure-Sentinel2Go/grocery-list/Linux/demos/CVE-2021-38647-OMI at master · OTRF/Azure-Sentinel2Go (github.com) 

 

Deploying the Lab Environment 

Remember that this vulnerability is actively being exploited. Therefore, make sure you do not expose your lab environment to the Internet.

  • Click on the “Deploy to Azure” Button 

 

linux_lab_deploy_button.png

 

  • Fill out the following parameters: 
    • Subscription (selected by default) 
    • Resource group 
    • Region (selected by default) 
    • Admin Username 
    • Admin Password 
    • Remote Access Mode (AllowPublicIP selected by default. You can also use Azure Bastion Host. You would just need to set the Allowed IP Addresses parameter to *) 
    • Allowed IP Addresses (If you use the default access mode AllowPublicIP, use your home or office public IP address to only allow access from secure places. Remember that this vulnerability is actively being exploited. Therefore, make sure you do not expose your lab environment to the Internet.) 

 

deploy_environment_parameters.png

 

  • Click the Review > Create buttons to start the deployment 

 

deployment_in_progress.png

 

  • You can go to your resource group and explore all the resources being deployed

 

resources_being_created.png

 

  • Wait around 5-10 minutes! You should be good to go! 

 

Validate Deployment 

It is very important to validate if everything was deployed properly before doing research. 

 

OMI Server 

SSH to your virtual machines and check the OMI version to confirm it is 1.6.8-0 

 

/opt/omi/bin/omiserver -v 

 

validate_omi_server_version.png

 

Check if the OMI service is running 

 

systemctl status omid 

 

validate_omid_server_is_running.png

 

Check if port 5986 is open (You might have to update your package manager and install net-tools) 

 

netstat -na | grep :5986 

 

validate_omi_5986_port.png

 

AUOMS Setup 

Check if the AUOMS service is running with the following two commands: 

 

sudo /opt/microsoft/auoms/bin/auomsctl status 

 

validate_auoms_server_is_running.png

 

systemctl status auoms 

 

validate_auoms_server_is_running_2.png

 

Check if events are being sent to the OMS Agent: 

  • Open another SSH session to your virtual machine and in one run the following command:

 

sudo /opt/microsoft/auoms/bin/auomsctl monitor 

 

  • Then, in the other session run whoami. If everything is connected properly, you will be able to see events flowing through your first session as shown below: 

 

validate_auoms_can_send_events.png

 

You can continue using `sudo /opt/microsoft/auoms/bin/auomsctl monitor` if you want to do research locally. You can have it running while you test the exploitation of the OMI vulnerability. 

 

Azure Sentinel 

Check if logs are being sent to your Azure Sentinel instance. 

 

validate_azure_sentinel.png

 

  • Click on `logs` and explore the `Syslog` table 

 

validate_syslog_events_are_flowing.png

 

Learning About the OMI vulnerability 

After validating that everything was deployed properly, you should be ready to run a few public proofs of concepts to test the OMI vulnerability.

 

One thing to remember is that there are three ways to execute arbitrary code via OMI. They are all part of the SCX RunAsProvider and their execution context varies a little bit. 

 

Run Public POC (ExecuteShellCommand) 

  1. SSH to machine A 
  2. SSH to machine B
  3. On machine A, prepare the data that you want to send to machine B 
    1. Request without authorization header
    2. Set the command you want to execute. For this example we execute “id”.
    3. Use the ExecuteShellCommand method in data.
  4. Send HTTP request 

 

execute_poc_executeshellcommand.png

 

Explore data in Azure Sentinel (ExecuteShellCommand) 

You can run the following hunting query to explore the execution context:

 

Syslog 
  | parse SyslogMessage with "type=" EventType " audit(" * "): " EventData 
  | where EventType =~ "AUOMS_EXECVE" and EventData has '/var/opt/microsoft/scx/tmp' 
  | project TimeGenerated, EventType, Computer, EventData 
  | parse EventData with * "syscall=" syscall " syscall_r=" * " success=" success " exit=" exit " a0" *
" 
ppid=" ppid " pid=" pid " audit_user=" audit_user " auid=" auid " user=" user " uid=" uid " group=" group "
gid=" gid "
effective_user=" effective_user " euid=" euid " set_user=" set_user " suid=" suid
" 
filesystem_user=" filesystem_user " fsuid=" fsuid " effective_group=" effective_group " egid=" egid 
" 
set_group=" set_group " sgid=" sgid " filesystem_group=" filesystem_group " fsgid=" fsgid " tty=" tty 
" 
ses=" ses " comm=\"" comm "\" exe=\"" exe "\"" * "cwd=\"" cwd "\"" * "name=\"" name "\"" * "cmdline=\"" cmdline "\"" *
 
  | where uid == '0' 
  | where cwd == '/var/opt/microsoft/scx/tmp' 
  | where comm == 'sh' 
  | extend Timestamp = TimeGenerated, HostCustomEntity = Computer, AccountCustomEntity = user 

 

executeshellcommand_kql_query.png

 

We observed that the execution was happening from the `current working directory (cwd): /var/opt/microsoft/scx/tmp`. This is an indicator that repeats across the other two methods to execute arbitrary code abusing the OMI vulnerability. Group the results by the command line values to identify initial outliers.

 

Run Public POC (ExecuteScript) 

  1. SSH to machine A 
  2. SSH to machine B
  3. On machine A, prepare the data that you want to send to machine B 
    1. Request without authorization header 
    2. Set the script you want to execute: 
      1. Pick a command. Let's say whoami 
      2. Base64 encode the command: d2hvYW1p 
    3. Use the ExecuteScript method in data. 
  4. Send HTTP request

 

execute_poc_executescript.png

 

Explore data in Azure Sentinel (ExecuteScript) 

You can run the previous hunting query again and explore the results. You will see that the current working directory (cwd) is the same, but the command line or in this case the script is now being hosted at the following directory:  /etc/opt/microsoft/scx/conf/tmpdir/.  The name of the scripts in that directory has the string “scx” as a prefix. For example: scxzEPOS4. 

 

Syslog 
  | parse SyslogMessage with "type=" EventType " audit(" * "): " EventData 
  | where EventType =~ "AUOMS_EXECVE" and EventData has '/var/opt/microsoft/scx/tmp' 
  | project TimeGeneratedEventType, Computer, EventData 
  | parse EventData with * "syscall=" syscall " syscall_r=" * " success=" success " exit=" exit " a0" * " 
ppid=" ppid " pid=" pid " audit_user=" audit_user " auid=" auid " user=" user " uid=" uid " group=" group
" gid=" gid "
effective_user=" effective_user " euid=" euid " set_user=" set_user " suid=" suid
filesystem_user=" filesystem_user " fsuid=" fsuid " effective_group=" effective_group " egid=" egid 
set_group=" set_group " sgid=" sgid " filesystem_group=" filesystem_group " fsgid=" fsgid " tty=" tty 
ses=" ses " comm=\"" comm "\" exe=\"" exe "\"" * "cwd=\"" cwd "\"" * "name=\"" name "\"" * "cmdline=\"" cmdline "\"" *
 
  | where uid == '0' 
  | where cwd == '/var/opt/microsoft/scx/tmp' 
  | where comm == 'sh' 
  | extend Timestamp = TimeGeneratedHostCustomEntity = Computer, AccountCustomEntity = user 

 

executescript_kql_query_execution.png

 

 

I was wondering what process had created that file in that directory. I ran the following query to answer that question:

 

let syscallsList = dynamic(["unlink","openat","chmod"]); 
Syslog 
| parse SyslogMessage with "type=" EventType " audit(" * "): " EventData 
| where EventType =~ "AUOMS_SYSCALL" and EventData contains "/etc/opt/microsoft/scx/conf/tmpdir/" 
| project TimeGenerated, EventType, Computer, EventData 
| parse EventData with * "syscall=" syscall " syscall_r=" * " success=" success " exit=" exit " a0" *
" 
ppid=" ppid " pid=" pid " audit_user=" audit_user " auid=" auid " user=" user " uid=" uid " group=" group
" gid=" gid "
effective_user=" effective_user " euid=" euid " set_user=" set_user " suid=" suid
" 
filesystem_user=" filesystem_user " fsuid=" fsuid " effective_group=" effective_group " egid=" egid 
" 
set_group=" set_group " sgid=" sgid " filesystem_group=" filesystem_group " fsgid=" fsgid " tty=" tty 
" 
ses=" ses " comm=\"" comm "\" exe=\"" exe "\"" * "cwd=\"" cwd "\"" * "name=\"" name "\"" *
" 
path_name=" path_name " path_nametype=" path_nametype " path_mode=" * " proctitle=" cmdline " redactors=" *
 
| where syscall in (syscallsList) 
| extend fileAction = (parse_json(path_nametype))[1] 
| where fileAction in ("CREATE","DELETE") 

 

executescript_kql_query_file_creation_deletion.png

 

It seems that the omiagent creates and deletes the file. The file is available only during the execution of the script. Once the execution is done, the file gets deleted. After doing some more research and reading some of the SCXCore code in GitHub, this the behavior of the ExecuteScript method: 

 

executescript_behavior.png

 

That’s it! I am sure there is more to explore! I hope this lab environment can help you to test a few things in a safer way and experience what it might look like if it happens in your environment.

 

Once again, we highly recommend upgrading the OMI agent to version 1.6.8-1+, and if possible, controlling access to ports 5986,5985 and 1270.  Remember that this vulnerability is actively being exploited. Therefore, make sure you do not expose your lab environment to the Internet.

 

In addition, remember that the behavior documented in this post is not malicious. The lab was created to help us understand how the execution of commands or a script was being handled by OMI from a data perspective. You must then go through the results of those queries and validate what is legitimate behavior or not depending on your organization’s baseline. 

 

Also, use this knowledge to map data you collect to every single action documented ;) You might be collecting data from other sources that provide the same or similar visibility.  

 

Resources

 

References 

 

REMEMBER: these articles are REPUBLISHED. Your best bet to get a reply is to follow the link at the top of the post to the ORIGINAL post! BUT you're more than welcome to start discussions here:

This site uses Akismet to reduce spam. Learn how your comment data is processed.