This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.
Abstract
In this article, we describe how the newly added support for custom execution hooks in NetApp Astra Control Service® (ACS) enables you to freeze/quiesce database transactions for taking application consistent snapshots. We demonstrate how to add custom pre- and post- scripts for MongoDB® managed by Astra Control Service and create application consistent snapshots of the database.
Co-authors: Sayan Saha and Patric Uebele, NetApp
Introduction
NetApp Astra Control is a solution that makes it easier to manage, protect, and move data-rich Kubernetes workloads within and across public clouds and on-premises. Astra Control provides persistent container storage that leverages NetApp’s proven and expansive storage portfolio in the public cloud and on premises. It also offers a rich set of advanced application-aware data management functionality (like snapshot and revert, backup and restore, activity logs, and active cloning) for local data protection, disaster recovery, data audit, and mobility use cases for your modern apps. Astra Control provides complete protection of stateful Kubernetes applications by saving both data and metadata, like deployments, config maps, services, secrets, that constitute an application in Kubernetes. Astra Control can be managed via its user interface, accessed by any web browser, or via its powerful REST API.
For a set of validated applications (MySQL, MariaDB, PostgreSQL, and Jenkins), Astra Control Service already includes the necessary hooks to guarantee application consistent snapshots and backups. The update of Astra Control in December 2021 allows us to add custom scripts to be executed before and after taking snapshots of applications managed by Astra Control. With Owner, Admin, or Member permissions in ACS, we can define custom execution hooks for non-validated applications. A few templates for execution hook scripts can be found in the ACS documentation.
Astra Control has two variants:
- Astra Control Service (ACS) – A fully managed application-aware data management service that supports Azure Kubernetes Service (AKS), Azure Disk, and Azure NetApp Files® (ANF).
- Astra Control Center (ACC) – application-aware data management for on-premises Kubernetes clusters, delivered as a customer-managed Kubernetes application from NetApp.
Scenario
In this article we will use MongoDB to demonstrate the functionality of adding custom scripts to be executed before and after taking a snapshot of the database. The MongoDB documentation recommends to flush writes to disk and lock the database to prevent further writes (using the db.fsyncLock(); method) before taking a snapshot and unlock the database (db.fsyncUnlock();) after the snapshot completes.
Based on the templates for custom execution hooks in the ACS documentation, we write a simple script to lock and unlock the MongoDB database, add the script as pre- and post-snapshot execution hooks to ACS, and test its functionality.
Preparation
First, we register an AKS cluster demo-aks-pu to our Astra Control Service account and set the default storage class (provisioner disk.csi.azure.com) as default storage class:
Next, install a MongoDB instance in namespace mongodb1 via helm chart:
and manage the discovered application with ACS:
Based on the execution hooks framework examples in the ACS documentation and the snapshot procedure in the MongoDB manual, the below script with arguments pre or post can be used both for a pre- and post-snapshot hook for MongoDB:
Store the script on your local system, we’ll upload it to ACS later.
Adding custom execution hooks to MongoDB app
Going to Applications and selecting the mongodb1 app, we can manage its execution hooks settings in the Execution hooks tab. As a non-validated app, there are no execution hooks provided by NetApp for MongoDB. Take a note of the container images displayed and click Add new hook:
Now we can enter the details for the respective custom execution hooks.
Pre-snapshot custom hook
To use the hooks_mongodb.sh script as pre-snapshot hook, we fill in the Add execution hook form accordingly:
- Select Pre-snapshot.
- Give the hook a name: prehook-mongodb1. Note: The hook name must be unique across all hooks and managed applications.
- Enter pre as argument to be passed to the hook script. Hit Enter after entering the argument to record it.
- Enter the container image name scheme for the mongodb container in the Container image names to match field. We can use regular expressions to describe the matching images (like mongodb:4* in our case).
- Select the Upload file option to upload the hooks_mongodb.sh script and upload it.
- The name of the uploaded script is displayed.
- Change the suggested script name to prehook_mongodb1.sh Note: This script name must be unique across all managed apps and hooks.
- Click Add hook.
The pre-snapshot hook will be added and enabled automatically. In the Execution hooks tab, we can check that the pre-snapshot hook will be applied to the correct container image and start adding the post-snapshot hook by selecting Add hook one more time:
Post-snapshot custom hook
To use the same hooks_mongodb.sh script as post-snapshot hook, we fill in the Add execution hook form again, this time with post as hook argument:
- Select Post-snapshot.
- Give the hook a name: posthook-mongodb1. Note: The hook name must be unique across all hooks and managed applications.
- Enter post as argument to be passed to the hook script. Hit Enter after entering the argument to record it.
- Enter the container image name scheme for the mongodb container in the Container image names to match field. We can use regular expressions to describe the matching images (like mongodb:4* in our case).
- Select the Upload file option to upload the hooks_mongodb.sh script and upload it.
- The name of the uploaded script is displayed.
- Change the suggested script name to posthook_mongodb1.sh Note: This script name must be unique across all managed apps and hooks.
- Click Add hook.
We’re taken back to the Execution hooks tab and can check again for container image matches and that both hooks are enabled:
Changing configured execution hooks
In the Execution hooks tab, each hook can be disabled or deleted (ensure to not accidentally leave a database in locked state after taking a snapshot by only disabling or deleting the post-snapshot hook).
If we want to make changes to one of the newly added hooks, we can select Edit in the actions column for the hook we want to change. In the Edit hook details screen, we could change the hook details, edit the script directly or upload a new version:
As we don’t want to modify the execution hook, we click Cancel.
Note: For validated apps, one can only disable (not recommended) the execution hooks provided by NetApp, but not modify or delete them. Custom execution hooks can be added to even validated apps, but the ones provided by NetApp will be executed first if both are enabled.
Create a snapshot with custom execution hooks
With the custom pre- and post-snapshot execution hooks for MongoDB enabled in ACS, we can now test their functionality. To create a snapshot, we follow the normal snapshot creation workflow in ACS. In the Data protection tab of the application view, we start the snapshot creation:
Accept the suggested snapshot name and click Snapshot in the summary view:
The snapshot process starts immediately:
Checking the logs of the mongodb container, we can confirm that the database writes were flushed, and the database locked (line #6):
And after a few seconds, once the snapshot is taken, the post-snapshot execution hook unlocks the database again (line #5):
In the mongodb1 namespace, a volumesnapshot does exist now (line #18):
The ACS activity view shows details about the execution phase of the hooks during snapshot creation, like application and cluster name, snapshot name, hooks source, arguments passed to the hook, as well as container and image name:
By copying the respective log details, we can also find out the execution time (3s) of the pre-snapshot hook by comparing the timestamps:
The same information can be found for the post-snapshot execution hook:
The run time of the post-snapshot hook is 4s:
Summary
Astra Control freezes application database transactions before taking a snapshot automatically for validated apps (MySQL, MariaDB, PostgreSQL, Jenkins).
In this article, we demonstrated how the newly added custom execution hooks feature enables you to take application consistent snapshots easily and quickly for custom and home-grown apps by creating custom pre- and post-snapshot hooks using MongoDB as an example.
Resources
Here are some helpful links: