Ingest once, use in multiple clusters (for different scenarios, teams and cost allocation)

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.

Follower database can be used to utilize ingested data in multiple clusters. You can now build a data pipeline to collect organizational data that can be used in multiple clusters for different scenarios.

For example, an automotive IoT company collects data from devices in its cars. They stream a high data volume to their Azure Data Explorer cluster to monitor health and performance and mitigate issues. This is their live, mission critical environment. In parallel, a data science team explores the correlation between device performance and ambient temperature, humidity, etc. They also need access to the data for their research. The company does not want to risk their production setup by letting the data science team perform long running, unmonitored queries on the same environment. Follower database allows the data science team to use a separate cluster to access the relevant data without risking the production environment and without ingesting the data into their cluster once more.

Follower database allows you to attach a read-only single database, multiple databases, or all databases in a specific source cluster to different target clusters. The source cluster’s database won’t be impacted by queries performed on the attached database in the target cluster. The two databases (source and target) can have different caching policies as per their needs to optimize the costs of each team and scenario. Being able to associate cost to different teams per their use case and avoid loading all cost on the source cluster is a significant benefit of follower database.

Follower database serves as the foundation to share data between teams and organizations using Azure Data Share for data hosted in Azure Data Explorer.

Today you can set up follower databases using C#, Python, JAVA and ARM template.

For more information, read the follower database documentation.

 

Azure Data Explorer team

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.