Just Shipped: Azure Data Factory template for bulk copy from database to Azure Data Explorer

This post has been republished via RSS; it originally appeared at: Azure Data Explorer articles.

If you want to copy large amounts of data from a database (SQL Server, Oracle, DB2, Google Big Query, MySQL, Netezza, ODBC, PostgreSQL, etc.) into Azure Data Explorer, leverage Azure Data Factory template for bulk copy from database to Azure Data Explorer. ADF templates are predefined Azure Data Factory pipelines that allow you to get started quickly with Data Factory and reduce development time for building data integration projects. The Bulk copy from database to Azure Data Explorer template is created using Lookup and ForEach activities. You can use the template to create many pipelines per database or table for faster copying of data.

 

pipeline-from-template.png

Once you choose Bulk copy from database to Azure Data Explorer template you will be asked to fill few parameters:

  • ControlTableDataset - control table indicates what data is copied from source to destination and where it will be placed in the destination. you can fill as many records as you want.
  • SourceDataset – Linked service to source database.
  • AzureDataExplorerTable - Azure Data Explorer dataset.
  • once a pipeline is being created, define the Batch count in Command activity ForEachPartition. This parameter determines the number of pipelines that run in parallel (1..50) until the ControlTableDataset number of rows is reached.

Read more on Azure Data Factory template for bulk copy from database to Azure Data Explorer here 

 

Learn more about Azure Data Explorer (Kusto):

  1. Azure Data Explorer
  2. Documentation
  3. Course – Basics of KQL
  4. Query explorer
  5. Azure Portal
  6. User Voice
  7. Cost Estimator

Join us to share questions, thoughts, or ideas about Azure Data Explorer (Kusto) and receive answers from the diverse and knowledgeable Azure Data Explorer community.

 

Azure Data Explorer product team

“Join the conversation on the Azure Data Explorer community”.

REMEMBER: these articles are REPUBLISHED. Your best bet to get a reply is to follow the link at the top of the post to the ORIGINAL post! BUT you're more than welcome to start discussions here:

This site uses Akismet to reduce spam. Learn how your comment data is processed.