This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.
In the world of data streaming, efficient data processing is key to unlocking valuable insights and powering real-time decision-making. With the recent integration of Event Hubs Schema Registry, Azure Stream Analytics can retrieve schema from the Schema Registry and deserialize data from Event Hubs input.
By moving schema metadata into the Schema Registry, Azure Stream Analytics eliminates the need for manual management and ensures consistent data quality through schema enforcement. It also enhances development efficiency, reduces message overhead, and allows for smooth transitions when introducing new schema versions.
Currently, the Schema Registry integration in Azure Stream Analytics is supported for Event Hubs inputs only. It allows one or multiple Schema Registries within one single job. However, it's important to note that a given Event Hub input can be configured with only one Schema Registry.
Once the Schema Registry is configured with the Event Hubs input in your Stream Analytics job, you will be able to preview your data on the query testing page. This allows you to ensure that your data is correctly deserialized using the desired schema. Additionally, the query testing page serves as a convenient means to verify the connection to the Schema Registry. Unlike input and output adapters, the Schema Registry configuration does not include explicit connection testing, making the query testing page an excellent tool for validating the integration and ensuring smooth data processing.
Before Adding Schema Registry to your Azure Stream Analytics job, there are two essential steps:
1. Managed Identity:
The authentication method of the Schema Registry configuration in Azure Stream Analytics only supports Managed Identity. The prerequisite to start integrating with the Schema Registry is to make sure that you either use system-assigned identity or user-assigned identity for your ASA job.
2. Add job access.
To access the Event Hubs Schema Registry and add it to your Stream Analytics job, you need to first get the security principal name of the ASA job, and then assign Schema Registry Reader role to it from the Event Hubs access control page.
Please see the article for more detailed instructions on how to set up the Schema Registry integration.
This feature will be available by May 27th, 2023, in all regions.
Feedback
For any feedback or feature requests, please reach out to askasa@microsoft.com.