This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.
Final Update: Wednesday, 08 January 2020 05:13 UTC
We've confirmed that all systems are back to normal with no customer impact as of 01/08, 03:50 UTC. Our logs show the incident started on 01/07, 20:04 UTC and that during the ~7 hours and 46 minutes that it took to resolve the issue some customers in South Central US may have received failure notifications when processing queries and difficulties accessing data in the Azure Portal. Customers may also have seen alerts not firing or delays in Log Search Alerts.
-Mohini
We've confirmed that all systems are back to normal with no customer impact as of 01/08, 03:50 UTC. Our logs show the incident started on 01/07, 20:04 UTC and that during the ~7 hours and 46 minutes that it took to resolve the issue some customers in South Central US may have received failure notifications when processing queries and difficulties accessing data in the Azure Portal. Customers may also have seen alerts not firing or delays in Log Search Alerts.
- Root Cause: The failure was due to non-impacting maintenance window for one of our backend service exceeded the expected threshold.
- Incident Timeline: 7 Hours & 46 minutes - 01/07, 20:04 UTC through 01/08, 03:50 UTC
-Mohini
Update: Tuesday, 07 January 2020 22:35 UTC
Root cause has been isolated to maintenance activity in the backend clusters which was impacting data access, query failures in Application Insights and Log Analytics for customers in South Central US. Customers will also be experiencing alerting delays or failures in Log Search Alerts. We shall provide the final update once we get confirmation from our backend team that the maintenance work is completed. We apologize for sending out resolved communications earlier, the scheduled maintenance had missed nodes.
Root cause has been isolated to maintenance activity in the backend clusters which was impacting data access, query failures in Application Insights and Log Analytics for customers in South Central US. Customers will also be experiencing alerting delays or failures in Log Search Alerts. We shall provide the final update once we get confirmation from our backend team that the maintenance work is completed. We apologize for sending out resolved communications earlier, the scheduled maintenance had missed nodes.
- Work Around:None
- Next Update: Before 01/08 03:00 UTC
Update: Tuesday, 07 January 2020 21:19 UTC
We've confirmed that all systems are back to normal with no customer impact as of 01/07, 21:54 UTC. Our logs show the incident started on 01/07, 20:04 UTC and that during the 1 hour and 50 minutes that it took to resolve the issue some customers in South Central US experienced issues with data access, query failures in Application Insights and alerting issues in Log Search Alerts.
-Jayadev
We've confirmed that all systems are back to normal with no customer impact as of 01/07, 21:54 UTC. Our logs show the incident started on 01/07, 20:04 UTC and that during the 1 hour and 50 minutes that it took to resolve the issue some customers in South Central US experienced issues with data access, query failures in Application Insights and alerting issues in Log Search Alerts.
- Root Cause: The failure was due to one of the backend service.
- Incident Timeline: 1 hour and 50 minutes - 01/07, 20:04 UTC through 01/07, 21:54 UTC
-Jayadev
Initial Update: Tuesday, 07 January 2020 20:52 UTC
We are aware of issues within Application Insights and are actively investigating. Some customers in South Central US may experience issues with Data Access, Query Failures and alerting failures in Log Search Alerts.
-Jayadev
We are aware of issues within Application Insights and are actively investigating. Some customers in South Central US may experience issues with Data Access, Query Failures and alerting failures in Log Search Alerts.
- Work Around:
- Next Update: Before 01/08 01:00 UTC
-Jayadev