This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.
Final Update: Thursday, 16 April 2020 20:57 UTC
We've confirmed that all systems are back to normal with no customer impact as of 4/16, 19:30 UTC. Our logs show the incident started on 3/30 and that during ~2.5 weeks that it took to resolve the issue customers experienced failures while creating/updating/deleting alert rules.
-Anupama
We've confirmed that all systems are back to normal with no customer impact as of 4/16, 19:30 UTC. Our logs show the incident started on 3/30 and that during ~2.5 weeks that it took to resolve the issue customers experienced failures while creating/updating/deleting alert rules.
- Root Cause: The failure was due to one of the backend services using incorrect version.
- Incident Timeline: ~2.5 weeks - 3/30 through 4/16, 19:30 UTC
-Anupama
Update: Thursday, 16 April 2020 20:02 UTC
Engineers continue to work on deploying the service to correct version. The deployment process is in progress. Some customers may experience failures while creating/updating/deleting alert rules. We estimate 2 hours before all failures are addressed.
Engineers continue to work on deploying the service to correct version. The deployment process is in progress. Some customers may experience failures while creating/updating/deleting alert rules. We estimate 2 hours before all failures are addressed.
- Work Around: None
- Next Update: Before 04/16 23:30 UTC
Update: Thursday, 16 April 2020 16:35 UTC
Root cause has been isolated to one of the backend services using old version, which was impacting creation/update/deletion of alert rules in Application insights. To address this issue engineers are working on deploying the service with correct version. Some customers may experience failures while creating/updating/deleting alert rules. We estimate 3 hours before all failures are addressed.
Root cause has been isolated to one of the backend services using old version, which was impacting creation/update/deletion of alert rules in Application insights. To address this issue engineers are working on deploying the service with correct version. Some customers may experience failures while creating/updating/deleting alert rules. We estimate 3 hours before all failures are addressed.
- Work Around: None
- Next Update: Before 04/16 20:00 UTC
Initial Update: Thursday, 16 April 2020 12:52 UTC
We are aware of issues within Application Insights in Moon cake region and are actively investigating. Customers will not be able to create/update/delete alert rules on Application Insights metrics, Application Insights availability tests and custom metrics.
-Santhosh
We are aware of issues within Application Insights in Moon cake region and are actively investigating. Customers will not be able to create/update/delete alert rules on Application Insights metrics, Application Insights availability tests and custom metrics.
- Work Around: None
- Next Update: Before 04/16 17:00 UTC
-Santhosh