This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .
Final Update: Wednesday, 01 June 2022 05:45 UTC
We've confirmed that all systems are back to normal with no customer impact as of 06/01, 05:30 UTC. Our logs show the incident started on 05/19, 23:00 UTC and that during the 12 days 6 hours & 30 minutes that it took to resolve the issue. Some of the customers might have experienced viewing two alerts for notification. When a resource is unhealthy, two notifications are being raised and similarly for alert resolution as well.
-Srikanth
We've confirmed that all systems are back to normal with no customer impact as of 06/01, 05:30 UTC. Our logs show the incident started on 05/19, 23:00 UTC and that during the 12 days 6 hours & 30 minutes that it took to resolve the issue. Some of the customers might have experienced viewing two alerts for notification. When a resource is unhealthy, two notifications are being raised and similarly for alert resolution as well.
- Root Cause: The failure was due to one of the backend dependency.
- Incident Timeline: 12 days 6 Hours & 30 minutes - 05/19, 23:00 UTC through 06/01, 05:30 UTC.
-Srikanth
Update: Tuesday, 31 May 2022 23:28 UTC
Root cause has been isolated to a bad deployment. To address this issue we are in the process of rolling back. As a customer of Metric Alerts you could be experiencing viewing two alerts for unhealthy resources. When a resource is unhealthy, two notifications are being raised and similarly for alert resolution, two notifications are being sent
Root cause has been isolated to a bad deployment. To address this issue we are in the process of rolling back. As a customer of Metric Alerts you could be experiencing viewing two alerts for unhealthy resources. When a resource is unhealthy, two notifications are being raised and similarly for alert resolution, two notifications are being sent
- Work Around:
- Start time: 5/19/2022
- Next Update: Before 06/01 05:30 UTC