This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.
Final Update: Thursday, 06 June 2019 18:41 UTC
We've confirmed that all systems are back to normal with no customer impact as of June 6 2019, 18:32 UTC. Our logs show the incident started on June 6,2019 08:05 UTC and during impact timeline that it took to resolve the issue 100% of customers who subscribed to Webtest Alert Activation notification would have missed receiving their activation notification emails.
-chandar
We've confirmed that all systems are back to normal with no customer impact as of June 6 2019, 18:32 UTC. Our logs show the incident started on June 6,2019 08:05 UTC and during impact timeline that it took to resolve the issue 100% of customers who subscribed to Webtest Alert Activation notification would have missed receiving their activation notification emails.
- Root Cause: The failure was due to downstream notification service.
- Lessons Learned: We are looking to the telemetry for improvements on our ability to detect such issues sooner.
- Incident Timeline: 8 Hours & 27 minutes - June 6,2019 08:05 UTC through June 6 2019, 18:32 UTC
We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.
-chandar
Update: Thursday, 06 June 2019 17:42 UTC
Root cause has been isolated to a template in one of the downstream notification service which was impacting Webtest alert mail activated notifications. We are deploying a hotfix to address this issue . Gen2 alerts are working as expected. We estimate 2 hours before all impacts are addressed.
Root cause has been isolated to a template in one of the downstream notification service which was impacting Webtest alert mail activated notifications. We are deploying a hotfix to address this issue . Gen2 alerts are working as expected. We estimate 2 hours before all impacts are addressed.
- Work Around: None
- Next Update: Before 06/06 20:00 UTC