Resolved
Full services have been restored. Please contact support if you continue to experience issues connecting to your site.
Monitoring
Microsoft has identified and resolved the problem and we are seeing customer traffic returning. We have put the traffic migration on hold while we continue to monitor the recovery and will update this space once we see traffic return to normal.
Here is the latest Microsoft update regarding the situation:
"We identified a recent change that was strongly correlated with the onset of impact. We have completed the rollback of this change and telemetry across services are continuing to show signs of recovery. Customer should be observing recovery at this time."
Investigating
Microsoft has provided another update: “We have successfully redirected portions of traffic away from potentially affected network paths and tested mitigation actions. At this time, those actions have not produced a broad improvement in impacted service telemetry. Engineering teams continue investigating routing and network-path anomalies that align with the start of the incident and are evaluating additional mitigation options. While we do not have an estimated time for mitigation at this stage, we will provide updates as they become available.”
ikaun remains focused on migrating customer traffic to an alternate data center. We will update this space once all US traffic has been migrated.
Investigating
Microsoft has provided the following update:
"We are exploring mitigation options on affected network devices. Early indications suggest the issue is related to network traffic flow through the impacted infrastructure; however our analysis remains ongoing. We are closely monitoring traffic while continuing our investigation."
ikaun is continuing to prepare to divert traffic to a new US datacenter which we plan to begin by 1:00 Eastern.
Investigating
This has been traced to an outage in a Microsoft Azure datacenter. Microsoft is investigating and we will provide updates when available. We are making preparations to divert customer traffic to an unaffected datacenter.
Investigating
We are investigating an issue where US customers are unable to log into new sessions.