Platform: Intermittent login failures due to email delivery issues
Postmortem
Incident Overview
On October 20, 2025, some users experienced intermittent login failures when accessing the tenant. Impacted users saw an HTTP 504 error caused by delays in receiving MFA codes via email. Users utilizing email-based MFA were affected, while those using other authentication methods such as authenticator app or SMS were able to log in successfully.
The impact occurred during the following time windows:
- First Window: Oct 20, 2025, from 07:00–09:10 UTC
- Second Window: Oct 20, 2025, from 15:34–18:20 UTC
Root Cause
The issue was caused by a service disruption at our cloud provider in the US East region, which affected downstream email delivery services. This disruption led to email queuing delays and retries, resulting in intermittent failures when sending MFA codes via email. As a result, users depending on email-based MFA were unable to complete login attempts intermittently during the impacted periods.
Normal service was restored once the cloud provider mitigated the regional disruption and email delivery performance stabilized.
Preventive Actions
To strengthen reliability and minimize future impact, we are implementing the following measures:
- Transitioning to global email delivery endpoints across multiple regions to improve redundancy and resilience.
- Evaluating a secondary email delivery provider to provide automatic failover in the event of a regional or provider-wide outage.
- Enhancing monitoring and alerting for email delivery performance to enable faster detection and response to similar issues.
Resolved
Monitoring
Our engineering teams continue to monitor the service to ensure full recovery. We will provide additional updates as more information becomes available.
Identified
Users with email-based MFA are affected. As a workaround, please use other available MFA methods (e.g., authenticator app or SMS).
Our engineering teams are actively working to restore the email-based MFA service. We appreciate your patience and will provide an update as soon as more information becomes available.
Investigating
Our engineering teams are fully engaged and working to restore normal service as quickly as possible. We apologize for any inconvenience this may cause and appreciate your patience during this time.