Platform/SSC: Customers in Canada region are unable to access their tenants

Incident Report for Delinea

Postmortem

Impact 

A subset of users located in Canada could not access Delinea Platform and Secret Server Cloud tenants.  

  • Start of Impact: March 03, 2025, at 16:32 UTC 
  • End of Impact: March 03, 2025, at 17:15 UTC 

Incident Overview 

A subset of users in Canada experienced an inability to access the Delinea Platform and Secret Server Cloud tenants due to a connectivity issue. The impact was isolated to traffic routed through the Toronto Point of Presence (PoP), where 54% of traffic failed due to an issue with an external Internet Service Provider. However, traffic routed through alternative regional PoPs remained unaffected, allowing many users to continue accessing Delinea services without disruption. Since the issue was limited to Toronto PoP, other locations and services remained fully operational.  

Root cause 

The root cause of this incident was a service disruption within the Internet Service Provider (ISP) network connected to the Toronto PoP on Imperva’s side. The ISP experienced two network disruptions lasting 3 minutes and 40 minutes consecutively. At 17:13 UTC, Imperva Network Operations temporarily bypassed the impacted ISP to mitigate customer impact. At 20:50 UTC, Imperva Network Operations removed the bypass and confirmed the full stability of the service. 

Preventative Actions 

Since the issue was caused by an external Internet Service Provider outside of our direct control, the following actions are recommended to mitigate similar incidents in the future: 

Enhance Traffic Routing Resilience: Improve routing policies to dynamically redirect traffic to alternative PoPs in the event of regional disruptions. 

Collaboration with Telecom Providers: Engage with key telecommunication partners to understand their redundancy plans and advocate for improved reliability.

Posted Mar 13, 2025 - 13:51 EDT

Resolved

The issue affecting Platform and Secret Server Cloud tenants in the Canada region has been resolved. We will provide a root cause analysis on our status page within seven days.
Posted Mar 03, 2025 - 14:17 EST

Monitoring

The issue impacting Platform and Secret Server Cloud tenants in the Canada region has been mitigated. We are continuing to monitor the situation and will share more details as they become available.
Posted Mar 03, 2025 - 12:29 EST

Investigating

We are currently experiencing an outage affecting Platform and Secret Server Cloud tenants hosted in the Canada region. Users may encounter error messages such as proxy failures or TCP connection timeouts.

Our team is actively investigating and working to resolve the issue. For any questions or concerns, please contact our support team at https://support.delinea.com.

We appreciate your patience and will provide updates as we learn more.
Posted Mar 03, 2025 - 12:23 EST
This incident affected: CA (Platform, Secret Server Cloud).