Platform: US - Users are unable to login and experiencing error 500s

Incident Overview

Delinea received reports from customers in the US data boundary experiencing intermittent Platform access issues, including HTTP 500 errors and login failures. The incident began at 08:48 PM UTC and was resolved shortly thereafter. Service was fully restored by 09:21 PM UTC.

Root cause

The incident was triggered by resource saturation across the US regional clusters. During the incident, one of the service auto-scaled excessively resulting in overconsumption and exhausting the shared cluster resources, preventing other services from getting resources as needed. As a result, several platform services experienced performance degradation, leading to HTTP 500 errors and connection failures for customers. Our team identified the contributing factors and took action to restore service health. Systems stabilized shortly after mitigation, and all services are now operating normally.

Preventative Measures

• Our team is actively reviewing and optimizing service resource configurations to ensure efficient usage while maintaining expected performance levels.
• We are improving monitoring and alerting mechanisms to better detect unusual auto-scaling activity.