Platform: US - Slow pages with intermittent HTTP failures
Incident Report for Delinea
Postmortem

Impact

A subset of Delinea Platform users within the US data boundary experienced slowness navigating between web pages of their tenant

Start of Impact (UTC): Oct 15, 2024, 12:12 PM

End of Impact (UTC): Oct 14, 2024, 01:45 PM

Incident Overview

We identified an issue where platform users experienced HTTP errors in the web browser console and slow page load times while accessing their tenant.

Root cause

The issue was caused by resource contention on a secondary backend database service. High resource utilization led to intermittent timeouts on some API calls. To mitigate the impact, we promptly rolled back the latest release, reverting all API calls to the primary database.

Preventative Actions

To avoid similar incidents in the future, we are implementing the following improvements:

  • Reviewing and optimizing timeout settings for cache operations
  • Fine-tuning auto-scaling thresholds to better handle resource demand
  • Enhancing monitoring for potential resource contention

We sincerely apologize for any inconvenience this incident may have caused and appreciate your understanding as we continue to enhance the reliability of our platform.

Posted Dec 18, 2024 - 12:59 EST

Resolved
This incident has been resolved.
Posted Oct 15, 2024 - 09:41 EDT
Investigating
We are currently investigating an issue where the Platform pages are taking longer than expected with some HTTP failure messages.
Posted Oct 15, 2024 - 08:28 EDT
This incident affected: US (Platform).