Platform: US - Disruption in data fetching for Privilege Control for Cloud Entitlements (PCCE)

Incident Overview

On June 5, 2025, at 5:25 PM EDT, we identified an issue where the Privilege Control for Cloud Entitlements (PCCE) integration service stopped fetching new data. Despite this disruption in data ingestion, the PCCE user interface (UI) remained fully functional, allowing users uninterrupted access to previously fetched data.

Root Cause

The issue was traced to failures within our ArangoDB infrastructure, which supports PCCE data ingestion. The primary ArangoDB cluster encountered stability issues, and recovery efforts on the existing cluster were unsuccessful. We executed our contingency plan, treating this as a disaster recovery scenario, and migrated customer databases to a healthy ArangoDB cluster. Once the migration was completed, the data pipeline resumed normal operation.

Preventive Actions

• We are actively working to significantly reduce the write load on the database, which we believe is a contributing factor to the issue.
• This effort will also enable us to leverage a secondary hot backup instance for faster and more seamless recovery in future incidents.