Authors: SRE, Data Eng, Support team

Date: 2020-04-23

Status: Resolved

Summary: Degradation in web application

Impact: Minor - The performance degradation on the application happened from Apr 23 20:34 \(UTC\) to Apr 23 20:55 \(UTC\).

Root Cause:

The identified root cause of this degradation was an unexpected behavior in the structures that control the connections to our databases.

Detection and resolution:

The issue was detected by our internal monitoring system that triggered an alert and informed the team.

After the root cause was identified, the structure was rebooted and the requests were sent and received normally.Action plan: Preventive action items

Pipefy’s SRE team is already working on an improved structure that will provide a permanent solution and mitigate these issues. We will, however, contact the service that provides us with the structure that failed in order to identify what caused the situation.

The instability has been fixed and the performance of the platform has been restored.As soon as the preventive investigation process is over, we’ll share further details about the causes, implemented fixes and preventive actions to be implemented.

The system instability has been fixed and the performance of the platform has been restored.We are currently monitoring the system to ensure all features are working as expected. As soon as the monitoring process is over we’ll share further details about the causes, investigation and preventive actions to be implemented.