Pipefy is down
Incident Report for Pipefy
Postmortem

Root Cause

The system experienced an overload because the implemented scalability strategy of the new model was insufficient to handle the increased load on our new configuration, which resulted in the database being overwhelmed.

Resolution

To solve the problem, a percentage of users were reverted to the application's default configuration and we manually set the scalability strategy temporarily

Action Plan

We have made enhancements to our system to improve its scalability and stability, including but not liited to optimizing the database access mode and adjusting the application scalability. Additionally, we have implemented additional monitoring systems to track performance and ensure stability effectively.

Posted Jul 11, 2024 - 15:03 UTC

Resolved
We will continue to internally monitor our infra load to ensure that all users can use the platform, but everything is back to normal
Posted Jul 08, 2024 - 20:52 UTC
Update
We are monitoring our current infra load to ensure that all users can use the platform. Some webhooks might have faced delays and some users might be having slowness when clicking to open cards/pipes.
Posted Jul 08, 2024 - 19:14 UTC
Update
We are monitoring our current infra load to ensure that all users can use the platform without latency. Some webhooks might have faced delays.
Posted Jul 08, 2024 - 19:03 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Jul 08, 2024 - 18:16 UTC
Update
We are continuing to investigate this issue.
Posted Jul 08, 2024 - 18:00 UTC
Update
We are continuing to investigate this issue.
Posted Jul 08, 2024 - 17:47 UTC
Update
We are continuing to investigate this issue.
Posted Jul 08, 2024 - 17:38 UTC
Investigating
Some customers might face latency and 503 error pages
Posted Jul 08, 2024 - 17:25 UTC
This incident affected: Application.