Write-up published
Resolved
Summary:Slowness and timeout caused by an overload on our reader database
Impact:Major, Pipefy was inaccessible for less than a minute
Detection:Engineering team noticed the overload on the database through the monitoring tools
Root Cause:An index in production started to be inefficient for a specific query, which degraded the performance of the core database
Resolution:A new index was created increasing the efficiency for that query.
Resolved
The issue has been solved
Monitoring
The fix has been implemented and the Pipefy is now stable. The performance of one of our filters in the database was not good and we improve it and reduce the time spent to execute the query.
We will keep monitoring
Identified
We identified where the issue was happening and we are deploying a fix.
Investigating
We are currently investigating what is causing the slowness