April 12th – Service outage
10:32 AM UTC. We are investigating reports of intermittent issues with our apps.
We apologize for the inconvenience caused.
11:51 AM UTC. Initial investigation points to high CPU utilization in one of the database machines as the root cause. We are restarting the servers to resolve the issue.
12:02 PM UTC. The fix that was attempted has not resolved the issue. The root cause of the issue is still being investigated. The issue affects both our web app and mobile apps.
2:41 PM UTC. The cause of the outage appears to be MongoDB Atlas – a managed database service that we use. We are working with their technical support teams to find the root cause of the issue and resolve this.
9.19 PM UTC. Most of the services have been restored and the apps are back online. However, the following functions are still unavailable:
- Website chat widget
- User avatars & file attachments in conversations
- Offline and Push notification
April 13th 6:50 AM UTC. All services except the website chat widget are now back online. We are still working on resolving the issue.
1.47 PM UTC. We are applying a temporary fix for the issue. The situation is still being closely monitored.
2.13 PM UTC. The fix that was applied has resolved the issue for now. We are still closely monitoring the situation and are investigating a more permanent fix for this issue.