March 22nd – Intermittent problem with personal voice messages

5:41 AM UTC. We are investigating reports that personalized messages uploaded to voice apps are sometimes not played during calls. The problem seems to happen intermittently for some calls only.

The engineering team is working on finding out the root cause of the problem and fixing it.

24th March 9:30 AM UTC. The problem was traced to one of the media servers that handles calls to voice response apps.
Since calls are automatically distributed between different servers – only some calls, that landed on this server, played the IVR incorrectly. Calls that were handled by the other servers did not have any problems.

This server is no longer handling calls which should fix the problem for now. We are still investigating the root cause of the problem.

Update: The root cause of the problem has been identified. The problem was with a 3rd party module that is used for syncing the personalized voice messages across different servers. The problem has been fixed and all the servers are back online.