Intermittent API access for some customers

Incident Report for Braze, Inc.

Resolved

Today at 5:29pm ET, one of our database servers that houses a very small fraction of our customers rapidly leaked memory and crashed, which caused some API calls to return 500 error codes. We quickly got the database operational and confirmed all API systems were fully back online for all customers by 5:36pm ET. Our DBAs have been investigating the root cause for past few hours and are escalating to the database vendor for support. We will update this page when we have a diagnosis.

UPDATE: Our DBAs believe they have identified the issue due to a corrupted metadata file. The file has since been rebuilt successfully.

Posted Nov 19, 2014 - 17:29 EST