Looking back on the April 19 data incident

On the evening of April 19, Toonkit had a data incident. Backups were being created automatically, but the structure meant they couldn't be restored independently when it mattered most. Here's what happened that day and what we changed in our backup and operational procedures afterward.