The disaster recovery exercise will be Monday May 14th, 2018 @ 11am Paris/Berlin time at https://gitter.im/freedomofpress/securedrop @fpoulain and myself will be there and anyone is welcome to join.
- Kill the packages host (because well, it’s not very precious or used at the moment by renaming the VM and suspend it
- Manually create a VM by the same name using the backup from the previous day
- Run ansible to sync the backup (necessary because the IP changes)
- Wait a few hours to verify domain name propagation and icinga are fine with the backup restauration
- Update the documentation
We can also destroy a machine such as the website and verify it comes back to life after a run of ansible. It is comparatively less scary because it does not contain information and can be re-generated from the ansible repository.
We should also think about what happens if the ansible host itself breaks, although I think it does not matter and we have a backup anyways.
Anything else you would like covered?