We’re in good shape to have CI for securedrop-club and it could be as simple as:
molecule test -s bind molecule test -s icinga molecule test -s backup molecule test -s postfix molecule test -s weblate ...
Over the past few weeks we managed to make it stable. However there is something we have no control over: the stability of the OpenStack API. Just like AWS, it will fail tests and create frustration. Today is one of these days: the API fails more than 25% of the time and nothing really works. I propose we deal with this by:
- cleary identifying (for human and machine benefit) when the cloud API fails
- retrying N times if the cloud API fails
- pause the CI, retry the last failed job every H hours, resume the CI unless it fails because of the cloud API
What do you think ?