* refactor(ci): use GitHub secrets and variables
We've been using values that are variable across multiple workflows,
and those can only be changed if modifying the workflows, but we should
be able to change the values without committing new changes in the code
for this purpose we're now using GitHub Variables, and even moving
non-sensitive information into variables instead of secrets. Allowing
more flexibility and other scenarios that should be easier to manage,
like deploying to Mainnet or Testnet.
* refactor(ci): use new GitHub variables for GCP auth
* fix(ci): typo
* fix(ci): do not use multiple variables for the same value
* fix(ci): typo in variable
* fix(vars): use different variables for machine types
* fix(vars): missing substitution
* fix: typo
* fix: make the input CI network override the default network
* Use the correct network variable for creating disks
---------
Co-authored-by: teor <teor@riseup.net>
* feat(ci): delete unused artifacts in registries
Previous behavior:
Docker artifacts are costing us a good part of our infrastructure budget,
and we needed a way to remove unused artifacts.
Expected behavior:
Delete unused (not just old) docker artifacts in GAR (Google Artifact Registry),
preferably using a generic solution is this needs to be expanded into other
Docker registries.
Solution:
Implement GCR Cleaner https://github.com/GoogleCloudPlatform/gcr-cleaner,
as this tools provided integration with `docker/login-action` to interact
with multiple Docker v2 registries.
* fix(action): use hours instead of days
* chore: add TODO
* Update .github/workflows/delete-gcp-resources.yml
Co-authored-by: teor <teor@riseup.net>
* fix: allow the action to fail if some images can't be deleted
Co-authored-by: teor <teor@riseup.net>
Previous behavior:
`gcloud` commands have been running without an appropiate authentication
as the `auth` auction was sucessfully executed, but the actual gcloud
CLI being used in further jobs was not using the correct configuration
nor credentials
Expected behavior:
All `gcloud` commands should be properly configured and authenticated.
Solution:
Add the `google-github-actions/setup-gcloud` action after each
`google-github-actions/auth` invocation, and before running any `gcloud`
command.
Remove the need of an OAuth Access token when not required by following
steps
* Delete test instances after 3 days
* Use correct delete command, improve shell quoting
* Use sed to provide the correct zone or region
* Fix quoting
* Fix IFS
* Fix IFS for multiple disks
* Document why we can't quote some shell variables
* Document that instances can get deleted
* Fix exact names in deletion docs
* Fix delete GCP resources commands
* Don't create a GCP credentials file
* Keep the latest 2 images
* Explain time
* Show the names of disks that are being deleted
* Actually run the image delete steps
* Only delete commit-based instance templates
* Document automated deletion
Previous behavior:
Disk images are being accumulated in GCP for a few years, but this
generates unneeded costs as we're not using images older than 1-2 weeks.
Expected behavior:
Delete previously unused images based on a timefrime.
Solution:
Delete images created on a pull request older than 30 days, from the
`main` branch if older than 60 days, and any other image older than 90
days.
A TODO is on place as we'd like to keep at least the 2 latest images of
each type (zebra checkpoint, zebra tip, lwd tip). Once we've excluded
those images, we can delete any older images after 1 week.
Previous behavior:
Sometimes Google Cloud authentication fails, this might happen before
IAM permissions are fully propagated
Expected behavior:
If the authentication fails, retry at least 3 times before exiting with
a non zero exit code
Applied solution:
Google GitHub Actions for auth recently added this a `retries` feature
which is now implemented to workaround this issue.
Note: 95a6bc2a27
Fixes https://github.com/ZcashFoundation/zebra/issues/4846
* feat(actions): delete old GCP resources
* fix(ci): delete old instances templates
* fix(actions): use correct date arguments and conversion
* fix(actions): missing command in gcloud
* fix(gcp): if an instance can't be deleted, continue
* refacor(action): cleanup and execute monthly
Gustavo Valverde
dependabot[bot]
teor