octo
/
Zebra
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

ref(workflows): consolidate workflows based on their purpose (#7616) 

* ref(workflows): consolidate workflows based on their purpose

This also renames the workflows to make their naming more consistent and adding a naming convention

Fixes: #6166
Fixes: #6167

* fix(workflows): use correct name for patch

* fix(workflow): docker unit tests

* fix(release): validation error

Error:
```
Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"}
```

Fixes: https://github.com/release-drafter/release-drafter/issues/1125

* fix(workflows): reference correct name

* fix: remove extra workflow

* fix(workflows): use larger runners

* fix(workflow): remove code already in docker unit-test

* fix(unit-tests): start zebra the right way

* fix: typo in patch name

* chore: move job to logical order

* imp(workflows): use better name for gcp tests

* add: missing merge changes

* chore: use better name for find-disks

* fix(ci): use the `entrypoint.sh` to change the Network

* fix(ci): add missing `ZEBRA_CONF_PATH` variable

* fix(ci): allow to build the entrypoint file with testnet

* fix(entrypoint): allow to create a dir and file with a single variable

* refactor: test config file in CI and CD with a reusable workflow

* fix(ci): wrong name used

* fix(ci): use checkout

* fix(ci): improve docker config tests

* fix(ci): use better name for protection rules

* Fix changed workflow file name in docs patch file

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Arya <aryasolhi@gmail.com>

* fix(cd): depend on file tests

* fix(docs): adapt to new workflow name

* fix: revert test coverage on CD

* chore: reduce diff

* fix(ci): allow using variable images for reusable workflows

* fix(dockerfile): use variables or default for config path and file

* fix(entrypoint): if `$ZEBRA_CONF_PATH` is set, do not override it

* Fix patch job names and remove failure job testnet dependencies

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Arya <aryasolhi@gmail.com>
This commit is contained in:
Gustavo Valverde 2023-10-18 07:16:02 +01:00 committed by GitHub
parent 08ce2ad500
commit fc0133e886
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
34 changed files with 601 additions and 483 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

39
.github/workflows/cd-deploy-nodes-gcp.patch.yml vendored Normal file
View File

@ -0,0 +1,39 @@
name: Deploy Nodes to GCP
on:
pull_request:
paths-ignore:
# code and tests
- '**/*.rs'
# hard-coded checkpoints and proptest regressions
- '**/*.txt'
# dependencies
- '**/Cargo.toml'
- '**/Cargo.lock'
# configuration files
- '.cargo/config.toml'
- '**/clippy.toml'
# workflow definitions
- 'docker/**'
- '.dockerignore'
- '.github/workflows/cd-deploy-nodes-gcp.yml'
- '.github/workflows/sub-build-docker-image.yml'
jobs:
build:
name: Build CD Docker / Build images
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-configuration-file:
name: Test CD default Docker config file / Test default-conf in Docker
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-zebra-conf-path:
name: Test CD custom Docker config file / Test custom-conf in Docker
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

136
.github/workflows/continous-delivery.yml → .github/workflows/cd-deploy-nodes-gcp.yml vendored
View File

@ -1,4 +1,4 @@
name: CD name: Deploy Nodes to GCP
# Ensures that only one workflow task will run at a time. Previous deployments, if # Ensures that only one workflow task will run at a time. Previous deployments, if
# already in process, won't get cancelled. Instead, we let the first to complete # already in process, won't get cancelled. Instead, we let the first to complete
@ -47,8 +47,8 @@ on:
# # workflow definitions # # workflow definitions
# - 'docker/**' # - 'docker/**'
# - '.dockerignore' # - '.dockerignore'
# - '.github/workflows/continous-delivery.yml' # - '.github/workflows/cd-deploy-nodes-gcp.yml'
# - '.github/workflows/build-docker-image.yml' # - '.github/workflows/sub-build-docker-image.yml'
# Only runs the Docker image tests, doesn't deploy any instances # Only runs the Docker image tests, doesn't deploy any instances
pull_request: pull_request:
@ -66,14 +66,13 @@ on:
# workflow definitions # workflow definitions
- 'docker/**' - 'docker/**'
- '.dockerignore' - '.dockerignore'
- '.github/workflows/continous-delivery.yml' - '.github/workflows/cd-deploy-nodes-gcp.yml'
- '.github/workflows/find-cached-disks.yml' - '.github/workflows/sub-build-docker-image.yml'
release: release:
types: types:
- published - published
jobs: jobs:
# If a release was made we want to extract the first part of the semver from the # If a release was made we want to extract the first part of the semver from the
# tag_name # tag_name
@ -108,7 +107,7 @@ jobs:
# The image will be commonly named `zebrad:<short-hash | github-ref | semver>` # The image will be commonly named `zebrad:<short-hash | github-ref | semver>`
build: build:
name: Build CD Docker name: Build CD Docker
uses: ./.github/workflows/build-docker-image.yml uses: ./.github/workflows/sub-build-docker-image.yml
with: with:
dockerfile_path: ./docker/Dockerfile dockerfile_path: ./docker/Dockerfile
dockerfile_target: runtime dockerfile_target: runtime
@ -118,99 +117,40 @@ jobs:
# Test that Zebra works using the default config with the latest Zebra version. # Test that Zebra works using the default config with the latest Zebra version.
test-configuration-file: test-configuration-file:
name: Test Zebra CD Docker config file name: Test CD default Docker config file
timeout-minutes: 15
runs-on: ubuntu-latest
needs: build needs: build
steps: uses: ./.github/workflows/sub-test-zebra-config.yml
- uses: r7kamura/rust-problem-matchers@v1.4.0 with:
test_id: 'default-conf'
- name: Inject slug/short variables docker_image: ${{ vars.GAR_BASE }}/zebrad@${{ needs.build.outputs.image_digest }}
uses: rlespinasse/github-slug-action@v4 grep_patterns: '-e "net.*=.*Main.*estimated progress to chain tip.*BeforeOverwinter"'
with: test_variables: '-e NETWORK'
short-length: 7 network: 'Mainnet'
# Make sure Zebra can sync at least one full checkpoint on mainnet
- name: Run tests using the default config
shell: /usr/bin/bash -exo pipefail {0}
run: |
docker pull ${{ vars.GAR_BASE }}/zebrad@${{ needs.build.outputs.image_digest }}
docker run --detach --name default-conf-tests -t ${{ vars.GAR_BASE }}/zebrad@${{ needs.build.outputs.image_digest }}
# Use a subshell to handle the broken pipe error gracefully
(
trap "" PIPE;
docker logs \
--tail all \
--follow \
default-conf-tests | \
tee --output-error=exit /dev/stderr | \
grep --max-count=1 --extended-regexp --color=always \
-e "net.*=.*Main.*estimated progress to chain tip.*BeforeOverwinter"
) || true
LOGS_EXIT_STATUS=$?
docker stop default-conf-tests
EXIT_STATUS=$(docker wait default-conf-tests || echo "Error retrieving exit status");
echo "docker exit status: $EXIT_STATUS";
# If grep found the pattern, exit with the Docker container exit status
if [ $LOGS_EXIT_STATUS -eq 0 ]; then
exit $EXIT_STATUS;
fi
# Handle other potential errors here
echo "An error occurred while processing the logs.";
exit 1;
# Test reconfiguring the docker image for testnet. # Test reconfiguring the docker image for testnet.
test-configuration-file-testnet: test-configuration-file-testnet:
name: Test testnet Zebra CD Docker config file name: Test CD testnet Docker config file
timeout-minutes: 15
runs-on: ubuntu-latest
needs: build needs: build
steps: # Make sure Zebra can sync the genesis block on testnet
- uses: r7kamura/rust-problem-matchers@v1.4.0 uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'testnet-conf'
docker_image: ${{ vars.GAR_BASE }}/zebrad@${{ needs.build.outputs.image_digest }}
grep_patterns: '-e "net.*=.*Test.*estimated progress to chain tip.*Genesis" -e "net.*=.*Test.*estimated progress to chain tip.*BeforeOverwinter"'
test_variables: '-e NETWORK'
network: 'Testnet'
- name: Inject slug/short variables # Test that Zebra works using $ZEBRA_CONF_PATH config
uses: rlespinasse/github-slug-action@v4 test-zebra-conf-path:
with: name: Test CD custom Docker config file
short-length: 7 needs: build
uses: ./.github/workflows/sub-test-zebra-config.yml
# Make sure Zebra can sync the genesis block on testnet with:
- name: Run tests using a testnet config test_id: 'custom-conf'
shell: /usr/bin/bash -exo pipefail {0} docker_image: ${{ vars.GAR_BASE }}/zebrad@${{ needs.build.outputs.image_digest }}
run: | grep_patterns: '-e "v1.0.0-rc.2.toml"'
docker pull ${{ vars.GAR_BASE }}/zebrad@${{ needs.build.outputs.image_digest }} test_variables: '-e NETWORK -e ZEBRA_CONF_PATH="zebrad/tests/common/configs/v1.0.0-rc.2.toml"'
docker run --env "NETWORK=Testnet" --detach --name testnet-conf-tests -t ${{ vars.GAR_BASE }}/zebrad@${{ needs.build.outputs.image_digest }} network: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Use a subshell to handle the broken pipe error gracefully
(
trap "" PIPE;
docker logs \
--tail all \
--follow \
testnet-conf-tests | \
tee --output-error=exit /dev/stderr | \
grep --max-count=1 --extended-regexp --color=always \
-e "net.*=.*Test.*estimated progress to chain tip.*Genesis" \
-e "net.*=.*Test.*estimated progress to chain tip.*BeforeOverwinter";
) || true
LOGS_EXIT_STATUS=$?
docker stop testnet-conf-tests
EXIT_STATUS=$(docker wait testnet-conf-tests || echo "Error retrieving exit status");
echo "docker exit status: $EXIT_STATUS";
# If grep found the pattern, exit with the Docker container exit status
if [ $LOGS_EXIT_STATUS -eq 0 ]; then
exit $EXIT_STATUS;
fi
# Handle other potential errors here
echo "An error occurred while processing the logs.";
exit 1;
# Deploy Managed Instance Groups (MiGs) for Mainnet and Testnet, # Deploy Managed Instance Groups (MiGs) for Mainnet and Testnet,
# with one node in the configured GCP region. # with one node in the configured GCP region.
@ -225,14 +165,14 @@ jobs:
# otherwise a new major version is deployed in a new MiG. # otherwise a new major version is deployed in a new MiG.
# #
# Runs: # Runs:
# - on every push/merge to the `main` branch # - on every push to the `main` branch
# - on every release, when it's published # - on every release, when it's published
deploy-nodes: deploy-nodes:
strategy: strategy:
matrix: matrix:
network: [Mainnet, Testnet] network: [Mainnet, Testnet]
name: Deploy ${{ matrix.network }} nodes name: Deploy ${{ matrix.network }} nodes
needs: [ build, test-configuration-file, versioning ] needs: [ build, versioning, test-configuration-file, test-zebra-conf-path ]
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 60 timeout-minutes: 60
permissions: permissions:
@ -334,7 +274,7 @@ jobs:
# Note: this instances are not automatically replaced or deleted # Note: this instances are not automatically replaced or deleted
deploy-instance: deploy-instance:
name: Deploy single ${{ inputs.network }} instance name: Deploy single ${{ inputs.network }} instance
needs: [ build, test-configuration-file ] needs: [ build, test-configuration-file, test-zebra-conf-path ]
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 30 timeout-minutes: 30
permissions: permissions:
@ -401,7 +341,7 @@ jobs:
failure-issue: failure-issue:
name: Open or update issues for release failures name: Open or update issues for release failures
# When a new job is added to this workflow, add it to this list. # When a new job is added to this workflow, add it to this list.
needs: [ versioning, build, test-configuration-file, deploy-nodes, deploy-instance ] needs: [ versioning, build, deploy-nodes, deploy-instance ]
# Only open tickets for failed or cancelled jobs that are not coming from PRs. # Only open tickets for failed or cancelled jobs that are not coming from PRs.
# (PR statuses are already reported in the PR jobs list, and checked by Mergify.) # (PR statuses are already reported in the PR jobs list, and checked by Mergify.)
if: (failure() && github.event.pull_request == null) || (cancelled() && github.event.pull_request == null) if: (failure() && github.event.pull_request == null) || (cancelled() && github.event.pull_request == null)

0
.github/workflows/delete-gcp-resources.yml → .github/workflows/chore-delete-gcp-resources.yml vendored
View File

0
.github/workflows/project-management.yml → .github/workflows/chore-project-management.yml vendored
View File

2
.github/workflows/build-crates-individually.patch.yml → .github/workflows/ci-build-crates.patch.yml vendored
View File

@ -14,7 +14,7 @@ on:
- '.cargo/config.toml' - '.cargo/config.toml'
- '**/clippy.toml' - '**/clippy.toml'
# workflow definitions # workflow definitions
- '.github/workflows/build-crates-individually.yml' - '.github/workflows/ci-build-crates.yml'
jobs: jobs:
matrix: matrix:

4
.github/workflows/build-crates-individually.yml → .github/workflows/ci-build-crates.yml vendored
View File

@ -22,7 +22,7 @@ on:
- '.cargo/config.toml' - '.cargo/config.toml'
- '**/clippy.toml' - '**/clippy.toml'
# workflow definitions # workflow definitions
- '.github/workflows/build-crates-individually.yml' - '.github/workflows/ci-build-crates.yml'
pull_request: pull_request:
paths: paths:
# production code and test code # production code and test code
@ -34,7 +34,7 @@ on:
- '.cargo/config.toml' - '.cargo/config.toml'
- '**/clippy.toml' - '**/clippy.toml'
# workflow definitions # workflow definitions
- '.github/workflows/build-crates-individually.yml' - '.github/workflows/ci-build-crates.yml'
env: env:
CARGO_INCREMENTAL: ${{ vars.CARGO_INCREMENTAL }} CARGO_INCREMENTAL: ${{ vars.CARGO_INCREMENTAL }}

2
.github/workflows/coverage.patch.yml → .github/workflows/ci-coverage.patch.yml vendored
View File

@ -12,7 +12,7 @@ on:
- '.cargo/config.toml' - '.cargo/config.toml'
- '**/clippy.toml' - '**/clippy.toml'
- 'codecov.yml' - 'codecov.yml'
- '.github/workflows/coverage.yml' - '.github/workflows/ci-coverage.yml'
jobs: jobs:
coverage: coverage:

4
.github/workflows/coverage.yml → .github/workflows/ci-coverage.yml vendored
View File

@ -27,7 +27,7 @@ on:
- '**/clippy.toml' - '**/clippy.toml'
# workflow definitions # workflow definitions
- 'codecov.yml' - 'codecov.yml'
- '.github/workflows/coverage.yml' - '.github/workflows/ci-coverage.yml'
pull_request: pull_request:
paths: paths:
- '**/*.rs' - '**/*.rs'
@ -38,7 +38,7 @@ on:
- '.cargo/config.toml' - '.cargo/config.toml'
- '**/clippy.toml' - '**/clippy.toml'
- 'codecov.yml' - 'codecov.yml'
- '.github/workflows/coverage.yml' - '.github/workflows/ci-coverage.yml'
env: env:
CARGO_INCREMENTAL: ${{ vars.CARGO_INCREMENTAL }} CARGO_INCREMENTAL: ${{ vars.CARGO_INCREMENTAL }}

52
.github/workflows/continous-integration-docker.patch.yml → .github/workflows/ci-integration-tests-gcp.patch.yml vendored
View File

@ -1,4 +1,4 @@
name: CI Docker name: Integration Tests on GCP
# These jobs *don't* depend on cached Google Cloud state disks, # These jobs *don't* depend on cached Google Cloud state disks,
# so they can be skipped when the modified files make the actual workflow run. # so they can be skipped when the modified files make the actual workflow run.
@ -20,10 +20,10 @@ on:
# workflow definitions # workflow definitions
- 'docker/**' - 'docker/**'
- '.dockerignore' - '.dockerignore'
- '.github/workflows/continous-integration-docker.yml' - '.github/workflows/ci-unit-tests-docker.yml'
- '.github/workflows/deploy-gcp-tests.yml' - '.github/workflows/sub-deploy-integration-tests-gcp.yml'
- '.github/workflows/find-cached-disks.yml' - '.github/workflows/sub-find-cached-disks.yml'
- '.github/workflows/build-docker-image.yml' - '.github/workflows/sub-build-docker-image.yml'
jobs: jobs:
# We don't patch the testnet job, because testnet isn't required to merge (it's too unstable) # We don't patch the testnet job, because testnet isn't required to merge (it's too unstable)
@ -39,48 +39,6 @@ jobs:
steps: steps:
- run: 'echo "No build required"' - run: 'echo "No build required"'
test-all:
name: Test all
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-all-getblocktemplate-rpcs:
name: Test all with getblocktemplate-rpcs feature
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-fake-activation-heights:
name: Test with fake activation heights
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-empty-sync:
name: Test checkpoint sync from empty state
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-lightwalletd-integration:
name: Test integration with lightwalletd
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-configuration-file:
name: Test Zebra default Docker config file
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-zebra-conf-path:
name: Test Zebra custom Docker config file
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-stateful-sync: test-stateful-sync:
name: Zebra checkpoint update / Run sync-past-checkpoint test name: Zebra checkpoint update / Run sync-past-checkpoint test
runs-on: ubuntu-latest runs-on: ubuntu-latest

253
.github/workflows/continous-integration-docker.yml → .github/workflows/ci-integration-tests-gcp.yml vendored
View File

@ -1,4 +1,4 @@
name: CI Docker name: Integration Tests on GCP
# Ensures that only one workflow task will run at a time. Previous builds, if # Ensures that only one workflow task will run at a time. Previous builds, if
# already in process, will get cancelled. Only the latest commit will be allowed # already in process, will get cancelled. Only the latest commit will be allowed
@ -62,10 +62,10 @@ on:
- '**/clippy.toml' - '**/clippy.toml'
# workflow definitions # workflow definitions
- 'docker/**' - 'docker/**'
- '.github/workflows/continous-integration-docker.yml' - '.github/workflows/ci-integration-tests-gcp.yml'
- '.github/workflows/deploy-gcp-tests.yml' - '.github/workflows/sub-deploy-integration-tests-gcp.yml'
- '.github/workflows/build-docker-image.yml' - '.github/workflows/sub-build-docker-image.yml'
- '.github/workflows/find-cached-disks.yml' - '.github/workflows/sub-find-cached-disks.yml'
push: push:
branches: branches:
@ -86,10 +86,10 @@ on:
# workflow definitions # workflow definitions
- 'docker/**' - 'docker/**'
- '.dockerignore' - '.dockerignore'
- '.github/workflows/continous-integration-docker.yml' - '.github/workflows/ci-integration-tests-gcp.yml'
- '.github/workflows/deploy-gcp-tests.yml' - '.github/workflows/sub-deploy-integration-tests-gcp.yml'
- '.github/workflows/find-cached-disks.yml' - '.github/workflows/sub-find-cached-disks.yml'
- '.github/workflows/build-docker-image.yml' - '.github/workflows/sub-build-docker-image.yml'
jobs: jobs:
# to also run a job on Mergify head branches, # to also run a job on Mergify head branches,
@ -101,20 +101,20 @@ jobs:
# The default network is mainnet unless a manually triggered workflow or repository variable # The default network is mainnet unless a manually triggered workflow or repository variable
# is configured differently. # is configured differently.
# #
# The outputs for this job have the same names as the workflow outputs in find-cached-disks.yml # The outputs for this job have the same names as the workflow outputs in sub-find-cached-disks.yml
get-available-disks: get-available-disks:
name: Check if cached state disks exist for ${{ inputs.network || vars.ZCASH_NETWORK }} name: Check if cached state disks exist for ${{ inputs.network || vars.ZCASH_NETWORK }}
uses: ./.github/workflows/find-cached-disks.yml uses: ./.github/workflows/sub-find-cached-disks.yml
with: with:
network: ${{ inputs.network || vars.ZCASH_NETWORK }} network: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Check if the cached state disks used by the tests are available for testnet. # Check if the cached state disks used by the tests are available for testnet.
# #
# The outputs for this job have the same names as the workflow outputs in find-cached-disks.yml # The outputs for this job have the same names as the workflow outputs in sub-find-cached-disks.yml
# Some outputs are ignored, because we don't run those jobs on testnet. # Some outputs are ignored, because we don't run those jobs on testnet.
get-available-disks-testnet: get-available-disks-testnet:
name: Check if cached state disks exist for testnet name: Check if cached state disks exist for testnet
uses: ./.github/workflows/find-cached-disks.yml uses: ./.github/workflows/sub-find-cached-disks.yml
with: with:
network: 'Testnet' network: 'Testnet'
@ -125,7 +125,7 @@ jobs:
# testnet when running the image. # testnet when running the image.
build: build:
name: Build CI Docker name: Build CI Docker
uses: ./.github/workflows/build-docker-image.yml uses: ./.github/workflows/sub-build-docker-image.yml
with: with:
dockerfile_path: ./docker/Dockerfile dockerfile_path: ./docker/Dockerfile
dockerfile_target: tests dockerfile_target: tests
@ -135,199 +135,6 @@ jobs:
rust_lib_backtrace: full rust_lib_backtrace: full
rust_log: info rust_log: info
# zebrad tests without cached state
# TODO: make the non-cached-state tests use:
# network: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Run all the zebra tests, including tests that are ignored by default.
# Skips tests that need a cached state disk or a lightwalletd binary.
#
# - We run all the tests behind the `getblocktemplate-rpcs` feature as a separated step.
# - We activate the gRPC feature to avoid recompiling `zebrad`, but we don't actually run any gRPC tests.
#
# TODO: turn this test and the getblocktemplate test into a matrix, so the jobs use exactly the same diagnostics settings
test-all:
name: Test all
runs-on: ubuntu-latest-xl
needs: build
if: ${{ github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
# Run unit, basic acceptance tests, and ignored tests, only showing command output if the test fails.
#
# If some tests hang, add "-- --nocapture" for just that test, or for all the tests.
- name: Run zebrad tests
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK --name zebrad-tests --tty ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} cargo test --locked --release --features "lightwalletd-grpc-tests" --workspace -- --include-ignored
env:
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# zebrad tests without cached state with `getblocktemplate-rpcs` feature
#
# Same as above but we run all the tests behind the `getblocktemplate-rpcs` feature.
test-all-getblocktemplate-rpcs:
name: Test all with getblocktemplate-rpcs feature
runs-on: ubuntu-latest-xl
needs: build
if: ${{ github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run zebrad tests
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK --name zebrad-tests --tty -e ${{ inputs.network || vars.ZCASH_NETWORK }} ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} cargo test --locked --release --features "lightwalletd-grpc-tests getblocktemplate-rpcs" --workspace -- --include-ignored
env:
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Run state tests with fake activation heights.
#
# This test changes zebra-chain's activation heights,
# which can recompile all the Zebra crates,
# so we want its build products to be cached separately.
#
# Also, we don't want to accidentally use the fake heights in other tests.
#
# (The gRPC feature is a zebrad feature, so it isn't needed here.)
test-fake-activation-heights:
name: Test with fake activation heights
runs-on: ubuntu-latest
needs: build
if: ${{ github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run tests with fake activation heights
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK -e TEST_FAKE_ACTIVATION_HEIGHTS --name zebrad-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} cargo test --locked --release --package zebra-state --lib -- --nocapture --include-ignored with_fake_activation_heights
env:
TEST_FAKE_ACTIVATION_HEIGHTS: '1'
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Test that Zebra syncs and checkpoints a few thousand blocks from an empty state.
#
# (We activate the gRPC feature to avoid recompiling `zebrad`, but we don't actually run any gRPC tests.)
test-empty-sync:
name: Test checkpoint sync from empty state
runs-on: ubuntu-latest
needs: build
if: ${{ github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run zebrad large sync tests
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK --name zebrad-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} cargo test --locked --release --features lightwalletd-grpc-tests --package zebrad --test acceptance -- --nocapture --include-ignored sync_large_checkpoints_
env:
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Test launching lightwalletd with an empty lightwalletd and Zebra state.
#
# (We activate the gRPC feature to avoid recompiling `zebrad`, but we don't actually run any gRPC tests.)
test-lightwalletd-integration:
name: Test integration with lightwalletd
runs-on: ubuntu-latest
needs: build
if: ${{ github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run tests with empty lightwalletd launch
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK -e ZEBRA_TEST_LIGHTWALLETD --name lightwalletd-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} cargo test --locked --release --features lightwalletd-grpc-tests --package zebrad --test acceptance -- --nocapture --include-ignored lightwalletd_integration
env:
ZEBRA_TEST_LIGHTWALLETD: '1'
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Test that Zebra works using the default config with the latest Zebra version
test-configuration-file:
name: Test Zebra default Docker config file
timeout-minutes: 15
runs-on: ubuntu-latest
needs: build
if: ${{ github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run tests using the default config
run: |
set -ex
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK --detach --name default-conf-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} zebrad start
EXIT_STATUS=$(docker logs --tail all --follow default-conf-tests 2>&1 | grep -q --extended-regexp --max-count=1 -e 'estimated progress to chain tip.*BeforeOverwinter'; echo $?; )
docker stop default-conf-tests
docker logs default-conf-tests
exit "$EXIT_STATUS"
env:
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Test that Zebra works using the $ZEBRA_CONF_PATH config
test-zebra-conf-path:
name: Test Zebra custom Docker config file
timeout-minutes: 15
runs-on: ubuntu-latest
needs: build
if: ${{ github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run tests using the $ZEBRA_CONF_PATH
run: |
set -ex
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK --detach -e ZEBRA_CONF_PATH --name variable-conf-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} zebrad start
EXIT_STATUS=$(docker logs --tail all --follow variable-conf-tests 2>&1 | grep -q --extended-regexp --max-count=1 -e 'v1.0.0-rc.2.toml'; echo $?; )
docker stop variable-conf-tests
docker logs variable-conf-tests
exit "$EXIT_STATUS"
env:
ZEBRA_CONF_PATH: 'zebrad/tests/common/configs/v1.0.0-rc.2.toml'
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# END TODO: make the non-cached-state tests use:
# network: ${{ inputs.network || vars.ZCASH_NETWORK }}
# zebrad cached checkpoint state tests # zebrad cached checkpoint state tests
# Regenerate mandatory checkpoint Zebra cached state disks. # Regenerate mandatory checkpoint Zebra cached state disks.
@ -340,7 +147,7 @@ jobs:
regenerate-stateful-disks: regenerate-stateful-disks:
name: Zebra checkpoint name: Zebra checkpoint
needs: [ build, get-available-disks ] needs: [ build, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !fromJSON(needs.get-available-disks.outputs.zebra_checkpoint_disk) || github.event.inputs.regenerate-disks == 'true' }} if: ${{ !fromJSON(needs.get-available-disks.outputs.zebra_checkpoint_disk) || github.event.inputs.regenerate-disks == 'true' }}
with: with:
app_name: zebrad app_name: zebrad
@ -369,7 +176,7 @@ jobs:
test-stateful-sync: test-stateful-sync:
name: Zebra checkpoint update name: Zebra checkpoint update
needs: [ regenerate-stateful-disks, get-available-disks ] needs: [ regenerate-stateful-disks, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_checkpoint_disk) || needs.regenerate-stateful-disks.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_checkpoint_disk) || needs.regenerate-stateful-disks.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: zebrad app_name: zebrad
@ -397,7 +204,7 @@ jobs:
test-full-sync: test-full-sync:
name: Zebra tip name: Zebra tip
needs: [ build, get-available-disks ] needs: [ build, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ github.event_name == 'schedule' || !fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || (github.event.inputs.run-full-sync == 'true' && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet') }} if: ${{ github.event_name == 'schedule' || !fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || (github.event.inputs.run-full-sync == 'true' && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet') }}
with: with:
app_name: zebrad app_name: zebrad
@ -440,7 +247,7 @@ jobs:
test-update-sync: test-update-sync:
name: Zebra tip update name: Zebra tip update
needs: [ test-full-sync, get-available-disks ] needs: [ test-full-sync, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: zebrad app_name: zebrad
@ -473,7 +280,7 @@ jobs:
checkpoints-mainnet: checkpoints-mainnet:
name: Generate checkpoints mainnet name: Generate checkpoints mainnet
needs: [ test-full-sync, get-available-disks ] needs: [ test-full-sync, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: zebrad app_name: zebrad
@ -508,7 +315,7 @@ jobs:
test-full-sync-testnet: test-full-sync-testnet:
name: Zebra tip on testnet name: Zebra tip on testnet
needs: [ build, get-available-disks-testnet ] needs: [ build, get-available-disks-testnet ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ (github.event_name == 'schedule' && vars.SCHEDULE_TESTNET_FULL_SYNC == 'true') || !fromJSON(needs.get-available-disks-testnet.outputs.zebra_tip_disk) || (github.event.inputs.run-full-sync == 'true' && (inputs.network || vars.ZCASH_NETWORK) == 'Testnet') }} if: ${{ (github.event_name == 'schedule' && vars.SCHEDULE_TESTNET_FULL_SYNC == 'true') || !fromJSON(needs.get-available-disks-testnet.outputs.zebra_tip_disk) || (github.event.inputs.run-full-sync == 'true' && (inputs.network || vars.ZCASH_NETWORK) == 'Testnet') }}
with: with:
app_name: zebrad app_name: zebrad
@ -554,7 +361,7 @@ jobs:
checkpoints-testnet: checkpoints-testnet:
name: Generate checkpoints testnet name: Generate checkpoints testnet
needs: [ test-full-sync-testnet, get-available-disks-testnet ] needs: [ test-full-sync-testnet, get-available-disks-testnet ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks-testnet.outputs.zebra_tip_disk) || needs.test-full-sync-testnet.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks-testnet.outputs.zebra_tip_disk) || needs.test-full-sync-testnet.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: zebrad app_name: zebrad
@ -587,7 +394,7 @@ jobs:
lightwalletd-full-sync: lightwalletd-full-sync:
name: lightwalletd tip name: lightwalletd tip
needs: [ test-full-sync, get-available-disks ] needs: [ test-full-sync, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
# Currently the lightwalletd tests only work on Mainnet # Currently the lightwalletd tests only work on Mainnet
if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && (github.event_name == 'schedule' || !fromJSON(needs.get-available-disks.outputs.lwd_tip_disk) || github.event.inputs.run-lwd-sync == 'true' ) }} if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && (github.event_name == 'schedule' || !fromJSON(needs.get-available-disks.outputs.lwd_tip_disk) || github.event.inputs.run-lwd-sync == 'true' ) }}
with: with:
@ -627,7 +434,7 @@ jobs:
lightwalletd-update-sync: lightwalletd-update-sync:
name: lightwalletd tip update name: lightwalletd tip update
needs: [ lightwalletd-full-sync, get-available-disks ] needs: [ lightwalletd-full-sync, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.lwd_tip_disk) || needs.lightwalletd-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.lwd_tip_disk) || needs.lightwalletd-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: lightwalletd app_name: lightwalletd
@ -659,7 +466,7 @@ jobs:
lightwalletd-rpc-test: lightwalletd-rpc-test:
name: Zebra tip JSON-RPC name: Zebra tip JSON-RPC
needs: [ test-full-sync, get-available-disks ] needs: [ test-full-sync, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: lightwalletd app_name: lightwalletd
@ -684,7 +491,7 @@ jobs:
lightwalletd-transactions-test: lightwalletd-transactions-test:
name: lightwalletd tip send name: lightwalletd tip send
needs: [ lightwalletd-full-sync, get-available-disks ] needs: [ lightwalletd-full-sync, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.lwd_tip_disk) || needs.lightwalletd-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.lwd_tip_disk) || needs.lightwalletd-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: lightwalletd app_name: lightwalletd
@ -711,7 +518,7 @@ jobs:
lightwalletd-grpc-test: lightwalletd-grpc-test:
name: lightwalletd GRPC tests name: lightwalletd GRPC tests
needs: [ lightwalletd-full-sync, get-available-disks ] needs: [ lightwalletd-full-sync, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.lwd_tip_disk) || needs.lightwalletd-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (inputs.network || vars.ZCASH_NETWORK) == 'Mainnet' && (fromJSON(needs.get-available-disks.outputs.lwd_tip_disk) || needs.lightwalletd-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: lightwalletd app_name: lightwalletd
@ -742,7 +549,7 @@ jobs:
get-block-template-test: get-block-template-test:
name: get block template name: get block template
needs: [ test-full-sync, get-available-disks ] needs: [ test-full-sync, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: zebrad app_name: zebrad
@ -768,7 +575,7 @@ jobs:
submit-block-test: submit-block-test:
name: submit block name: submit block
needs: [ test-full-sync, get-available-disks ] needs: [ test-full-sync, get-available-disks ]
uses: ./.github/workflows/deploy-gcp-tests.yml uses: ./.github/workflows/sub-deploy-integration-tests-gcp.yml
if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }} if: ${{ !cancelled() && !failure() && (fromJSON(needs.get-available-disks.outputs.zebra_tip_disk) || needs.test-full-sync.result == 'success') && github.event.inputs.regenerate-disks != 'true' && github.event.inputs.run-full-sync != 'true' && github.event.inputs.run-lwd-sync != 'true' }}
with: with:
app_name: zebrad app_name: zebrad
@ -789,8 +596,8 @@ jobs:
# #
# This list is for reliable tests that are run on the `main` branch. # This list is for reliable tests that are run on the `main` branch.
# Testnet jobs are not in this list, because we expect testnet to fail occasionally. # Testnet jobs are not in this list, because we expect testnet to fail occasionally.
needs: [ regenerate-stateful-disks, test-full-sync, lightwalletd-full-sync, test-all, test-all-getblocktemplate-rpcs, test-fake-activation-heights, test-empty-sync, test-lightwalletd-integration, test-configuration-file, test-zebra-conf-path, test-stateful-sync, test-update-sync, checkpoints-mainnet, lightwalletd-update-sync, lightwalletd-rpc-test, lightwalletd-transactions-test, lightwalletd-grpc-test, get-block-template-test, submit-block-test ] needs: [ regenerate-stateful-disks, test-full-sync, lightwalletd-full-sync, test-stateful-sync, test-update-sync, checkpoints-mainnet, lightwalletd-update-sync, lightwalletd-rpc-test, lightwalletd-transactions-test, lightwalletd-grpc-test, get-block-template-test, submit-block-test ]
# Only open tickets for failed or cancelled jobs that are not coming from PRs. # Only open tickets for failed scheduled jobs, manual workflow runs, or `main` branch merges.
# (PR statuses are already reported in the PR jobs list, and checked by Mergify.) # (PR statuses are already reported in the PR jobs list, and checked by Mergify.)
if: (failure() && github.event.pull_request == null) || (cancelled() && github.event.pull_request == null) if: (failure() && github.event.pull_request == null) || (cancelled() && github.event.pull_request == null)
runs-on: ubuntu-latest runs-on: ubuntu-latest

0
.github/workflows/lint.patch.yml → .github/workflows/ci-lint.patch.yml vendored
View File

2
.github/workflows/lint.yml → .github/workflows/ci-lint.yml vendored
View File

@ -45,7 +45,7 @@ jobs:
**/Cargo.lock **/Cargo.lock
clippy.toml clippy.toml
.cargo/config.toml .cargo/config.toml
.github/workflows/lint.yml .github/workflows/ci-lint.yml
- name: Workflow files - name: Workflow files
id: changed-files-workflows id: changed-files-workflows

82
.github/workflows/ci-unit-tests-docker.patch.yml vendored Normal file
View File

@ -0,0 +1,82 @@
name: Docker Unit Tests
# These jobs *don't* depend on cached Google Cloud state disks,
# so they can be skipped when the modified files make the actual workflow run.
on:
pull_request:
paths-ignore:
# code and tests
- '**/*.rs'
# hard-coded checkpoints and proptest regressions
- '**/*.txt'
# test data snapshots
- '**/*.snap'
# dependencies
- '**/Cargo.toml'
- '**/Cargo.lock'
# configuration files
- '.cargo/config.toml'
- '**/clippy.toml'
# workflow definitions
- 'docker/**'
- '.dockerignore'
- '.github/workflows/ci-unit-tests-docker.yml'
- '.github/workflows/sub-deploy-integration-tests-gcp.yml'
- '.github/workflows/sub-find-cached-disks.yml'
- '.github/workflows/sub-build-docker-image.yml'
jobs:
# We don't patch the testnet job, because testnet isn't required to merge (it's too unstable)
get-available-disks:
name: Check if cached state disks exist for Mainnet / Check if cached state disks exist
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
build:
name: Build CI Docker / Build images
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-all:
name: Test all
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-all-getblocktemplate-rpcs:
name: Test all with getblocktemplate-rpcs feature
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-fake-activation-heights:
name: Test with fake activation heights
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-empty-sync:
name: Test checkpoint sync from empty state
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-lightwalletd-integration:
name: Test integration with lightwalletd
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-configuration-file:
name: Test CI default Docker config file / Test default-conf in Docker
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-zebra-conf-path:
name: Test CI custom Docker config file / Test custom-conf in Docker
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

269
.github/workflows/ci-unit-tests-docker.yml vendored Normal file
View File

@ -0,0 +1,269 @@
name: Docker Unit Tests
# Ensures that only one workflow task will run at a time. Previous builds, if
# already in process, will get cancelled. Only the latest commit will be allowed
# to run, cancelling any workflows in between
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
on:
workflow_dispatch:
inputs:
network:
default: 'Mainnet'
description: 'Network to deploy: Mainnet or Testnet'
required: true
no_cache:
description: 'Disable the Docker cache for this build'
required: false
type: boolean
default: false
pull_request:
paths:
# code and tests
- '**/*.rs'
# hard-coded checkpoints and proptest regressions
- '**/*.txt'
# test data snapshots
- '**/*.snap'
# dependencies
- '**/Cargo.toml'
- '**/Cargo.lock'
# configuration files
- '.cargo/config.toml'
- '**/clippy.toml'
# workflow definitions
- 'docker/**'
- '.github/workflows/ci-unit-tests-docker.yml'
- '.github/workflows/sub-deploy-integration-tests-gcp.yml'
- '.github/workflows/sub-build-docker-image.yml'
- '.github/workflows/sub-find-cached-disks.yml'
push:
branches:
- main
paths:
# code and tests
- '**/*.rs'
# hard-coded checkpoints and proptest regressions
- '**/*.txt'
# test data snapshots
- '**/*.snap'
# dependencies
- '**/Cargo.toml'
- '**/Cargo.lock'
# configuration files
- '.cargo/config.toml'
- '**/clippy.toml'
# workflow definitions
- 'docker/**'
- '.dockerignore'
- '.github/workflows/ci-unit-tests-docker.yml'
- '.github/workflows/sub-deploy-integration-tests-gcp.yml'
- '.github/workflows/sub-find-cached-disks.yml'
- '.github/workflows/sub-build-docker-image.yml'
jobs:
# Build the docker image used by the tests.
#
# The default network in the Zebra config in the image is mainnet, unless a manually triggered
# workflow or repository variable is configured differently. Testnet jobs change that config to
# testnet when running the image.
build:
name: Build CI Docker
uses: ./.github/workflows/sub-build-docker-image.yml
with:
dockerfile_path: ./docker/Dockerfile
dockerfile_target: tests
image_name: ${{ vars.CI_IMAGE_NAME }}
no_cache: ${{ inputs.no_cache || false }}
rust_backtrace: full
rust_lib_backtrace: full
rust_log: info
# Run all the zebra tests, including tests that are ignored by default.
#
# - We run all the tests behind the `getblocktemplate-rpcs` feature as a separated step.
# - We activate the gRPC feature to avoid recompiling `zebrad`, but we don't actually run any gRPC tests.
#
# TODO: turn this test and the getblocktemplate test into a matrix, so the jobs use exactly the same diagnostics settings
test-all:
name: Test all
runs-on: ubuntu-latest-xl
needs: build
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
# Run unit, basic acceptance tests, and ignored tests, only showing command output if the test fails.
#
# If some tests hang, add "-- --nocapture" for just that test, or for all the tests.
- name: Run zebrad tests
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }}
docker run -e NETWORK --name zebrad-tests --tty ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }} cargo test --locked --release --features "lightwalletd-grpc-tests" --workspace -- --include-ignored
env:
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# zebrad tests without cached state with `getblocktemplate-rpcs` feature
#
# Same as above but we run all the tests behind the `getblocktemplate-rpcs` feature.
test-all-getblocktemplate-rpcs:
name: Test all with getblocktemplate-rpcs feature
runs-on: ubuntu-latest-xl
needs: build
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run zebrad tests
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }}
docker run -e NETWORK --name zebrad-tests --tty -e ${{ inputs.network || vars.ZCASH_NETWORK }} ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }} cargo test --locked --release --features "lightwalletd-grpc-tests getblocktemplate-rpcs" --workspace -- --include-ignored
env:
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Run state tests with fake activation heights.
#
# This test changes zebra-chain's activation heights,
# which can recompile all the Zebra crates,
# so we want its build products to be cached separately.
#
# Also, we don't want to accidentally use the fake heights in other tests.
#
# (The gRPC feature is a zebrad feature, so it isn't needed here.)
test-fake-activation-heights:
name: Test with fake activation heights
runs-on: ubuntu-latest
needs: build
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run tests with fake activation heights
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }}
docker run -e NETWORK -e TEST_FAKE_ACTIVATION_HEIGHTS --name zebrad-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }} cargo test --locked --release --package zebra-state --lib -- --nocapture --include-ignored with_fake_activation_heights
env:
TEST_FAKE_ACTIVATION_HEIGHTS: '1'
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Test that Zebra syncs and checkpoints a few thousand blocks from an empty state.
#
# (We activate the gRPC feature to avoid recompiling `zebrad`, but we don't actually run any gRPC tests.)
test-empty-sync:
name: Test checkpoint sync from empty state
runs-on: ubuntu-latest
needs: build
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run zebrad large sync tests
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }}
docker run -e NETWORK --name zebrad-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }} cargo test --locked --release --features lightwalletd-grpc-tests --package zebrad --test acceptance -- --nocapture --include-ignored sync_large_checkpoints_
env:
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Test launching lightwalletd with an empty lightwalletd and Zebra state.
#
# (We activate the gRPC feature to avoid recompiling `zebrad`, but we don't actually run any gRPC tests.)
test-lightwalletd-integration:
name: Test integration with lightwalletd
runs-on: ubuntu-latest
needs: build
steps:
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- name: Run tests with empty lightwalletd launch
run: |
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }}
docker run -e NETWORK -e ZEBRA_TEST_LIGHTWALLETD --name lightwalletd-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }} cargo test --locked --release --features lightwalletd-grpc-tests --package zebrad --test acceptance -- --nocapture --include-ignored lightwalletd_integration
env:
ZEBRA_TEST_LIGHTWALLETD: '1'
NETWORK: ${{ inputs.network || vars.ZCASH_NETWORK }}
# Test that Zebra works using the default config with the latest Zebra version.
test-configuration-file:
name: Test CI default Docker config file
needs: build
uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'default-conf'
docker_image: ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }}
grep_patterns: '-e "net.*=.*Main.*estimated progress to chain tip.*BeforeOverwinter"'
test_variables: '-e NETWORK'
network: 'Mainnet'
# Test reconfiguring the the docker image for tesnet.
test-configuration-file-testnet:
name: Test CI testnet Docker config file
needs: build
# Make sure Zebra can sync the genesis block on testnet
uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'testnet-conf'
docker_image: ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }}
grep_patterns: '-e "net.*=.*Test.*estimated progress to chain tip.*Genesis" -e "net.*=.*Test.*estimated progress to chain tip.*BeforeOverwinter"'
# TODO: improve the entrypoint to avoid using `ENTRYPOINT_FEATURES=""`
test_variables: '-e NETWORK -e ZEBRA_CONF_PATH="/etc/zebrad/zebrad.toml" -e ENTRYPOINT_FEATURES=""'
network: 'Testnet'
# Test that Zebra works using $ZEBRA_CONF_PATH config
test-zebra-conf-path:
name: Test CI custom Docker config file
needs: build
uses: ./.github/workflows/sub-test-zebra-config.yml
with:
test_id: 'custom-conf'
docker_image: ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}@${{ needs.build.outputs.image_digest }}
grep_patterns: '-e "v1.0.0-rc.2.toml"'
test_variables: '-e NETWORK -e ZEBRA_CONF_PATH="zebrad/tests/common/configs/v1.0.0-rc.2.toml"'
network: ${{ inputs.network || vars.ZCASH_NETWORK }}
failure-issue:
name: Open or update issues for main branch failures
# When a new test is added to this workflow, add it to this list.
#
# This list is for reliable tests that are run on the `main` branch.
# Testnet jobs are not in this list, because we expect testnet to fail occasionally.
needs: [ test-all, test-all-getblocktemplate-rpcs, test-fake-activation-heights, test-empty-sync, test-lightwalletd-integration, test-configuration-file, test-zebra-conf-path ]
# Only open tickets for failed scheduled jobs, manual workflow runs, or `main` branch merges.
# (PR statuses are already reported in the PR jobs list, and checked by Mergify.)
# TODO: if a job times out, we want to create a ticket. Does failure() do that? Or do we need cancelled()?
if: failure() && github.event.pull_request == null
runs-on: ubuntu-latest
steps:
- uses: jayqi/failed-build-issue-action@v1
with:
title-template: "{{refname}} branch CI failed: {{eventName}} in {{workflow}}"
# New failures open an issue with this label.
# TODO: do we want a different label for each workflow, or each kind of workflow?
label-name: S-ci-fail-auto-issue
# If there is already an open issue with this label, any failures become comments on that issue.
always-create-new-issue: false
github-token: ${{ secrets.GITHUB_TOKEN }}

4
.github/workflows/continous-integration-os.patch.yml → .github/workflows/ci-unit-tests-os.patch.yml vendored
View File

@ -1,4 +1,4 @@
name: CI OSes name: Multi-OS Unit Tests
on: on:
pull_request: pull_request:
@ -11,7 +11,7 @@ on:
- '**/deny.toml' - '**/deny.toml'
- '.cargo/config.toml' - '.cargo/config.toml'
- '**/clippy.toml' - '**/clippy.toml'
- '.github/workflows/continous-integration-os.yml' - '.github/workflows/ci-unit-tests-os.yml'
jobs: jobs:
test: test:

45
.github/workflows/continous-integration-os.yml → .github/workflows/ci-unit-tests-os.yml vendored
View File

@ -1,4 +1,4 @@
name: CI OSes name: Multi-OS Unit Tests
# Ensures that only one workflow task will run at a time. Previous builds, if # Ensures that only one workflow task will run at a time. Previous builds, if
# already in process, will get cancelled. Only the latest commit will be allowed # already in process, will get cancelled. Only the latest commit will be allowed
@ -12,6 +12,25 @@ on:
# we build Rust and Zcash parameter caches on main, # we build Rust and Zcash parameter caches on main,
# so they can be shared by all branches: # so they can be shared by all branches:
# https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache # https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache
pull_request:
paths:
# code and tests
- '**/*.rs'
# hard-coded checkpoints and proptest regressions
- '**/*.txt'
# test data snapshots
- '**/*.snap'
# dependencies
- '**/Cargo.toml'
- '**/Cargo.lock'
- '**/deny.toml'
# configuration files
- '.cargo/config.toml'
- '**/clippy.toml'
# workflow definitions
- '.github/workflows/ci-unit-tests-os.yml'
push: push:
branches: branches:
- main - main
@ -31,24 +50,8 @@ on:
- '.cargo/config.toml' - '.cargo/config.toml'
- '**/clippy.toml' - '**/clippy.toml'
# workflow definitions # workflow definitions
- '.github/workflows/continous-integration-os.yml' - '.github/workflows/ci-unit-tests-os.yml'
pull_request: - '.github/workflows/sub-build-docker-image.yml'
paths:
# code and tests
- '**/*.rs'
# hard-coded checkpoints and proptest regressions
- '**/*.txt'
# test data snapshots
- '**/*.snap'
# dependencies
- '**/Cargo.toml'
- '**/Cargo.lock'
- '**/deny.toml'
# configuration files
- '.cargo/config.toml'
- '**/clippy.toml'
# workflow definitions
- '.github/workflows/continous-integration-os.yml'
env: env:
CARGO_INCREMENTAL: ${{ vars.CARGO_INCREMENTAL }} CARGO_INCREMENTAL: ${{ vars.CARGO_INCREMENTAL }}
@ -58,6 +61,9 @@ env:
COLORBT_SHOW_HIDDEN: ${{ vars.COLORBT_SHOW_HIDDEN }} COLORBT_SHOW_HIDDEN: ${{ vars.COLORBT_SHOW_HIDDEN }}
jobs: jobs:
########################################
### Build and test Zebra on all OSes ###
########################################
test: test:
name: Test ${{ matrix.rust }} on ${{ matrix.os }}${{ matrix.features }} name: Test ${{ matrix.rust }} on ${{ matrix.os }}${{ matrix.features }}
# The large timeout is to accommodate: # The large timeout is to accommodate:
@ -233,7 +239,6 @@ jobs:
- name: Install last version of Protoc - name: Install last version of Protoc
uses: arduino/setup-protoc@v2.1.0 uses: arduino/setup-protoc@v2.1.0
with: with:
# TODO: increase to latest version after https://github.com/arduino/setup-protoc/issues/33 is fixed
version: '23.x' version: '23.x'
repo-token: ${{ secrets.GITHUB_TOKEN }} repo-token: ${{ secrets.GITHUB_TOKEN }}

35
.github/workflows/continous-delivery.patch.yml vendored
View File

@ -1,35 +0,0 @@
name: CD
on:
# Only patch the Docker image test jobs
pull_request:
paths-ignore:
# code and tests
- '**/*.rs'
# hard-coded checkpoints and proptest regressions
- '**/*.txt'
# dependencies
- '**/Cargo.toml'
- '**/Cargo.lock'
# configuration files
- '.cargo/config.toml'
- '**/clippy.toml'
# workflow definitions
- 'docker/**'
- '.dockerignore'
- '.github/workflows/continous-delivery.yml'
- '.github/workflows/find-cached-disks.yml'
jobs:
build:
name: Build CD Docker / Build images
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-configuration-file:
name: Test Zebra CD Docker config file
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

26
.github/workflows/continous-integration-docker.patch-always.yml vendored
View File

@ -1,26 +0,0 @@
# These jobs can be skipped based on cached Google Cloud state disks,
# and some of them just run on the `main` branch,
# so the patch jobs always need to run on every PR.
name: CI Docker
on:
pull_request:
jobs:
regenerate-stateful-disks:
name: Zebra checkpoint / Run sync-to-checkpoint test
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
test-full-sync:
name: Zebra tip / Run full-sync test
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'
lightwalletd-full-sync:
name: lightwalletd tip / Run lwd-full-sync test
runs-on: ubuntu-latest
steps:
- run: 'echo "No build required"'

2
.github/workflows/docs.patch.yml → .github/workflows/docs-deploy-firebase.patch.yml vendored
View File

@ -18,7 +18,7 @@ on:
- '.cargo/config.toml' - '.cargo/config.toml'
- '**/clippy.toml' - '**/clippy.toml'
# workflow definitions # workflow definitions
- '.github/workflows/docs.yml' - '.github/workflows/docs-deploy-firebase.yml'
jobs: jobs:
build-docs-book: build-docs-book:

2
.github/workflows/docs.yml → .github/workflows/docs-deploy-firebase.yml vendored
View File

@ -26,7 +26,7 @@ on:
- '.cargo/config.toml' - '.cargo/config.toml'
- '**/clippy.toml' - '**/clippy.toml'
# workflow definitions # workflow definitions
- '.github/workflows/docs.yml' - '.github/workflows/docs-deploy-firebase.yml'
pull_request: pull_request:
branches: branches:

0
.github/workflows/dockerhub-description.yml → .github/workflows/docs-dockerhub-description.yml vendored
View File

0
.github/workflows/zcashd-manual-deploy.yml → .github/workflows/manual-zcashd-deploy.yml vendored
View File

6
.github/workflows/release-binaries.yml vendored
View File

@ -1,5 +1,5 @@
# This workflow is meant to trigger a build of Docker binaries when a release # This workflow is meant to trigger a build of Docker binaries when a release
# is published, it uses the existing `build-docker-image.yml` workflow # is published, it uses the existing `sub-build-docker-image.yml` workflow
# #
# We use a separate action as we might want to trigger this under # We use a separate action as we might want to trigger this under
# different circumstances than a Continuous Deployment, for example. # different circumstances than a Continuous Deployment, for example.
@ -21,7 +21,7 @@ jobs:
# The image will be named `zebra:<semver>` # The image will be named `zebra:<semver>`
build: build:
name: Build Release Docker name: Build Release Docker
uses: ./.github/workflows/build-docker-image.yml uses: ./.github/workflows/sub-build-docker-image.yml
with: with:
dockerfile_path: ./docker/Dockerfile dockerfile_path: ./docker/Dockerfile
dockerfile_target: runtime dockerfile_target: runtime
@ -33,7 +33,7 @@ jobs:
# The image will be named `zebra:<semver>.experimental` # The image will be named `zebra:<semver>.experimental`
build-mining-testnet: build-mining-testnet:
name: Build Release Testnet Mining Docker name: Build Release Testnet Mining Docker
uses: ./.github/workflows/build-docker-image.yml uses: ./.github/workflows/sub-build-docker-image.yml
with: with:
dockerfile_path: ./docker/Dockerfile dockerfile_path: ./docker/Dockerfile
dockerfile_target: runtime dockerfile_target: runtime

1
.github/workflows/release-drafter.yml vendored
View File

@ -37,6 +37,7 @@ jobs:
- uses: release-drafter/release-drafter@v5 - uses: release-drafter/release-drafter@v5
with: with:
config-name: release-drafter.yml config-name: release-drafter.yml
commitish: main
#disable-autolabeler: true #disable-autolabeler: true
env: env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

0
.github/workflows/build-docker-image.yml → .github/workflows/sub-build-docker-image.yml vendored
View File

4
.github/workflows/zcash-lightwalletd.patch.yml → .github/workflows/sub-build-lightwalletd.patch.yml vendored
View File

@ -1,4 +1,4 @@
name: zcash-lightwalletd name: Build lightwalletd
# When the real job doesn't run because the files aren't changed, # When the real job doesn't run because the files aren't changed,
# run a fake CI job to satisfy the branch protection rules. # run a fake CI job to satisfy the branch protection rules.
@ -10,7 +10,7 @@ on:
- 'zebrad/src/config.rs' - 'zebrad/src/config.rs'
- 'zebrad/src/commands/start.rs' - 'zebrad/src/commands/start.rs'
- 'docker/zcash-lightwalletd/Dockerfile' - 'docker/zcash-lightwalletd/Dockerfile'
- '.github/workflows/zcash-lightwalletd.yml' - '.github/workflows/sub-build-lightwalletd.yml'
jobs: jobs:
build: build:

6
.github/workflows/zcash-lightwalletd.yml → .github/workflows/sub-build-lightwalletd.yml vendored
View File

@ -1,6 +1,6 @@
# TODO: we should stop using this build approach with lightwalletd and move to using our # TODO: we should stop using this build approach with lightwalletd and move to using our
# reusable workflow to building all the docker images of our repo # reusable workflow to building all the docker images of our repo
name: zcash-lightwalletd name: Build lightwalletd
# Ensures that only one workflow task will run at a time. Previous builds, if # Ensures that only one workflow task will run at a time. Previous builds, if
# already in process, will get cancelled. Only the latest commit will be allowed # already in process, will get cancelled. Only the latest commit will be allowed
@ -29,7 +29,7 @@ on:
- 'zebrad/src/commands/start.rs' - 'zebrad/src/commands/start.rs'
# these workflow definitions actually change the docker image # these workflow definitions actually change the docker image
- 'docker/zcash-lightwalletd/Dockerfile' - 'docker/zcash-lightwalletd/Dockerfile'
- '.github/workflows/zcash-lightwalletd.yml' - '.github/workflows/sub-build-lightwalletd.yml'
# Update the lightwalletd image when each related PR changes # Update the lightwalletd image when each related PR changes
pull_request: pull_request:
@ -42,7 +42,7 @@ on:
- 'zebrad/src/commands/start.rs' - 'zebrad/src/commands/start.rs'
# these workflow definitions actually change the docker image # these workflow definitions actually change the docker image
- 'docker/zcash-lightwalletd/Dockerfile' - 'docker/zcash-lightwalletd/Dockerfile'
- '.github/workflows/zcash-lightwalletd.yml' - '.github/workflows/sub-build-lightwalletd.yml'
env: env:
IMAGE_NAME: lightwalletd IMAGE_NAME: lightwalletd

8
.github/workflows/zcash-params.yml → .github/workflows/sub-build-zcash-params.yml vendored
View File

@ -1,4 +1,4 @@
name: zcash-params name: Build zcash-params
# Ensures that only one workflow task will run at a time. Previous deployments, if # Ensures that only one workflow task will run at a time. Previous deployments, if
# already in process, won't get cancelled. Instead, we let the first to complete # already in process, won't get cancelled. Instead, we let the first to complete
@ -28,13 +28,13 @@ on:
# workflow definitions # workflow definitions
- 'docker/zcash-params/Dockerfile' - 'docker/zcash-params/Dockerfile'
- '.dockerignore' - '.dockerignore'
- '.github/workflows/zcash-params.yml' - '.github/workflows/sub-build-zcash-params.yml'
- '.github/workflows/build-docker-image.yml' - '.github/workflows/sub-build-docker-image.yml'
jobs: jobs:
build: build:
name: Build Zcash Params Docker name: Build Zcash Params Docker
uses: ./.github/workflows/build-docker-image.yml uses: ./.github/workflows/sub-build-docker-image.yml
with: with:
dockerfile_path: ./docker/zcash-params/Dockerfile dockerfile_path: ./docker/zcash-params/Dockerfile
dockerfile_target: release dockerfile_target: release

8
.github/workflows/deploy-gcp-tests.yml → .github/workflows/sub-deploy-integration-tests-gcp.yml vendored
View File

@ -1,4 +1,4 @@
name: Deploy GCP tests name: Deploy Tests to GCP
on: on:
workflow_call: workflow_call:
@ -311,7 +311,7 @@ jobs:
# Passes the disk name to subsequent steps using $CACHED_DISK_NAME env variable # Passes the disk name to subsequent steps using $CACHED_DISK_NAME env variable
# Passes the state version to subsequent steps using $STATE_VERSION env variable # Passes the state version to subsequent steps using $STATE_VERSION env variable
# #
# TODO: move this script into a file, and call it from find-cached-disks.yml as well. # TODO: move this script into a file, and call it from sub-find-cached-disks.yml as well.
- name: Find ${{ inputs.test_id }} cached state disk - name: Find ${{ inputs.test_id }} cached state disk
id: get-disk-name id: get-disk-name
run: | run: |
@ -414,7 +414,7 @@ jobs:
# - /var/cache/zebrad-cache -> ${{ inputs.root_state_path }}/${{ inputs.zebra_state_dir }} -> $ZEBRA_CACHED_STATE_DIR # - /var/cache/zebrad-cache -> ${{ inputs.root_state_path }}/${{ inputs.zebra_state_dir }} -> $ZEBRA_CACHED_STATE_DIR
# #
# This path must match the variable used by the tests in Rust, which are also set in # This path must match the variable used by the tests in Rust, which are also set in
# `continous-integration-docker.yml` to be able to run this tests. # `ci-unit-tests-docker.yml` to be able to run this tests.
# #
# Although we're mounting the disk root, Zebra will only respect the values from # Although we're mounting the disk root, Zebra will only respect the values from
# $ZEBRA_CACHED_STATE_DIR. The inputs like ${{ inputs.zebra_state_dir }} are only used # $ZEBRA_CACHED_STATE_DIR. The inputs like ${{ inputs.zebra_state_dir }} are only used
@ -485,7 +485,7 @@ jobs:
# delete the whole cache directory.) # delete the whole cache directory.)
# #
# This paths must match the variables used by the tests in Rust, which are also set in # This paths must match the variables used by the tests in Rust, which are also set in
# `continous-integration-docker.yml` to be able to run this tests. # `ci-unit-tests-docker.yml` to be able to run this tests.
# #
# Although we're mounting the disk root to both directories, Zebra and Lightwalletd # Although we're mounting the disk root to both directories, Zebra and Lightwalletd
# will only respect the values from $ZEBRA_CACHED_STATE_DIR and $LIGHTWALLETD_DATA_DIR, # will only respect the values from $ZEBRA_CACHED_STATE_DIR and $LIGHTWALLETD_DATA_DIR,

0
.github/workflows/find-cached-disks.yml → .github/workflows/sub-find-cached-disks.yml vendored
View File

79
.github/workflows/sub-test-zebra-config.yml vendored Normal file
View File

@ -0,0 +1,79 @@
name: Test Zebra Config Files
on:
workflow_call:
inputs:
# Status and logging
test_id:
required: true
type: string
description: 'Unique identifier for the test'
grep_patterns:
required: true
type: string
description: 'Patterns to grep for in the logs'
# Test selection and parameters
docker_image:
required: true
type: string
description: 'Docker image to test'
test_variables:
required: true
type: string
description: 'Environmental variables used to select and configure the test'
network:
required: false
type: string
default: Mainnet
description: 'Zcash network to test against'
jobs:
test-docker-config:
name: Test ${{ inputs.test_id }} in Docker
timeout-minutes: 15
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4.1.0
with:
persist-credentials: false
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
- uses: r7kamura/rust-problem-matchers@v1.4.0
- name: Run ${{ inputs.test_id }} test
run: |
docker pull ${{ inputs.docker_image }}
docker run ${{ inputs.test_variables }} --detach --name ${{ inputs.test_id }} -t ${{ inputs.docker_image }} zebrad start
# Use a subshell to handle the broken pipe error gracefully
(
trap "" PIPE;
docker logs \
--tail all \
--follow \
${{ inputs.test_id }} | \
tee --output-error=exit /dev/stderr | \
grep --max-count=1 --extended-regexp --color=always \
${{ inputs.grep_patterns }}
) || true
LOGS_EXIT_STATUS=$?
docker stop ${{ inputs.test_id }}
EXIT_STATUS=$(docker wait ${{ inputs.test_id }} || echo "Error retrieving exit status");
echo "docker exit status: $EXIT_STATUS";
# If grep found the pattern, exit with the Docker container exit status
if [ $LOGS_EXIT_STATUS -eq 0 ]; then
exit $EXIT_STATUS;
fi
# Handle other potential errors here
echo "An error occurred while processing the logs.";
exit 1;
env:
NETWORK: '${{ inputs.network }}'

2
book/src/dev/continuous-integration.md
View File

@ -90,7 +90,7 @@ This means that the entire workflow must be re-run when a single test fails.
1. Look for the earliest job that failed, and find the earliest failure. 1. Look for the earliest job that failed, and find the earliest failure.
For example, this failure doesn't tell us what actually went wrong: For example, this failure doesn't tell us what actually went wrong:
> Error: The template is not valid. ZcashFoundation/zebra/.github/workflows/build-docker-image.yml@8bbc5b21c97fafc83b70fbe7f3b5e9d0ffa19593 (Line: 52, Col: 19): Error reading JToken from JsonReader. Path '', line 0, position 0. > Error: The template is not valid. ZcashFoundation/zebra/.github/workflows/sub-build-docker-image.yml@8bbc5b21c97fafc83b70fbe7f3b5e9d0ffa19593 (Line: 52, Col: 19): Error reading JToken from JsonReader. Path '', line 0, position 0.
https://github.com/ZcashFoundation/zebra/runs/8181760421?check_suite_focus=true#step:41:4 https://github.com/ZcashFoundation/zebra/runs/8181760421?check_suite_focus=true#step:41:4

6
docker/Dockerfile
View File

@ -13,7 +13,7 @@
# #
# Build zebrad with these features # Build zebrad with these features
# Keep these in sync with: # Keep these in sync with:
# https://github.com/ZcashFoundation/zebra/blob/main/.github/workflows/build-docker-image.yml#L37 # https://github.com/ZcashFoundation/zebra/blob/main/.github/workflows/sub-build-docker-image.yml#L37
ARG FEATURES="default-release-binaries" ARG FEATURES="default-release-binaries"
ARG TEST_FEATURES="lightwalletd-grpc-tests zebra-checkpoints" ARG TEST_FEATURES="lightwalletd-grpc-tests zebra-checkpoints"
@ -190,8 +190,8 @@ ARG FEATURES
ENV FEATURES=${FEATURES} ENV FEATURES=${FEATURES}
# Path and name of the config file # Path and name of the config file
ENV ZEBRA_CONF_DIR=/etc/zebrad ENV ZEBRA_CONF_DIR=${ZEBRA_CONF_DIR:-/etc/zebrad}
ENV ZEBRA_CONF_FILE=zebrad.toml ENV ZEBRA_CONF_FILE=${ZEBRA_CONF_FILE:-zebrad.toml}
# Expose configured ports # Expose configured ports
EXPOSE 8233 18233 EXPOSE 8233 18233

5
docker/entrypoint.sh
View File

@ -79,7 +79,7 @@ fi
: "${ENTRYPOINT_FEATURES:=}" : "${ENTRYPOINT_FEATURES:=}"
# Configuration file path # Configuration file path
if [[ -n "${ZEBRA_CONF_DIR}" ]] && [[ -n "${ZEBRA_CONF_FILE}" ]]; then if [[ -n "${ZEBRA_CONF_DIR}" ]] && [[ -n "${ZEBRA_CONF_FILE}" ]] && [[ -z "${ZEBRA_CONF_PATH}" ]]; then
ZEBRA_CONF_PATH="${ZEBRA_CONF_DIR}/${ZEBRA_CONF_FILE}" ZEBRA_CONF_PATH="${ZEBRA_CONF_DIR}/${ZEBRA_CONF_FILE}"
fi fi
@ -90,8 +90,7 @@ fi
# Users have to opt-in to additional functionality by setting environmental variables. # Users have to opt-in to additional functionality by setting environmental variables.
if [[ -n "${ZEBRA_CONF_PATH}" ]] && [[ ! -f "${ZEBRA_CONF_PATH}" ]] && [[ -z "${ENTRYPOINT_FEATURES}" ]]; then if [[ -n "${ZEBRA_CONF_PATH}" ]] && [[ ! -f "${ZEBRA_CONF_PATH}" ]] && [[ -z "${ENTRYPOINT_FEATURES}" ]]; then
# Create the conf path and file # Create the conf path and file
mkdir -p "${ZEBRA_CONF_DIR}" || { echo "Error creating directory ${ZEBRA_CONF_DIR}"; exit 1; } (mkdir -p "$(dirname "${ZEBRA_CONF_PATH}")" && touch "${ZEBRA_CONF_PATH}") || { echo "Error creating file ${ZEBRA_CONF_PATH}"; exit 1; }
touch "${ZEBRA_CONF_PATH}" || { echo "Error creating file ${ZEBRA_CONF_PATH}"; exit 1; }
# Populate the conf file # Populate the conf file
cat <<EOF > "${ZEBRA_CONF_PATH}" cat <<EOF > "${ZEBRA_CONF_PATH}"
[network] [network]