* zebra-chain changes from the subtree-boundaries branch
```sh
git checkout -b subtree-boundaries-zebra-chain main
git checkout origin/subtree-boundaries zebra-chain
git commit
```
* Temporarily populate new subtree fields with None - for revert
This temporary commit needs to be reverted in the next PR.
* Applies suggestions from code review
* removes from_repr_unchecked methods
* simplifies loop
* adds subtrees to zebra-state
* uses split_at, from_repr, & updates state-db-upgrades.md
* Update book/src/dev/state-db-upgrades.md
Co-authored-by: teor <teor@riseup.net>
* renames partial_subtree to subtree_data
* tests that subtree serialization format
* adds raw data format serialization round-trip test
* decrements minor version and skips inserting subtrees in db
---------
Co-authored-by: teor <teor@riseup.net>
* Move the state format into a new doc
* Add upgrade instructions
* Link to the format upgrade docs from the upgrade code
* Fix typo
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* Add `fastmod --hidden` to mass-renames.md
* Do multiple skip paths
* Do multiple sed renames in the same sed
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* rename verifiers
* rename `PreparedBlock` to `SemanticallyVerifiedBlock`
* rename `CommitBlock` to `SemanticallyVerifiedBlock`
* rename `FinalizedBlock` to `CheckpointVerifiedBlock`
* rename `CommitFinalizedBlock` to `CommitCheckpointVerifiedBlock`
* rename `FinalizedWithTrees` to `ContextuallyVerifiedBlockWithTrees`
* rename `ContextuallyValidBlock` to `ContextuallyVerifiedBlock`
* change some `finalized` variables or function arguments to `checkpoint_verified`
* fix docs
* document the difference between `CheckpointVerifiedBlock` and `ContextuallyVerifiedBlock`
* fix doc links
* apply suggestions to request
Co-authored-by: Marek <mail@marek.onl>
* apply suggestions to service
Co-authored-by: Marek <mail@marek.onl>
* apply suggestions to finalized_state.rs and write.rs
Co-authored-by: Marek <mail@marek.onl>
* fmt
* change some more variable names
* change a few missing generics
* fix checkpoint log issue
* rename more `prepared` vars `semantically_verified`
* fix test regex
* fix test regex 2
---------
Co-authored-by: Marek <mail@marek.onl>
* refuse to run Zebra if it is too old
* update the release checklist to consider the constants
* bring newline back
* apply new end of support code
* attempt to add tests (not working yet)
* move eos to progress task
* move tests
* add acceptance test (not working)
* fix tests
* change to block height checks (ugly code)
* change warn days
* refactor estimated blocks per day, etc
* move end of support code to its own task
* change test
* fix some docs
* move constants
* remove uneeded conversions
* downgrade tracing
* reduce end of support time, fix ci changing debugs to info again
* update instructions
* add failure messages
* cargo lock update
* unify releaase name constant
* change info msg
* clippy fixes
* add a block explorer
* ignore testnet in end of support task
* change panic to 16 weeks
* add some documentation about end of support
* Tweak docs wording
---------
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Update audit scope based on audit-v1.0.0-rc.0 branch
* Minor wording fixes and clarifications
Co-authored-by: Arya <aryasolhi@gmail.com>
* Set scope for `reddsa` and explain code movement
* Just base everything on rc.0
* I'm just about to make it into a tag, not a branch
* Make it clearer that the Crates categories are Zebra crates
* Add reddsa and redjubjub
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
* Fix links to red(jubjub|dsa) signing_key
* Add missing space
* Remove `redjubjub` from "Out of scope"
* Fix formatting
* Make previous audit clearer
* Add missing space
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
* make `zebra-checkpoint` util work with zebra as the backend
* update snapshots
* update documentation
* applies suggestions from code review
* irefactor zebra-checkpoints to work with zebra using deserialization of the raw block
* fix imports and derives
* rename mode to backend
* remove old stuff
* fix docs
Co-authored-by: arya2 <aryasolhi@gmail.com>
* Updates design overview page of book with:
- a simplified graph of service dependencies in design overview
- links to notable blog posts
- removals of some out-of-date text
also corrects issue/pr refs in an rfc
* Updates service dependencies diagram
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* updates diagram and adds doc links
* uses opacity to indicate clickability instead of drop shadow
Co-authored-by: teor <teor@riseup.net>
* Clean up references to fpe that were not removed when we removed orchard code
* Update Cargo.lock
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Fix the branch name in the release template
* Use a docker command with colour and Ctrl-C support
* Make branch name example more readable
* Fix a link typo
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* add a list of dependencies for auditors
* change intro
Co-authored-by: teor <teor@riseup.net>
* add zebra-checkpoints to audit
* replace colors in the legend as they don't render in github
* make crypto deps out of the scope
* add a reason to keep rocksdb out of the scope
* add a not of why chrono is out of the scope
* add note for bitvec dep
* move structopt to out of scope
* add zcash_script as partially in scope
* applies suggestion, updates versions, bolds ed25519-zebra
* adds zcash_proofs audit notes
* Splits list of zebra deps into sections for full audits, partial audits, and out of scope.
* Update book/src/dev/zebra-dependencies-for-audit.md
Co-authored-by: teor <teor@riseup.net>
* Update book/src/dev/zebra-dependencies-for-audit.md
* Add missing newline
* Update book/src/dev/zebra-dependencies-for-audit.md
Co-authored-by: teor <teor@riseup.net>
* updates crate versions
* Update zcash_script to 0.1.8
* Only link to the audit version of each crate or file
* Clarify out of scope versions
* Fix source paths
* Fix zcash_script paths
* Improve formatting
* Update zebra-dependencies-for-audit.md
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: arya2 <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Document parameter download workaround, particularly for macOS
* Replace outdated build docs with a link to the README
* Remove incorrectly rendered heading links
* Add an initial draft audit process
* Add dependency criteria
* draft
* docs(release): add information on how we tag and release
* docs(release): remove outdated document
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Remove rough draft analysis of alternatives
Co-authored-by: teor <teor@riseup.net>
* Explain how to find Mergify failures
* Explain how to fix cache errors
* Fix instructions - clear all caches
* Fix which errors need which actions
* Add a newline to appease GitHub markdown renderer
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Delete test instances after 3 days
* Use correct delete command, improve shell quoting
* Use sed to provide the correct zone or region
* Fix quoting
* Fix IFS
* Fix IFS for multiple disks
* Document why we can't quote some shell variables
* Document that instances can get deleted
* Fix exact names in deletion docs
* Increase the full sync timeout to 28 hours
And put the timeout constant with the rest of them.
* Update developer documentation for the new constant location
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Fix delete GCP resources commands
* Don't create a GCP credentials file
* Keep the latest 2 images
* Explain time
* Show the names of disks that are being deleted
* Actually run the image delete steps
* Only delete commit-based instance templates
* Document automated deletion
* Explain how to resolve duplicate dependency warnings
* Explain why the errors happen and add the CI job name
* Add extra steps
* Add some examples, tools, and extra steps
* feat(build): deploy long running instances on release
Previous behavior:
Each time we merged to main new nodes would be deployed, this is an
expected behavior as we need to ensure nodes get deployed and run
without issues, but this could also replace nodes very hastily.
Expected behavior:
We want instances which would run for a longer time, to allow us to
troubleshoot issues or inspect the behavior of this instances for longer
periods of time (2+ weeks)
Applied solution:
Deploy a versioned manage instance group (MiG) using the major version
of the release semver. We just use the first part of the version to
replace old instances, and change it when a major version is released
to keep a segregation between new and old versions.
* ci(build): allow v0 as a major version tag
* fix(build): use rust conventions for versioning
* fix(deploy): improve documentation and trigger on release
* Update .github/workflows/continous-delivery.yml
Co-authored-by: teor <teor@riseup.net>
* fix(versioning): typo
* fix(deploy): use `zebrad-v1` as the instance name, with no SHA
* fix(deploy): create and update MiG must use the same name
* docs(deployments): add Continuous Delivery process
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Save cached state on full syncs and updates
* Add an -update suffix to CI images created by updating cached state
* Make disk image names unique by adding a time suffix
* Use the latest image from any branch, but prefer the current commit if available
* Document Zebra's continuous integration tests
* Fix typos in environmental variable names
* Expand documentation
* Fix variable name typo
* Fix shell syntax
* Rename `block_by_height` to `block_header_by_height` in fin state
* Rename `tx_by_hash` to `tx_loc_by_hash` in both (non & fin) states
* Rename `utxo_by_outpoint` to `utxo_by_out_loc` in finalized state
* Reorder the column families so that they match the docs
* Update `struct Chain` in the RFCs
* Increment `DATABASE_FORMAT_VERSION` to 25
* Remove obsolete docs from `0004-asynchronous-script-verification.md`
* Remove an obsolete `TODO` from `disk_db.rs`
* Delete unused snapshots
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Re-order column families in design in dependency order
* Minor RFC design tweaks and fixes
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Simplify the database design using prefix iterators
* Fix typos and missed changes
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Marek <mail@marek.onl>
* refactor(db): simplify block height serialization
* refactor(db): make height serialization length generic
* refactor(db): create a TransactionIndex type
This changes the names of some snapshot types,
but doesn't change any data.
* refactor(db): create transparent OutputIndex and OutputLocation types
This keeps the same serialization, to avoid changing the database version.
* doc(rfc/db): make transparent database type names consistent
* doc(rfc/db): fix a bug in the Utxo.is_coinbase derivation
* fix(db): use the correct serialized size for OutputLocation
teor
Arya
Deirdre Connolly
Alfredo Garcia
Jack Gavigan
Pili Guerra
Gustavo Valverde
Marek
Conrado Gouvea