* Remove duplicate asserts
There are the same two asserts above the two removed ones.
* Remove workarounds for inserting trees into NFS
NFS = non finalized state
* Use correct height for constructing new chain
We were using the height of the last block instead of the initial block
to construct a new chain.
* Don't push the 0th block into a chain
* Don't commit two blocks at the same height
* Fix typo
* Generate chains with at least two blocks
---------
Co-authored-by: teor <teor@riseup.net>
* Add an async-error feature and an initial module structure
* Implement checking for panics in OS threads and async tasks
* Implement waiting for panics in OS threads and async tasks
* Add a TODO to simplify some state request error handling
* Use the new panic-checking methods in zebra-state
* Use new panic-checking methods in zebra-network
* fixup! Implement waiting for panics in OS threads and async tasks
* Replace existing async code with generic panic-checking methods
* Simplify trait to a single method
* Move thread panic code into generic trait impls
* Simplify option handling
Co-authored-by: Arya <aryasolhi@gmail.com>
* Fix comment
Co-authored-by: Arya <aryasolhi@gmail.com>
* Add missing track_caller
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* update ecc dependencies no serialization
* use zcash_primitives merke tree functions for serialization/deserialization in orchard and sapling trees
* some more work in sapling/orchard serialization/deserialization
* clippy
* fix doc links
* fix missing doc
* make orchard trees serializa/deserialize as they were before upgrade
* make sapling trees serialize/deserialize as they were before upgrade
* use legacy for sprout
* remove unused code
* readd snapshot tests
* repalce some code
* upgrade zcash_proofs
* remove legacy code for sprout
* fix the count method
* add root to serialize
* fixes in as_bytes and from_bytes
* use legacy code
* add todo about pow2 tests
* remove unused sprout code
* fix doc typos
* Add a recalculate_root() method to trees for tests
* Rename test tree types to make them easier to change
* Add TODOs for tests for old and new serialization formats
* fix doc typos
* add more test to note commitment trees
* fix comment
* fix leaf serializatiuon, reverse to old hashes in pow2 tests
* fix serialization
* put sapling SerializedTree code back
* put orchard SerializedTree code back
* clippy
* add duplicated dependencies until zebra_script updates
* fix a doc link
* minor cleanup
* remove todo comment from tests
* add one more check to tests
* update zebra_script
* update deny.toml
* replace custom function with library
* fix some tests
* update docs
* Remove duplicate dependencies from deny.toml
Based on the list at:
https://github.com/ZcashFoundation/zebra/actions/runs/5557139662/jobs/10150543673?pr=7053#step:5:14
---------
Co-authored-by: teor <teor@riseup.net>
* Move format upgrades to their own module and enum
* Launch a format change thread if needed, and shut it down during shutdown
* Add some TODOs and remove a redundant timer
* Regularly check for panics in the state upgrade task
* Only run example upgrade once, change version field names
* Increment database format to 25.0.2: add format change task
* Log the running and initial disk database format versions on startup
* Add initial disk and running state versions to cached state images in CI
* Fix missing imports
* Fix typo in logs workflow command
* Add a force_save_to_disk argument to the CI workflow
* Move use_internet_connection into zebrad_config()
* fastmod can_spawn_zebrad_for_rpc can_spawn_zebrad_for_test_type zebra*
* Add a spawn_zebrad_without_rpc() function
* Remove unused copy_state() test code
* Assert that upgrades and downgrades happen with the correct versions
* Add a kill_and_return_output() method for tests
* Add a test for new_state_format() versions (no upgrades or downgrades)
* Add use_internet_connection to can_spawn_zebrad_for_test_type()
* Fix workflow parameter passing
* Check that reopening a new database doesn't upgrade (or downgrade) the format
* Allow ephemeral to be set to false even if we don't have a cached state
* Add a test type that will accept any kind of state
* When re-using a directory, configure the state test config with that path
* Actually mark newly created databases with their format versions
* Wait for the state to be opened before testing the format
* Run state format tests on mainnet and testnet configs (no network access)
* run multiple reopens in tests
* Test upgrades run correctly
* Test that version downgrades work as expected (best effort)
* Add a TODO for testing partial updates
* Fix missing test arguments
* clippy if chain
* Fix typo
* another typo
* Pass a database instance to the format upgrade task
* Fix a timing issue in the tests
* Fix version matching in CI
* Use correct env var reference
* Use correct github env file
* Wait for the database to be written before killing Zebra
* Use correct workflow syntax
* Version changes aren't always upgrades
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add a recalculate_root() method to trees for tests
* Rename test tree types to make them easier to change
* Add TODOs for tests for old and new serialization formats
* fix doc typos
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add an assert_frontier_eq() method to note commitment trees for tests
* Check round-trip serialization for note commitment trees
* fix typos
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove duplicate "running" message, send log file opening logs to stderr
* Add missing block progress bar desc, make them consistent
* Put progress bars in a specific order in each section
* Actually make block progress bar desc consistent
* Make progress bar order even more consistent
* Fix fork blocks plural for 1 block
* Use the correct number of chain fork bars
* Disable confusing partial work display
* Add struct field category comments
* Silence a verbose inventory log
* claridy some checkpoint verifier docs
* update documentation of `Request::CommitSemanticallyVerifiedBlock` and `Request::CommitCheckpointVerifiedBlock`
* replace `prepared` with `semantically_verified` in state service checks code
* replace `non-finalized` where needed in docs of the state service
* fix double space in doc
* replace `finalized` where needed in docs of the state service
* change some docs in state queued_blocks.rs
* Rewrite pending UTXO checkpoint block comment
* Fix trailing space in docs
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
---------
Co-authored-by: teor <teor@riseup.net>
* Activate production features on docs.rs
* Make version requirements more flexible to work around unpublished versions
* Remove a redundant feature requirement that causes publishing issues
* fix release auto-replacement format
* cargo release replace --verbose --execute --package zebrad
* Add a missing 1.0.1 changelog entry
* Fix incorrect commands in the release checklist
* Check the crates.io release script in CI, to avoid release failures
* Fix release date in changelog
* Fix workflow typo
* Add --no-confirm to the workflow
* Fix outdated version in release check command
* After the release, add a "remove do-not-merge" step
* Set git identity before committing
* Don't overwrite tweaked versions on upgrade
* cargo release version doesn't support --dependent-version
* Update the same commands in the release checklist
* Commit to git after each change
* And in the checklist
* Add cargo release config
* In CI, allow any branch and provide a fake previous tag name
* Remove redundant `cargo release commit`
* Use a simpler release version process
* Start simplifying the release dry run in CI
* Fix a cargo dependency lint
* just skip the failing crates for now
* Fix the release checklist to match CI
* Use the latest versions of all Zebra dependencies
* Update license description in README for MIT-only crates
* Draft changelog with trivial issues
* Remove trivial issues
* Update changelog entries as of commit 2a31972 and PR #7103
* Update mainnet and testnet checkpoints as of 2023-06-30
* chore: Release
* Estimate release height for Zebra v1.0.1
Block height 2139118 at 2023-06-30 01:55:38 UTC
Release is likely to be 2023-07-01
2139118 + 1152 * 3 = 2142574
Then round up to the nearest 1000.
* Add methods for loading entire column families from the database
* Add a method that loads all the sprout trees from the database
* Add snapshot tests for sprout note commitment trees
* Add round-trip proptests for tree root database serialization
* Add a manual sprout note commitment tree database serialization snapshot test
* Add tests for 1,2,4,8 note commitments in a tree
* Remove redundant "rand" package rename in dependencies
* Randomly cache roots rather than only caching even roots
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add and use `FinalizableBlock`
This commit adds `FinalizableBlock`, and uses it instead of
`ContextuallyVerifiedBlockWithTrees` in `commit_finalized_direct()`
* Use `ContextuallyVerifiedBlockWithTrees`
This commit passes `ContextuallyVerifiedBlockWithTrees` instead of
passing separate `finalized`, `history_tree` and `note_commitment_trees`
when storing blocks in the finalized state.
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* add docs to new methods
* fix existing doc
* rename `ContextuallyVerifiedBlockWithTrees` to `SemanticallyVerifiedBlockWithTrees`
* Refactor docs
* Refactor comments
* Add missing docs, fix typo
* Fix rustfmt
---------
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Remove openssl dependency and prevent it coming back
* Put the arguments in the right place
* Put comment in the right place
* Add a default-docker feature to zebrad and use it in workflows and Docker files
* Fix a comment typo
* Make sure that Docker production builds don't use openssl
* Rename feature to default-release-binaries
* Use Arc::into_inner() to avoid potential concurrency issues
* Remove some outdated clippy lint workarounds (fixed in Rust 1.66)
* Update the required Rust version to 1.70
* Refactor `CheckpointVerifiedBlock`
This commit turns `CheckpointVerifiedBlock` into a wrapper of
`SemanticallyVerifiedBlock` since both structs have the same fields.
* Refactor `ContextuallyVerifiedBlockWithTrees`
This commit uses `SemanticallyVerifiedBlock` in
`ContextuallyVerifiedBlockWithTrees` instead of
`CheckpointVerifiedBlock`.
* fix: prevent progress bar from panicking using workaround
* remove more progress bar lengths
* re-add set_len() calls commented out; restore one that shouldn't cause panics
* Update zebra-state/src/service/non_finalized_state.rs
Co-authored-by: teor <teor@riseup.net>
---------
Co-authored-by: teor <teor@riseup.net>
* change(crates): Publish tower-batch-control
* Cargo fields for zebra-test
* Cargo fields for tower-fallback
* Cargo fields for zebra-chain
* Cargo fields for zebra-consensus
* Cargo fields for zebra-network
* Cargo fields for zebra-node-services
* Cargo fields for zebra-rpc
* Cargo fields for zebra-script
* Cargo fields for zebra-state
* Cargo fields for zebrad
* Cargo fields for zebra-test
* Include zebra-test version and path for zebra-chain
* Include zebra-chain version and path for zebra-node-services
* Include zebra-chain version and path for zebra-script
* Include zebra-chain and zebra-test versions and paths for zebra-state
* Include zebra-* versions and paths for zebra-consensus
* Include zebra-* and tower-fallback versions and paths for tower-batch-control
* Include tower-* versions and paths for zebra-consensus
* Fixed tower-fallback license string
* Remove versions for dev-dependencise
* Update zebra-rpc/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* tower-batch-control license MUST remain MIT only
* Align zebra-network with upstream tower's MIT license
* Update zebra-test/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* Update zebra-utils/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* fix(legal): Use correct licenses for tower-batch-control and zebra-network (#6927)
* Don't modify whitespace
* Resolve rustdoc and cargo-release warnings
* Add missing zebra internal dependency versions
* Workaround missing zebra-utils test dependency in zebrad test-only feature
* Fix missing zebra internal dependency versions
* Add README, home page, and search terms to every crate
* Fix README publish warnings
* Change tower crates to beta versions
* Add zebra-utils to the list of allowed unused dependencies
* Update tower-batch-control/Cargo.toml
* Update tower-fallback/Cargo.toml
---------
Co-authored-by: teor <teor@riseup.net>
* Implement minor and patch database format versions
* Log and update database format versions when opening database
* Refactor the current list of column families into a constant
* Open all available column families, including from future Zebra versions
* Refactor note commitment tree lookups to go through the height methods
* Make Sapling/Orchard note commitment tree lookup forwards compatible
* Ignore errors reading column family lists from disk
* Update format version comments and TODOs
* Correctly log newly created database formats
* Fix a new cargo lint about resolver versions
* cargo clippy --fix --all-features --all-targets
* cargo fmt --all
* Add missing tokio feature in the state, revealed by the new resolver
* Add missing dev dependencies in zebra-node-services
* Add a missing `tokio` feature from PR #6813
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Enable cancel_all_background_work() only on macOS
* Ignore expected "during shutdown" errors, and log other errors
* Disable cancel_all_background_work() but keep the updated docs and error handling
* Add the macOS shutdown crash to the README known issues
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Implement minor and patch database format versions
* Log and update database format versions when opening database
* Refactor the current list of column families into a constant
* Open all available column families, including from future Zebra versions
* Refactor note commitment tree lookups to go through the height methods
* Make Sapling/Orchard note commitment tree lookup forwards compatible
* Ignore errors reading column family lists from disk
* Update format version comments and TODOs
* Correctly log newly created database formats
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Rewrite some state cache docs to clarify
* Add a zebra_network::Config.cache_dir for peer address caches
* Add new config test files and fix config test failure message
* Create some zebra-chain and zebra-network convenience functions
* Add methods for reading and writing the peer address cache
* Add cached disk peers to the initial peers list
* Add metrics and logging for loading and storing the peer cache
* Replace log of useless redacted peer IP addresses
* Limit the peer cache minimum and maximum size, don't write empty caches
* Add a cacheable_peers() method to the address book
* Add a peer disk cache updater task to the peer set tasks
* Document that the peer cache is shared by multiple instances unless configured otherwise
* Disable peer cache read/write in disconnected tests
* Make initial peer cache updater sleep shorter for tests
* Add unit tests for reading and writing the peer cache
* Update the task list in the start command docs
* Modify the existing persistent acceptance test to check for peer caches
* Update the peer cache directory when writing test configs
* Add a CacheDir type so the default config can be enabled, but tests can disable it
* Update tests to use the CacheDir config type
* Rename some CacheDir internals
* Add config file test cases for each kind of CacheDir config
* Panic if the config contains invalid socket addresses, rather than continuing
* Add a network directory to state cache directory contents tests
* Add new network.cache_dir config to the config parsing tests
* rename verifiers
* rename `PreparedBlock` to `SemanticallyVerifiedBlock`
* rename `CommitBlock` to `SemanticallyVerifiedBlock`
* rename `FinalizedBlock` to `CheckpointVerifiedBlock`
* rename `CommitFinalizedBlock` to `CommitCheckpointVerifiedBlock`
* rename `FinalizedWithTrees` to `ContextuallyVerifiedBlockWithTrees`
* rename `ContextuallyValidBlock` to `ContextuallyVerifiedBlock`
* change some `finalized` variables or function arguments to `checkpoint_verified`
* fix docs
* document the difference between `CheckpointVerifiedBlock` and `ContextuallyVerifiedBlock`
* fix doc links
* apply suggestions to request
Co-authored-by: Marek <mail@marek.onl>
* apply suggestions to service
Co-authored-by: Marek <mail@marek.onl>
* apply suggestions to finalized_state.rs and write.rs
Co-authored-by: Marek <mail@marek.onl>
* fmt
* change some more variable names
* change a few missing generics
* fix checkpoint log issue
* rename more `prepared` vars `semantically_verified`
* fix test regex
* fix test regex 2
---------
Co-authored-by: Marek <mail@marek.onl>
dependabot[bot]
Marek
teor
Alfredo Garcia
Conrado Gouvea
Deirdre Connolly