Commit Graph

36 Commits

Author SHA1 Message Date
teor 9df78ffdba
change(tests): Do round-trip tests for note commitment tree data structure and RPC serialisation (#7147)
* Add an assert_frontier_eq() method to note commitment trees for tests

* Check round-trip serialization for note commitment trees

* fix typos

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-07-06 01:04:28 +00:00
dependabot[bot] 45a96b5adc
build(deps): bump serde from 1.0.156 to 1.0.158 (#6358)
* build(deps): bump serde from 1.0.156 to 1.0.158

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.156 to 1.0.158.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.156...v1.0.158)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make MERKLE_DEPTH into a u8 to avoid a spurious serde_derive warning

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2023-03-22 17:12:34 +00:00
teor 9452487c61
fix(state): Fix minute-long delays in block verification after a chain fork (#6122)
* Split Chain fields into sections

* Replace Chain.sprout_note_commitment_tree with a lookup method

* Add TODOs

* Show full debug info when tests fail because chains aren't equal

* Print sprout and sapling tree Nodes as hex when debugging

* Correctly revert temporary finalized tip trees and anchors

* Fix tests

* Refactor removal functions

* Replace the Chain.sapling_note_commitment_tree field with a lookup method

* Replace the Chain.orchard_note_commitment_tree field with a lookup method

* Replace the Chain.history_tree field with a lookup method and remove redundant code

* Update comments

* Ignore clippy::unwrap_in_result

* Remove redundant fork() Result

* Put conditional code in blocks

* fastmod history_tree_at_tip history_block_commitment_tree zebra-state
2023-02-13 21:44:31 +00:00
teor 66ea40f215
Resolve unnecessary borrow lint and remove redundant imports (#5045)
```sh
cargo clippy --fix --all-features --all-targets
```
2022-09-02 18:20:32 +00:00
teor 6ad445eb97
1. fix(perf): Run CPU-intensive state updates in parallel rayon threads (#4802)
* Split disk reads from CPU-heavy Sprout interstitial tree cryptography

* Improve anchor validation debugging and error messages

* Work around a test data bug, and save some CPU

* Remove redundant checks for empty shielded data

* Skip generating unused interstitial treestates

* Do disk fetches and quick checks, then CPU-heavy cryptography

* Wrap HistoryTree in an Arc in the state

* Run CPU-intensive chain validation and updates in parallel rayon threads

* Refactor to prepare for parallel tree root calculations

* Run finalized state note commitment tree root updates in parallel rayon threads

* Update finalized state note commitment trees using parallel rayon threads

* Fix a comment typo and add a TODO

* Split sprout treestate fetch into its own function

* Move parallel note commitment trees to zebra-chain

* Re-calculate the tree roots in the same parallel batches

* Do non-finalized note commitment tree updates in parallel threads

* Update comments about note commitment tree rebuilds

* Do post-fork tree updates in parallel threads

* Add a TODO for parallel tree updates in tests

* Fix broken intra-doc links

* Clarify documentation for sprout treestates

* Sort Cargo.toml dependencies
2022-07-22 12:19:11 -04:00
Alfredo Garcia 97fb85dca9
lint(clippy): add `unwrap_in_result` lint (#4667)
* `unwrap_in_result` in zebra-chain crate

* `unwrap_in_result` in zebra-script crate

* `unwrap_in_result` in zebra-state crate

* `unwrap_in_result` in zebra-consensus crate

* `unwrap_in_result` in zebra-test crate

* `unwrap_in_result` in zebra-network crate

* `unwrap_in_result` in zebra-rpc crate

* `unwrap_in_result` in zebrad crate

* rustfmt

* revert `?` and add exceptions

* explain some panics better

* move some lint positions

* replace a panic with error

* Fix rustfmt?

Co-authored-by: teor <teor@riseup.net>
2022-06-28 06:22:07 +00:00
teor 3825caae03
fix(clippy): Resolve some lifetime and reference lints (#4578)
* Fix significant drop in match scrutinee

https://github.com/rust-lang/rust/issues/93883

* Fix deref immutable value

* Fix explicit 0 index when first() would do

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-14 06:21:07 +00:00
Marek b8b35f8da9
fix(doc): Fix various doc warnings, part 1 (#4514)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

* Fix some warnings produced by `cargo doc`

* Fix some rustdoc warnings

* Fix some warnings

* Refactor some changes

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-06-02 15:07:35 +00:00
Marek 6f896ef5a5
fix(doc): Fix the syntax of links in comments (#4494)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-05-30 20:12:11 +00:00
teor be91ab29ee
fix(clippy): resolve various clippy warnings (#4473)
* clippy: unused import on non-linux platforms

* Fix some instances of clippy::derive_partial_eq_without_eq

* Move a deref to fix clippy::significant_drop_in_scrutinee

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-25 20:56:32 +00:00
teor 3291db35c0
fix(shielded): use RwLock for note commitment tree root caches (#3809)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-09 23:26:49 +00:00
Marek ece2d0f09b
Document the consensus rules for note commitment trees (#3319)
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-14 01:28:35 +00:00
Marek 8963007397
Sprout note commitment trees (#3051)
* Implement incremental note commitment Merkle tree for Sprout

* Add tests for Sprout note commitment tree

* Remove the `Arbitrary` attribute

* Reverse the vector of empty roots

* Add more tests

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustdoc

* Rustdoc

* rustdoc links

* Oops, need the trait in scope to use it

* Avoid accessing the wrapped hash directly

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustfmt

* Add typing

* Avoid accessing the wrapped hash directly

* Implement incremental note commitment Merkle tree for Sprout

* Add tests for Sprout note commitment tree

* Remove the `Arbitrary` attribute

* Reverse the vector of empty roots

* Add more tests

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustdoc

* Rustdoc

* rustdoc links

* Oops, need the trait in scope to use it

* Avoid accessing the wrapped hash directly

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustfmt

* Add typing

* Avoid accessing the wrapped hash directly

* Add Overwinter final roots (test vectors)

* Test sprout note commitments trees on Overwinter blocks

* Add new test vectors

* Finish the tests for the note commitment trees

* Make the wrapped hash in `Root` private

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2021-11-18 23:05:52 +00:00
Deirdre Connolly e719c46b1b
Track anchors and note commitment trees in zebra-state (#2458)
* Tidy chain Cargo.toml

* Organize imports

* Add method to get note commitments from all Actions in Orchard shielded data

* Add method to get note commitments from all JoinSplits in Sprout JoinSplitData

* Add Request and Response variants for awaiting anchors

* Add anchors and note commitment trees to finalized state db

* Add (From|Into)Disk impls for tree::Roots and stubs for NoteCommitmentTrees

* Track anchors and note commitment trees in Chain

Append note commitments to their trees when doing update_chain_state_with,
then use the resulting Sapling and Orchard roots to pass to history_tree, and add
new roots to the anchor sets.

* Handle errors when appending to note commitment trees

* Add comments explaining why note commitment are not removed from the tree in revert_chain_state_with

* Implementing note commitments in finalized state

* Finish serialization of Orchard tree; remove old tree when updating finalize state

* Add serialization and finalized state updates for Sprout and Sapling trees

* Partially handle trees in non-finalized state. Use Option for trees in Chain

* Rebuild trees when forking; change finalized state tree getters to not require height

* Pass empty trees to tests; use empty trees by default in Chain

* Also rebuild anchor sets when forking

* Use empty tree as default in finalized state tree getters (for now)

* Use HashMultiSet for anchors in order to make pop_root() work correctly

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS even more

* Apply suggestions from code review

* Add comments about order of note commitments and related methods/fields

* Don't use Option for trees

* Set DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES=1 and restore MAX_PARTIAL_CHAIN_BLOCKS

* Remove unneeded anchor set rebuilding in fork()

* Improve proptest formatting

* Add missing comparisons to eq_internal_state

* Renamed sprout::tree::NoteCommitmentTree::hash() to root()

* Improve comments

* Add asserts, add issues to TODOs

* Remove impl Default for Chain since it was only used by tests

* Improve documentation and assertions; add tree serialization tests

* Remove Sprout code, which will be moved to another branch

* Add todo! in Sprout tree append()

* Remove stub request, response *Anchor* handling for now

* Add test for validating Sapling note commitment tree using test blocks

* Increase database version (new columns added for note commitment trees and anchors)

* Update test to make sure the order of sapling_note_commitments() is being tested

* Improve comments and structure of the test

* Improve variable names again

* Rustfmt

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Conrado P. L. Gouvea <conradoplg@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-07-29 09:37:18 -04:00
teor f0549b2f7c
Derive Arbitrary impls for a bunch of chain and network types (#2179)
Enable proptests for internal and external network protocol messages,
using times with the correct protocol-specific ranges. (4 or 8 bytes.)
2021-05-24 11:10:07 -04:00
teor 873127aac1 Replace smart quotes with ascii quotes
Some tools don't deal well with unicode text. And we're not using it
consistently in Zebra anyway.
2021-03-15 03:18:10 -04:00
Alfredo Garcia 128643d81e
Call `zebra_test::init` where needed. (#1227)
* Add missing `zebra_test::init()` to zebra-chain
* Add missing `zebra_test::init()` to zebra-consensus
* Add missing `zebra_test::init()` to zebra-network
* Add missing `zebra_test::init()` to zebra-state
* Add missing `zebra_test::init()` to zebra-test
* Add missing `zebra_test::init()` to zebrad
2020-11-10 10:29:25 +10:00
Alfredo Garcia bcb027ebc5 change canopy.pdf to stable protocol.pdf 2020-10-28 11:34:53 -04:00
Deirdre Connolly f7095ae9a2 Add some more human-readable language to the doc comment on merkle_crh_sprout 2020-10-05 23:44:25 -04:00
Deirdre Connolly 0cf24be8e0 Use VecDeque in places
The unwrap()'s are all after checks to make sure the value should never be None.
2020-10-05 23:44:25 -04:00
Deirdre Connolly d6cac9ea0a Allow dead code for now 2020-10-05 23:44:25 -04:00
Deirdre Connolly ee8ac1fd38 Remove LittleEndian import 2020-10-05 23:44:25 -04:00
Deirdre Connolly b85fcdfce9 Hex-encode empty root test vectors to height 29 2020-10-05 23:44:25 -04:00
Deirdre Connolly 8ab0f92d45 Persist the byte-reversed commitments 2020-10-05 23:44:25 -04:00
Deirdre Connolly ebbb494fb3 It woooooorks 2020-10-05 23:44:25 -04:00
Deirdre Connolly 30491c2c55 Fix merkle_crh_sprout 2020-10-05 23:44:25 -04:00
Deirdre Connolly 064021b7bf Update test vectors 2020-10-05 23:44:25 -04:00
Deirdre Connolly 24fa507e41 Tidy tests 2020-10-05 23:44:25 -04:00
Deirdre Connolly 867576db21 Add a (theoretically) working Sprout note commitment tree impl and some tests 2020-10-05 23:44:25 -04:00
Deirdre Connolly 6732084928 Revert Sprout note commitment tree stuff for now 2020-10-05 23:43:23 -04:00
Deirdre Connolly 3d45209f05 Sprout NoteCommitmentTree root computation 2020-10-05 23:43:23 -04:00
Deirdre Connolly 568edb54e6 Add merkle_crh_sprout() and tidy note* things 2020-10-05 23:43:23 -04:00
Jane Lusby 352721bd88
Implement RFC5: State updates `Chain` type (#1069)
* Begin work on RFC5 implementation

* I think this is necessary

* holy shit supertrait implemented via subtrait

* implement most of the chain functions

* change to slightly better name

* implement fork

* fix outpoint handling in Chain struct

* update expect for work

* resolve review comment

* split utxo into two sets

* update the Chain definition

* just a little more

* update comment

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* apply changes from code review

* remove allow attribute in zebra-state/lib.rs

* Update zebra-state/src/memory_state.rs

Co-authored-by: teor <teor@riseup.net>

* merge ChainSet type into MemoryState

* rename state impl types

* Add error messages to asserts

* add module doc comment

* update RFC for utxos

* add missing header

Co-authored-by: teor <teor@riseup.net>
2020-09-24 15:46:04 -07:00
Jane Lusby 0b4e974c9e
export proptest impls for use in downstream crates (#1092)
* export proptest impls for use in downstream crates

* add testjob for disabled feature in zebra-chain

* run rustfmt

* try to fix github actions syntax

* differentiate name

* prove that github action tests zebra-chain build without features

* revert change from last commit now that test is running

* remove accidentally introduced newline

* Update .github/workflows/ci.yml

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2020-09-23 18:52:52 -07:00
Deirdre Connolly b467a75e08 Rename tree::*NoteTreeRootHash to tree::Root 2020-08-28 04:46:31 -04:00
Henry de Valence 312c66264a chain: extract sprout code into sprout module. 2020-08-17 11:46:34 -07:00