Commit Graph

138 Commits

Author SHA1 Message Date
Alfredo Garcia e2f010eb4f
build(deps): update ecc dependencies for `zcashd` 5.6.0, and create legacy state format compatibility layer (#7053)
* update ecc dependencies no serialization

* use zcash_primitives merke tree functions for serialization/deserialization in orchard and sapling trees

* some more work in sapling/orchard serialization/deserialization

* clippy

* fix doc links

* fix missing doc

* make orchard trees serializa/deserialize as they were before upgrade

* make sapling trees serialize/deserialize as they were before upgrade

* use legacy for sprout

* remove unused code

* readd snapshot tests

* repalce some code

* upgrade zcash_proofs

* remove legacy code for sprout

* fix the count method

* add root to serialize

* fixes in as_bytes and from_bytes

* use legacy code

* add todo about pow2 tests

* remove unused sprout code

* fix doc typos

* Add a recalculate_root() method to trees for tests

* Rename test tree types to make them easier to change

* Add TODOs for tests for old and new serialization formats

* fix doc typos

* add more test to note commitment trees

* fix comment

* fix leaf serializatiuon, reverse to old hashes in pow2 tests

* fix serialization

* put sapling SerializedTree code back

* put orchard SerializedTree code back

* clippy

* add duplicated dependencies until zebra_script updates

* fix a doc link

* minor cleanup

* remove todo comment from tests

* add one more check to tests

* update zebra_script

* update deny.toml

* replace custom function with library

* fix some tests

* update docs

* Remove duplicate dependencies from deny.toml

Based on the list at:
https://github.com/ZcashFoundation/zebra/actions/runs/5557139662/jobs/10150543673?pr=7053#step:5:14

---------

Co-authored-by: teor <teor@riseup.net>
2023-07-17 22:06:27 +00:00
teor f9a48266ad
change(test): Add a recalculate_root() method to trees for tests (#7204)
* Add a recalculate_root() method to trees for tests

* Rename test tree types to make them easier to change

* Add TODOs for tests for old and new serialization formats

* fix doc typos

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-07-12 17:07:10 +00:00
teor 9df78ffdba
change(tests): Do round-trip tests for note commitment tree data structure and RPC serialisation (#7147)
* Add an assert_frontier_eq() method to note commitment trees for tests

* Check round-trip serialization for note commitment trees

* fix typos

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-07-06 01:04:28 +00:00
teor 147b8fa3a8
cleanup(rust): Fix new nightly clippy warnings (#7135)
* Fix "comparison is always true" warning

* Add missing Send bound

* cargo clippy --fix --all-features --all-targets

* incorrect implementation of clone on a Copy type

* cargo fmt --all
2023-07-05 07:11:27 +00:00
teor 3d2c5ef290
fix(concurrency): Use Arc::into_inner() to avoid potential concurrency issues, needs Rust 1.70 (#7032)
* Use Arc::into_inner() to avoid potential concurrency issues

* Remove some outdated clippy lint workarounds (fixed in Rust 1.66)

* Update the required Rust version to 1.70
2023-06-21 20:44:53 +00:00
Pili Guerra ec2e9ca276
Delete outdated `TODOs` refering to closed issues (#6732)
* ZIPs were updated to remove ambiguity, this was tracked in #1267.

* #2105 was fixed by #3039 and #2379 was closed by #3069

* #2230 was a duplicate of #2231 which was closed by #2511

* #3235 was obsoleted by #2156 which was fixed by #3505

* #1850 was fixed by #2944, #1851 was fixed by #2961 and #2902 was fixed by #2969

* We migrated to Rust 2021 edition in Jan 2022 with #3332

* #1631 was closed as not needed

* #338 was fixed by #3040 and #1162 was fixed by #3067

* #2079 was fixed by #2445

* #4794 was fixed by #6122

* #1678 stopped being an issue

* #3151 was fixed by #3934

* #3204 was closed as not needed

* #1213 was fixed by #4586

* #1774 was closed as not needed

* #4633 was closed as not needed

* Clarify behaviour of difficulty spacing

Co-authored-by: teor <teor@riseup.net>

* Update comment to reflect implemented behaviour

Co-authored-by: teor <teor@riseup.net>

* Update comment to reflect implemented behaviour when retrying block downloads

Co-authored-by: teor <teor@riseup.net>

* Update `TODO` to remove closed issue and clarify when we might want to fix

Co-authored-by: teor <teor@riseup.net>

* Update `TODO` to remove closed issue and clarify what we might want to change in future

Co-authored-by: teor <teor@riseup.net>

* Clarify benefits of how we do block verification

Co-authored-by: teor <teor@riseup.net>

* Fix rustfmt errors

---------

Co-authored-by: teor <teor@riseup.net>
2023-05-23 03:33:14 +00:00
Alfredo Garcia 85534ab027
fix(errors): Handle randomness generation and invalid random values as errors in cryptographic code (#6385)
* add error handling

* change error name

* Error types oriented around the primary types we expose in the zebra-chain API

* Fix Ok spelling

* orchard::note::new(): return NoteError if randomness produces invalid Pallas point

---------

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2023-04-20 17:49:21 +00:00
teor 3ba334bf20
change(deps): Upgrade shared dependencies to match `zcashd` 5.5.0 (#6536)
* Cleanup module docs and imports for sapling::arbitrary

* Upgrade dependencies shared with zcashd 5.5.0

```sh
cargo upgrade --workspace << EOF
bellman
bls12_381
group
halo2
halo2_proofs
jubjub
orchard
redjubjub
zcash_address
zcash_note_encryption
zcash_primitives
zcash_proofs
EOF
```

* Replace orchard::from_bytes_wide() with ff::from_uniform_bytes()

* Revert "disable test (#6285)" to re-enable verify_generated_halo2_proofs

This reverts commit 4d4accc0b9.
2023-04-18 10:11:38 +00:00
dependabot[bot] 45a96b5adc
build(deps): bump serde from 1.0.156 to 1.0.158 (#6358)
* build(deps): bump serde from 1.0.156 to 1.0.158

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.156 to 1.0.158.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.156...v1.0.158)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make MERKLE_DEPTH into a u8 to avoid a spurious serde_derive warning

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2023-03-22 17:12:34 +00:00
dependabot[bot] d06b0b15a1
build(deps): bump bitflags from 1.3.2 to 2.0.1 (#6346)
* build(deps): bump bitflags from 1.3.2 to 2.0.1

Bumps [bitflags](https://github.com/bitflags/bitflags) from 1.3.2 to 2.0.1.
- [Release notes](https://github.com/bitflags/bitflags/releases)
- [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md)
- [Commits](https://github.com/bitflags/bitflags/compare/1.3.2...2.0.1)

---
updated-dependencies:
- dependency-name: bitflags
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* make bitflag v2 work with zebra

* add doc suggestions

Co-authored-by: Marek <mail@marek.onl>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-03-22 02:49:21 +00:00
Conrado Gouvea 26c86cc088
use `reddsa` crate and remove duplicated RedPallas code (#6013)
* use `reddsa` crate and remove duplicated RedPallas code

* update old references to 'redpallas' crate

* Use reddsa 0.4.0

* update Cargo.lock

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2023-02-01 23:27:28 +00:00
teor 256b1c0008
fix(clippy): Fix new lints in nightly clippy (#5959)
* Derive default using #[default]

* Implement PartialEq manually to satisfy clippy

* Allow a manual derive in test-only code

* Fix some missing docs warnings in the Docker build
2023-01-18 00:27:42 +00:00
teor 8c451968ee
fix(diagnostics): Hex-encode debug format of commitments, nonces, and nullifiers (#5960)
* Format commitments and nonces as hex

* Format Sprout and Sapling nullifiers as hex when debugging

* Format Sprout commitments as hex when debugging

* Format redpallas keys as hex when debugging

* Update code that we're going to delete in the next PR anyway
2023-01-17 13:57:22 +00:00
teor 161bb80ce8
cleanup(cryptography): Remove unused shielded key and address code (#5476)
* Remove unused and buggy Sprout key and address code

* Remove unused, buggy Sapling address, key, and commitment code

* Delete unused Orchard key code

* Move almost all the buggy Orchard key code into a test-only module

* Remove Orchard keys and addresses that aren't used in production code

* Remove unused prf_expand() function and unimplemented poseidon_hash() function

* Remove unused Orchard key types

* Remove unused sinsemilla commit code

* Update zebra-chain/src/sprout/keys.rs

* Update zebra-chain/src/sprout/keys.rs

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-10-29 20:59:55 +00:00
teor d1bcc3292e
Resolve a few doc warnings about unquoted HTML tags (#5477) 2022-10-25 12:14:21 +00:00
Alfredo Garcia 233220ace0
remove dead code from zebra-chain (#5464)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-10-25 03:23:50 +00:00
teor 7daffb572a
chore(clippy): Fix clippy cast and closure warnings (#5378)
* Fix clippy cast and closure warnings

* Silence incorrect lints in Clippy 1.64

* Use correct clippy syntax

Co-authored-by: Marek <mail@marek.onl>

* Disable an incorrect Clippy 1.64 lint

* Disable lint on the correct code

* Try another place

Co-authored-by: Marek <mail@marek.onl>
2022-10-14 08:53:40 +00:00
teor 66ea40f215
Resolve unnecessary borrow lint and remove redundant imports (#5045)
```sh
cargo clippy --fix --all-features --all-targets
```
2022-09-02 18:20:32 +00:00
Conrado Gouvea 6fd750e168
build(deps): bump insta from 1.15.0 to 1.17.1 (#4884)
* build(deps): bump insta from 1.15.0 to 1.17.1

Bumps [insta](https://github.com/mitsuhiko/insta) from 1.15.0 to 1.17.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.15.0...1.17.1)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* make zebra_test::init() return the insta drop guard

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-04 15:44:44 +00:00
Deirdre Connolly bdd2808dbb
Disallow Orchard ivk = 0 on IncomingViewingKey::from & SpendingKey generation (#3962)
* Disallow Orchard ivk = 0 on IncomingViewingKey::from and SpendingKey generation

* Check that ivk scalar values parsed from bytes are never 0

* Update zebra-chain/src/orchard/keys.rs

Co-authored-by: Daira Hopwood <daira@jacaranda.org>

* Switch away from removed pallas::Scalar::from_bytes to PrimeField::from_repr

* Fix updated Orchard IVK derivation around updated BitVec API

* Remove spurious proptest regressions

* Update zebra-chain/src/orchard/keys.rs

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* allow `unwrap_in_result` lint in added code

Co-authored-by: Daira Hopwood <daira@jacaranda.org>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-08-01 09:06:37 +00:00
teor 6ad445eb97
1. fix(perf): Run CPU-intensive state updates in parallel rayon threads (#4802)
* Split disk reads from CPU-heavy Sprout interstitial tree cryptography

* Improve anchor validation debugging and error messages

* Work around a test data bug, and save some CPU

* Remove redundant checks for empty shielded data

* Skip generating unused interstitial treestates

* Do disk fetches and quick checks, then CPU-heavy cryptography

* Wrap HistoryTree in an Arc in the state

* Run CPU-intensive chain validation and updates in parallel rayon threads

* Refactor to prepare for parallel tree root calculations

* Run finalized state note commitment tree root updates in parallel rayon threads

* Update finalized state note commitment trees using parallel rayon threads

* Fix a comment typo and add a TODO

* Split sprout treestate fetch into its own function

* Move parallel note commitment trees to zebra-chain

* Re-calculate the tree roots in the same parallel batches

* Do non-finalized note commitment tree updates in parallel threads

* Update comments about note commitment tree rebuilds

* Do post-fork tree updates in parallel threads

* Add a TODO for parallel tree updates in tests

* Fix broken intra-doc links

* Clarify documentation for sprout treestates

* Sort Cargo.toml dependencies
2022-07-22 12:19:11 -04:00
Alfredo Garcia 1b17c57bd5
perf(commitments): generate sapling point outside the method (#4799)
* move generated point to a lazy_static

* move lazy static out of method
2022-07-22 09:17:09 +10:00
Alfredo Garcia 97fb85dca9
lint(clippy): add `unwrap_in_result` lint (#4667)
* `unwrap_in_result` in zebra-chain crate

* `unwrap_in_result` in zebra-script crate

* `unwrap_in_result` in zebra-state crate

* `unwrap_in_result` in zebra-consensus crate

* `unwrap_in_result` in zebra-test crate

* `unwrap_in_result` in zebra-network crate

* `unwrap_in_result` in zebra-rpc crate

* `unwrap_in_result` in zebrad crate

* rustfmt

* revert `?` and add exceptions

* explain some panics better

* move some lint positions

* replace a panic with error

* Fix rustfmt?

Co-authored-by: teor <teor@riseup.net>
2022-06-28 06:22:07 +00:00
Alfredo Garcia b7536c7f7e
introduce `fallible_impl_from` lint with exeptions (#4609)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-20 03:57:41 +00:00
teor 3825caae03
fix(clippy): Resolve some lifetime and reference lints (#4578)
* Fix significant drop in match scrutinee

https://github.com/rust-lang/rust/issues/93883

* Fix deref immutable value

* Fix explicit 0 index when first() would do

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-14 06:21:07 +00:00
Marek b8b35f8da9
fix(doc): Fix various doc warnings, part 1 (#4514)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

* Fix some warnings produced by `cargo doc`

* Fix some rustdoc warnings

* Fix some warnings

* Refactor some changes

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-06-02 15:07:35 +00:00
Marek 6f896ef5a5
fix(doc): Fix the syntax of links in comments (#4494)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-05-30 20:12:11 +00:00
teor be91ab29ee
fix(clippy): resolve various clippy warnings (#4473)
* clippy: unused import on non-linux platforms

* Fix some instances of clippy::derive_partial_eq_without_eq

* Move a deref to fix clippy::significant_drop_in_scrutinee

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-25 20:56:32 +00:00
Conrado Gouvea 55822dda41
change: NU5 mainnet dependency upgrades (#4405)
* upgrade dependencies to NU5 mainnet versions

* migrate to bitvec 1.0

* update deny.toml

* point to published redjubjub 0.5

* Point at zcash-script@0.1.6 on crates.io

* update Cargo.lock

* specify patch versions of dependencies

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-05-19 01:03:55 +00:00
Marek 7c726b246d
feat(rpc): Implement `z_gettreestate` RPC (#3990)
* Impl the elementary structure of the `z_gettreestate` RPC

* Fix merging bugs

* Fix a merge bug

* Fix a merge bug

* Move a derive attribute

Co-authored-by: teor <teor@riseup.net>

* Clarify the support of negative heights

* Add Orchard note commitment trees to the response

* Add the time to the response

* Finalize the `z_gettreestate` RPC

* Add a note that verified blocks have coinbase height

* Refactor `from_str` for `HashOrHeight`

* Fix a mistake in the docs

Co-authored-by: teor <teor@riseup.net>

* Clarify request types

Co-authored-by: teor <teor@riseup.net>

* Simplify `hash_or_height` conversion to height

Co-authored-by: teor <teor@riseup.net>

* Add a TODO about optimization

Co-authored-by: teor <teor@riseup.net>

* Add a doc comment

* Make sure Sapling & Orchard trees don't get mixed up

* Serialize Sapling commitment trees

* Refactor some comments

* Serialize Orchard commitment trees

* Serialize block heights

* Simplify the serialization of commitment trees

* Remove the block time from the RPC response

* Simplify the serialization of block heights

* Put Sapling & Orchard requests together

* Remove a redundant TODO

* Add block times to the RPC response

* Derive `Clone, Debug, Eq, PartialEq` for `GetTreestate`

Co-authored-by: teor <teor@riseup.net>

* Derive `Clone`, `Debug`, `Eq` and `PartialEq` for `SerializedTree`

* Document the fields of `GetTreestate`

* Skip the serialization of empty trees

This ensures compatibility with `zcashd` in the `z_gettreestate` RPC.

* Document the `impl` of `merkle_tree::Hashable` for nodes

* Make the structure of the JSON response consistent with `zcashd`

* Derive `Eq` for nodes

Co-authored-by: teor <teor@riseup.net>

* Convert Sapling commitment trees to a format compatible with zcashd

* Refactor the conversion of Sapling commitment trees

* Refactor some comments

* Refactor comments

* Add a description of the conversion

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Fix comment indenting

* Document the conversion between the dense and sparse formats

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2022-05-12 07:00:12 +00:00
Conrado Gouvea d79e71e969
change(nu5): use new V5 transaction script verification API (#3799)
* update librustzcash; adapt to new API

* add ticket reference for removing zcash_proofs duplicated dependencies

* update to new zcash_script V5 API

* use zp_tx shorthand

* update to Zcash 4.7.0 dependencies

* update protocol versions

* feat(rpc): Implement `getblockchaininfo` RPC method (#3891)

* Implement `getblockchaininfo` RPC method

* add a test for `get_blockchain_info`

* fix tohex/fromhex

* move comment

* Update lightwalletd acceptance test for getblockchaininfo RPC (#3914)

* change(rpc): Return getblockchaininfo network upgrades in height order (#3915)

* Update lightwalletd acceptance test for getblockchaininfo RPC

* Update some doc comments for network upgrades

* List network upgrades in order in the getblockchaininfo RPC

Also:
- Use a constant for the "missing consensus branch ID" RPC value
- Simplify fetching consensus branch IDs
- Make RPC type derives consistent
- Update RPC type documentation

* Make RPC type derives consistent

* Fix a confusing test comment

* get hashand height at the same time

* fix estimated_height

* fix lint

* add extra check

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix typo

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* split test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix(rpc): ignore an expected error in the RPC acceptance tests (#3961)

* Add ignored regexes to test command failure regex methods

* Ignore empty chain error in getblockchaininfo

We expect this error when zebrad starts up with an empty state.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Make sync error logs more user-friendly (#3944)

- use info level, there is nothing the user needs to do,
  particularly for a single error
- explain that the errors are temporary
- hide backtraces, because they look like crashes

* Update test.patch.yml with lightwalletd job (#3970)

* Update test.patch.yml with lightwalletd job

* Remove a workflow condition that will always be false

In general, patch workflows need the
opposite conditions to the original workflow.

But in this case, we know the result of the
condition will always be true, so we can just delete it.

Co-authored-by: teor <teor@riseup.net>

* fix(doc): Fix bugs in the lightwalletd database design (#3964)

* Re-order column families in design in dependency order

* Minor RFC design tweaks and fixes

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Repoint zebra image links to our new zfnd.org site for now (#3949)

* Repoint zebra image links to our new zfnd.org site for now

* Remove images/

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Fix typos (#3956)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* bump database version to trigger testnet rollback

* reduce minimum protocol version for now (will be changed later)

* update dependencies

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* update versions to match zcash 4.7.0

* deny.toml: update 'darling'

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Dimitris Apostolou <dimitris.apostolou@icloud.com>
2022-04-19 10:14:16 +10:00
Conrado Gouvea ee6a38dffc
build(deps): bump serde-big-array from 0.3.2 to 0.4.1 (#4004)
Bumps [serde-big-array](https://github.com/est31/serde-big-array) from 0.3.2 to 0.4.1.
- [Release notes](https://github.com/est31/serde-big-array/releases)
- [Commits](https://github.com/est31/serde-big-array/compare/v0.3.2...v0.4.1)

---
updated-dependencies:
- dependency-name: serde-big-array
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-31 01:33:52 +00:00
teor 3291db35c0
fix(shielded): use RwLock for note commitment tree root caches (#3809)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-09 23:26:49 +00:00
Conrado Gouvea b7b62af2f2
Document consensus rules from 4.6 Action Descriptions (#3549)
* docs: document consensus rules from 4.6 Action Descriptions

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Marek <mail@marek.onl>
2022-02-21 23:49:32 +00:00
Alfredo Garcia 9ede0fb52a
document transaction orchard types (#3507) 2022-02-11 03:16:35 +00:00
Alfredo Garcia 29ad801a35
docs: Transaction consensus rules: Size rules (#3461)
* refactor transaction size consensus rules

* quote mssing consensus rule

* nit

Co-authored-by: teor <teor@riseup.net>

* move consensus rule doc

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-08 20:28:40 +00:00
Marek ece2d0f09b
Document the consensus rules for note commitment trees (#3319)
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-14 01:28:35 +00:00
Deirdre Connolly 9734ea1014
Check jubjub key correctness independent of redjubjub / jubjub (#3154)
* Ensure that sapling::keys::TransmissionKey jubjub point is always in the prime order group

* Adjust TransmissionKey check; add AuthorizingKey check

* Move ValueCommitment small order check to deserialization

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Use is_torsion_free() instead of is_identity() and is_prime_order()

* Add EphemeralPublicKey small order check on instantiation; remove old checks

* Use VerificationKey instead of VerificationKeyBytes; fix tests

* Use ValidatingKey instead of VerificationKeyBytes for rk

* Reject identity when creating an Orchard EphemeralPublicKey

* Make documentation more consistent, use generator in tests

* s/JubJub/Jubjub/

* Fix zebra-consensus tests (insert_fake_orchard_shielded_data)

* Create NotSmallOrderValueCommitment, since intermediate values can be the identity

* Clarify documentation

* rustdoc

* Tidy rustdoc

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
2021-12-22 15:10:34 +00:00
Conrado Gouvea 88b09c812a
Check nSpendsSapling, nOutputsSapling, and nActionsOrchard 2^16 limit (#3069)
* Check nSpendsSapling, nOutputsSapling, and nActionsOrchard 2^16 limit

* Apply suggestions from code review

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Removed not required #[macro_use]

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-18 15:06:07 -03:00
Deirdre Connolly eda83ebe0e
Async Halo2 verifier service (#2645)
* First pass at async Halo2 verification service

Stubs out a batch verifier for the future.
The dependencies for orchard, halo2, librustzcash, zcash_primitives, have
not been resolved.

* Halo2 verifier service and test

* Remove redundant conversion

* Test async halo2 verifier service with pre-computed Orchard shielded data test vectors

* Fix typo

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Assert future result is_ok() in Halo2 verifier test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Shorten tower::Service trait constraints for Halo2 verifier tests

* Remove commented out trait constraints

* .expect() vs .unwrap() to parse orchard::redpallas::VerificationKey

* Use .to_vec() for some test vectors

* Fix self-referential Display impl

* Fix deps

* Distinguish orchard vs zebra_chain::orchard imports

* Add test that halo2 verifier fails with malformed proof inputs

* Use thiserror for Halo2Error

* Use ZcashFoundation/orchard instead of dconnolly/orchard

* Add a link to the issue to remove the zfnd fork of orchard crate

* Update zebra-consensus/Cargo.toml

Co-authored-by: teor <teor@riseup.net>

* Add note

* Move artificial Orchard shielded data test vectors to zebra-test

* Align brackets

* Tidy some trait constraints and debug statements

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2021-11-17 13:26:15 +10:00
Dimitris Apostolou afb8b3d477
Fix typos (#3055)
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-12 19:30:22 +00:00
teor c0c00b3f0d
Simplify preallocate tests (#3032)
* Simplify preallocation tests using a test function

* Use prop_assert in proptests
2021-11-11 07:53:21 +10:00
Janito Vaqueiro Ferreira Filho 9e78a8af40
Refactor mempool spend conflict checks to increase performance (#2826)
* Add `HashSet`s to help spend conflict detection

Keep track of the spent transparent outpoints and the revealed
nullifiers.

Clippy complained that the `ActiveState` had variants with large size
differences, but that was expected, so I disabled that lint on that
`enum`.

* Clear the `HashSet`s when clearing the mempool

Clear them so that they remain consistent with the set of verified
transactions.

* Use `HashSet`s to check for spend conflicts

Store new outputs into its respective `HashSet`, and abort if a
duplicate output is found.

* Remove inserted outputs when aborting

Restore the `HashSet` to its previous state.

* Remove tracked outputs when removing a transaction

Keep the mempool storage in a consistent state when a transaction is
removed.

* Remove tracked outputs when evicting from mempool

Ensure eviction also keeps the tracked outputs consistent with the
verified transactions.

* Refactor to create a `VerifiedSet` helper type

Move the code to handle the output caches into the new type. Also move
the eviction code to make things a little simpler.

* Refactor to have a single `remove` method

Centralize the code that handles the removal of a transaction to avoid
mistakes.

* Move mempool size limiting back to `Storage`

Because the evicted transactions must be added to the rejected list.

* Remove leftover `dbg!` statement

Leftover from some temporary testing code.

Co-authored-by: teor <teor@riseup.net>

* Remove unnecessary `TODO`

It is more speculation than planning, so it doesn't add much value.

Co-authored-by: teor <teor@riseup.net>

* Fix typo in documentation

The verb should match the subject "transactions" which is plural.

Co-authored-by: teor <teor@riseup.net>

* Add a comment to warn about correctness

There's a subtle but important detail in the implementation that should
be made more visible to avoid mistakes in the future.

Co-authored-by: teor <teor@riseup.net>

* Remove outdated comment

Left-over from the attempt to move the eviction into the `VerifiedSet`.

* Improve comment explaining lint removal

Rewrite the comment explaining why the Clippy lint was ignored.

* Check for spend conflicts in `VerifiedSet`

Refactor to avoid API misuse.

* Test rejected transaction rollback

Using two transactions, perform the same test adding a conflict to both
of them to check if the second inserted transaction is properly
rejected. Then remove any conflicts from the second transaction and add
it again. That should work, because if it doesn't it means that when the
second transaction was rejected it left things it shouldn't in the
cache.

* Test removal of multiple transactions

When removing multiple transactions from the mempool storage, all of the
ones requested should be removed and any other transaction should be
still be there afterwards.

* Increase mempool size to 4, so that spend conflict tests work

If the mempool size is smaller than 4,
these tests don't fail on a trivial removal bug.
Because we need a minimum number of transactions in the mempool
to trigger the bug.

Also commit a proptest seed that fails on a trivial removal bug.
(This seed fails if we remove indexes in order,
because every index past the first removes the wrong transaction.)

* Summarise transaction data in proptest error output

* Summarise spend conflict field data in proptest error output

* Summarise multiple removal field data in proptest error output

And replace the very large proptest debug output with the new summary.

Co-authored-by: teor <teor@riseup.net>
2021-10-10 23:54:46 +00:00
Janito Vaqueiro Ferreira Filho 339fefb6e6
Update shared NU5 dependencies, set the NU5 testnet activation network upgrade parameters (#2825)
* Check return value of zcash_script_new_precomputed_tx

* Set the NU5 testnet activation height to 1_590_000

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update Nu5 constants to new values

* Update ZIP-244 test vectors for new branch ID

* Squashed commit of the following:

commit bdb120a249e3e889a913114a712505defdade1d4
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:54:01 2021 -0400

    Use pallas::Base::from_str_vartime() in sinsemilla tests

commit e99fa4925857840fa65ccfb4a076ec412e416576
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:45:24 2021 -0400

    Compiles

commit a5200181146bfd2aa1e09abea2caaa7a7ceb006e
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 10:15:17 2021 -0400

    Incomplete upgrade of deps

* Squashed commit of the following:

commit 8d1b76ec5626517817c3a4d9f3950acc90a359df
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 04:02:26 2021 +0000

    Update `zcash_script` to support V5 transactions

    Use a newer version of `zcash_script` that has been updated to support
    V5 transactions.

commit 371233628ae61e0c25d6ba8f31d9dba42823becb
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:06:20 2021 +0000

    Update Zcash dependencies

    Update some Zcash crates:

    - `halo2`
    - `incrementalmerkletree' (patch version)
    - `orchard` (patch version)
    - `zcash_history` (patch version)
    - `zcash_note_encryption` (patch version)
    - `zcash_primitives` (patch version)

    And also update the `group` dependency so that the code remains
    compatible.

commit de5cf1ec40c3fc08670fc971cdf3e65e13d9f4c7
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:04:13 2021 +0000

    Update error message assertion

    Use the updated message for the expected error variant.

* Update `zcash_script` to support V5 transactions

Use a newer version of `zcash_script` that has been updated to support
V5 transactions.

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-10-06 11:08:41 +10:00
teor e5f5ac9ce8
Fix or disable recent nightly clippy lints (#2817)
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-10-01 15:26:06 +00:00
Deirdre Connolly 679b4768df
Use complete addition instead of incomplete addition inside sinsemilla_commit (#2807) 2021-09-29 13:00:25 -04:00
Alfredo Garcia c3c302309a
Add orchard binding_verification_key (#2441)
* add orchard binding_verification_key

* Merge branch 'main' into issue2102

* Merge branch 'main' into issue2102

* fix the build

* Merge branch 'main' into issue2102

* Merge branch 'main' into issue2102

* Fix docs

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* Merge branch 'main' into issue2102

* rustfmt

* readd binding validation

* remove #2103 from the TODO list

* Merge branch 'main' into issue2102
2021-08-16 15:16:25 -04:00
Conrado Gouvea eac95bdf10
Cache note commitment tree roots (#2584)
* Cache note commitment tree roots

* Add comments to cached root fields

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2021-08-10 10:33:34 -03:00
teor f09f2a9022
Check remaining transaction value & make value balance signs match the spec (#2566)
* Make Amount arithmetic more generic

To modify generated amounts, we need some extra operations on `Amount`.

We also need to extend existing operations to both `NonNegative` and
`NegativeAllowed` amounts.

* Add a constrain method for ValueBalance

* Derive Eq for ValueBalance

* impl Neg for ValueBalance

* Make some Amount arithmetic expectations explicit

* Explain why we use i128 for multiplication

And expand the overflow error details.

* Expand Amount::sum error details

* Make amount::Error field order consistent

* Rename an amount::Error variant to Constraint, so it's clearer

* Add specific pool variants to ValueBalanceError

* Update coinbase remaining value consensus rule comment

This consensus rule was updated recently to include coinbase transactions,
but Zebra doesn't check block subsidy or miner fees yet.

* Add test methods for modifying transparent values and shielded value balances

* Temporarily set values and value balances to zero in proptests

In both generated chains and proptests that construct their own transactions.

Using zero values reduces value calculation and value check test coverage.
A future change will use non-zero values, and fix them so the check passes.

* Add extra fields to remaining transaction value errors

* Swap the transparent value balance sign to match shielded value balances

This makes the signs of all the chain value pools consistent.

* Use a NonNegative constraint for transparent values

This fix:
* makes the type signature match the consensus rules
* avoids having to write code to handle negative values

* Allocate total generated transaction input value to outputs

If there isn't enough input value for an output, set it to zero.

Temporarily reduce all generated values to avoid overflow.
(We'll remove this workaround when we calculate chain value balances.)

* Consistently use ValueBalanceError for ValueBalances

* Make the value balance signs match the spec

And rename and document methods so their signs are clearer.

* Convert amount::Errors to specific pool ValueBalanceErrors

* Move some error changes to the next PR

* Add extra info to remaining transaction value errors (#2585)

* Distinguish between overflow and negative remaining transaction value errors

And make some error types cloneable.

* Add methods for updating chain value pools (#2586)

* Move amount::test to amount::tests:vectors

* Make ValueBalance traits more consistent with Amount

- implement Add and Sub variants with Result and Assign
- derive Hash

* Clarify some comments and expects

* Create ValueBalance update methods for blocks and transactions

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-09 14:22:26 -03:00
Deirdre Connolly e719c46b1b
Track anchors and note commitment trees in zebra-state (#2458)
* Tidy chain Cargo.toml

* Organize imports

* Add method to get note commitments from all Actions in Orchard shielded data

* Add method to get note commitments from all JoinSplits in Sprout JoinSplitData

* Add Request and Response variants for awaiting anchors

* Add anchors and note commitment trees to finalized state db

* Add (From|Into)Disk impls for tree::Roots and stubs for NoteCommitmentTrees

* Track anchors and note commitment trees in Chain

Append note commitments to their trees when doing update_chain_state_with,
then use the resulting Sapling and Orchard roots to pass to history_tree, and add
new roots to the anchor sets.

* Handle errors when appending to note commitment trees

* Add comments explaining why note commitment are not removed from the tree in revert_chain_state_with

* Implementing note commitments in finalized state

* Finish serialization of Orchard tree; remove old tree when updating finalize state

* Add serialization and finalized state updates for Sprout and Sapling trees

* Partially handle trees in non-finalized state. Use Option for trees in Chain

* Rebuild trees when forking; change finalized state tree getters to not require height

* Pass empty trees to tests; use empty trees by default in Chain

* Also rebuild anchor sets when forking

* Use empty tree as default in finalized state tree getters (for now)

* Use HashMultiSet for anchors in order to make pop_root() work correctly

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS even more

* Apply suggestions from code review

* Add comments about order of note commitments and related methods/fields

* Don't use Option for trees

* Set DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES=1 and restore MAX_PARTIAL_CHAIN_BLOCKS

* Remove unneeded anchor set rebuilding in fork()

* Improve proptest formatting

* Add missing comparisons to eq_internal_state

* Renamed sprout::tree::NoteCommitmentTree::hash() to root()

* Improve comments

* Add asserts, add issues to TODOs

* Remove impl Default for Chain since it was only used by tests

* Improve documentation and assertions; add tree serialization tests

* Remove Sprout code, which will be moved to another branch

* Add todo! in Sprout tree append()

* Remove stub request, response *Anchor* handling for now

* Add test for validating Sapling note commitment tree using test blocks

* Increase database version (new columns added for note commitment trees and anchors)

* Update test to make sure the order of sapling_note_commitments() is being tested

* Improve comments and structure of the test

* Improve variable names again

* Rustfmt

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Conrado P. L. Gouvea <conradoplg@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-07-29 09:37:18 -04:00