Commit Graph

120 Commits

Author SHA1 Message Date
Conrado Gouvea 6fd750e168
build(deps): bump insta from 1.15.0 to 1.17.1 (#4884)
* build(deps): bump insta from 1.15.0 to 1.17.1

Bumps [insta](https://github.com/mitsuhiko/insta) from 1.15.0 to 1.17.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.15.0...1.17.1)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* make zebra_test::init() return the insta drop guard

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-04 15:44:44 +00:00
Deirdre Connolly bdd2808dbb
Disallow Orchard ivk = 0 on IncomingViewingKey::from & SpendingKey generation (#3962)
* Disallow Orchard ivk = 0 on IncomingViewingKey::from and SpendingKey generation

* Check that ivk scalar values parsed from bytes are never 0

* Update zebra-chain/src/orchard/keys.rs

Co-authored-by: Daira Hopwood <daira@jacaranda.org>

* Switch away from removed pallas::Scalar::from_bytes to PrimeField::from_repr

* Fix updated Orchard IVK derivation around updated BitVec API

* Remove spurious proptest regressions

* Update zebra-chain/src/orchard/keys.rs

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* allow `unwrap_in_result` lint in added code

Co-authored-by: Daira Hopwood <daira@jacaranda.org>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-08-01 09:06:37 +00:00
teor 6ad445eb97
1. fix(perf): Run CPU-intensive state updates in parallel rayon threads (#4802)
* Split disk reads from CPU-heavy Sprout interstitial tree cryptography

* Improve anchor validation debugging and error messages

* Work around a test data bug, and save some CPU

* Remove redundant checks for empty shielded data

* Skip generating unused interstitial treestates

* Do disk fetches and quick checks, then CPU-heavy cryptography

* Wrap HistoryTree in an Arc in the state

* Run CPU-intensive chain validation and updates in parallel rayon threads

* Refactor to prepare for parallel tree root calculations

* Run finalized state note commitment tree root updates in parallel rayon threads

* Update finalized state note commitment trees using parallel rayon threads

* Fix a comment typo and add a TODO

* Split sprout treestate fetch into its own function

* Move parallel note commitment trees to zebra-chain

* Re-calculate the tree roots in the same parallel batches

* Do non-finalized note commitment tree updates in parallel threads

* Update comments about note commitment tree rebuilds

* Do post-fork tree updates in parallel threads

* Add a TODO for parallel tree updates in tests

* Fix broken intra-doc links

* Clarify documentation for sprout treestates

* Sort Cargo.toml dependencies
2022-07-22 12:19:11 -04:00
Alfredo Garcia 1b17c57bd5
perf(commitments): generate sapling point outside the method (#4799)
* move generated point to a lazy_static

* move lazy static out of method
2022-07-22 09:17:09 +10:00
Alfredo Garcia 97fb85dca9
lint(clippy): add `unwrap_in_result` lint (#4667)
* `unwrap_in_result` in zebra-chain crate

* `unwrap_in_result` in zebra-script crate

* `unwrap_in_result` in zebra-state crate

* `unwrap_in_result` in zebra-consensus crate

* `unwrap_in_result` in zebra-test crate

* `unwrap_in_result` in zebra-network crate

* `unwrap_in_result` in zebra-rpc crate

* `unwrap_in_result` in zebrad crate

* rustfmt

* revert `?` and add exceptions

* explain some panics better

* move some lint positions

* replace a panic with error

* Fix rustfmt?

Co-authored-by: teor <teor@riseup.net>
2022-06-28 06:22:07 +00:00
Alfredo Garcia b7536c7f7e
introduce `fallible_impl_from` lint with exeptions (#4609)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-20 03:57:41 +00:00
teor 3825caae03
fix(clippy): Resolve some lifetime and reference lints (#4578)
* Fix significant drop in match scrutinee

https://github.com/rust-lang/rust/issues/93883

* Fix deref immutable value

* Fix explicit 0 index when first() would do

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-14 06:21:07 +00:00
Marek b8b35f8da9
fix(doc): Fix various doc warnings, part 1 (#4514)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

* Fix some warnings produced by `cargo doc`

* Fix some rustdoc warnings

* Fix some warnings

* Refactor some changes

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-06-02 15:07:35 +00:00
Marek 6f896ef5a5
fix(doc): Fix the syntax of links in comments (#4494)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-05-30 20:12:11 +00:00
teor be91ab29ee
fix(clippy): resolve various clippy warnings (#4473)
* clippy: unused import on non-linux platforms

* Fix some instances of clippy::derive_partial_eq_without_eq

* Move a deref to fix clippy::significant_drop_in_scrutinee

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-25 20:56:32 +00:00
Conrado Gouvea 55822dda41
change: NU5 mainnet dependency upgrades (#4405)
* upgrade dependencies to NU5 mainnet versions

* migrate to bitvec 1.0

* update deny.toml

* point to published redjubjub 0.5

* Point at zcash-script@0.1.6 on crates.io

* update Cargo.lock

* specify patch versions of dependencies

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-05-19 01:03:55 +00:00
Marek 7c726b246d
feat(rpc): Implement `z_gettreestate` RPC (#3990)
* Impl the elementary structure of the `z_gettreestate` RPC

* Fix merging bugs

* Fix a merge bug

* Fix a merge bug

* Move a derive attribute

Co-authored-by: teor <teor@riseup.net>

* Clarify the support of negative heights

* Add Orchard note commitment trees to the response

* Add the time to the response

* Finalize the `z_gettreestate` RPC

* Add a note that verified blocks have coinbase height

* Refactor `from_str` for `HashOrHeight`

* Fix a mistake in the docs

Co-authored-by: teor <teor@riseup.net>

* Clarify request types

Co-authored-by: teor <teor@riseup.net>

* Simplify `hash_or_height` conversion to height

Co-authored-by: teor <teor@riseup.net>

* Add a TODO about optimization

Co-authored-by: teor <teor@riseup.net>

* Add a doc comment

* Make sure Sapling & Orchard trees don't get mixed up

* Serialize Sapling commitment trees

* Refactor some comments

* Serialize Orchard commitment trees

* Serialize block heights

* Simplify the serialization of commitment trees

* Remove the block time from the RPC response

* Simplify the serialization of block heights

* Put Sapling & Orchard requests together

* Remove a redundant TODO

* Add block times to the RPC response

* Derive `Clone, Debug, Eq, PartialEq` for `GetTreestate`

Co-authored-by: teor <teor@riseup.net>

* Derive `Clone`, `Debug`, `Eq` and `PartialEq` for `SerializedTree`

* Document the fields of `GetTreestate`

* Skip the serialization of empty trees

This ensures compatibility with `zcashd` in the `z_gettreestate` RPC.

* Document the `impl` of `merkle_tree::Hashable` for nodes

* Make the structure of the JSON response consistent with `zcashd`

* Derive `Eq` for nodes

Co-authored-by: teor <teor@riseup.net>

* Convert Sapling commitment trees to a format compatible with zcashd

* Refactor the conversion of Sapling commitment trees

* Refactor some comments

* Refactor comments

* Add a description of the conversion

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Fix comment indenting

* Document the conversion between the dense and sparse formats

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2022-05-12 07:00:12 +00:00
Conrado Gouvea d79e71e969
change(nu5): use new V5 transaction script verification API (#3799)
* update librustzcash; adapt to new API

* add ticket reference for removing zcash_proofs duplicated dependencies

* update to new zcash_script V5 API

* use zp_tx shorthand

* update to Zcash 4.7.0 dependencies

* update protocol versions

* feat(rpc): Implement `getblockchaininfo` RPC method (#3891)

* Implement `getblockchaininfo` RPC method

* add a test for `get_blockchain_info`

* fix tohex/fromhex

* move comment

* Update lightwalletd acceptance test for getblockchaininfo RPC (#3914)

* change(rpc): Return getblockchaininfo network upgrades in height order (#3915)

* Update lightwalletd acceptance test for getblockchaininfo RPC

* Update some doc comments for network upgrades

* List network upgrades in order in the getblockchaininfo RPC

Also:
- Use a constant for the "missing consensus branch ID" RPC value
- Simplify fetching consensus branch IDs
- Make RPC type derives consistent
- Update RPC type documentation

* Make RPC type derives consistent

* Fix a confusing test comment

* get hashand height at the same time

* fix estimated_height

* fix lint

* add extra check

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix typo

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* split test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix(rpc): ignore an expected error in the RPC acceptance tests (#3961)

* Add ignored regexes to test command failure regex methods

* Ignore empty chain error in getblockchaininfo

We expect this error when zebrad starts up with an empty state.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Make sync error logs more user-friendly (#3944)

- use info level, there is nothing the user needs to do,
  particularly for a single error
- explain that the errors are temporary
- hide backtraces, because they look like crashes

* Update test.patch.yml with lightwalletd job (#3970)

* Update test.patch.yml with lightwalletd job

* Remove a workflow condition that will always be false

In general, patch workflows need the
opposite conditions to the original workflow.

But in this case, we know the result of the
condition will always be true, so we can just delete it.

Co-authored-by: teor <teor@riseup.net>

* fix(doc): Fix bugs in the lightwalletd database design (#3964)

* Re-order column families in design in dependency order

* Minor RFC design tweaks and fixes

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Repoint zebra image links to our new zfnd.org site for now (#3949)

* Repoint zebra image links to our new zfnd.org site for now

* Remove images/

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Fix typos (#3956)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* bump database version to trigger testnet rollback

* reduce minimum protocol version for now (will be changed later)

* update dependencies

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* update versions to match zcash 4.7.0

* deny.toml: update 'darling'

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Dimitris Apostolou <dimitris.apostolou@icloud.com>
2022-04-19 10:14:16 +10:00
Conrado Gouvea ee6a38dffc
build(deps): bump serde-big-array from 0.3.2 to 0.4.1 (#4004)
Bumps [serde-big-array](https://github.com/est31/serde-big-array) from 0.3.2 to 0.4.1.
- [Release notes](https://github.com/est31/serde-big-array/releases)
- [Commits](https://github.com/est31/serde-big-array/compare/v0.3.2...v0.4.1)

---
updated-dependencies:
- dependency-name: serde-big-array
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-31 01:33:52 +00:00
teor 3291db35c0
fix(shielded): use RwLock for note commitment tree root caches (#3809)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-09 23:26:49 +00:00
Conrado Gouvea b7b62af2f2
Document consensus rules from 4.6 Action Descriptions (#3549)
* docs: document consensus rules from 4.6 Action Descriptions

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Marek <mail@marek.onl>
2022-02-21 23:49:32 +00:00
Alfredo Garcia 9ede0fb52a
document transaction orchard types (#3507) 2022-02-11 03:16:35 +00:00
Alfredo Garcia 29ad801a35
docs: Transaction consensus rules: Size rules (#3461)
* refactor transaction size consensus rules

* quote mssing consensus rule

* nit

Co-authored-by: teor <teor@riseup.net>

* move consensus rule doc

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-08 20:28:40 +00:00
Marek ece2d0f09b
Document the consensus rules for note commitment trees (#3319)
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-14 01:28:35 +00:00
Deirdre Connolly 9734ea1014
Check jubjub key correctness independent of redjubjub / jubjub (#3154)
* Ensure that sapling::keys::TransmissionKey jubjub point is always in the prime order group

* Adjust TransmissionKey check; add AuthorizingKey check

* Move ValueCommitment small order check to deserialization

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Use is_torsion_free() instead of is_identity() and is_prime_order()

* Add EphemeralPublicKey small order check on instantiation; remove old checks

* Use VerificationKey instead of VerificationKeyBytes; fix tests

* Use ValidatingKey instead of VerificationKeyBytes for rk

* Reject identity when creating an Orchard EphemeralPublicKey

* Make documentation more consistent, use generator in tests

* s/JubJub/Jubjub/

* Fix zebra-consensus tests (insert_fake_orchard_shielded_data)

* Create NotSmallOrderValueCommitment, since intermediate values can be the identity

* Clarify documentation

* rustdoc

* Tidy rustdoc

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
2021-12-22 15:10:34 +00:00
Conrado Gouvea 88b09c812a
Check nSpendsSapling, nOutputsSapling, and nActionsOrchard 2^16 limit (#3069)
* Check nSpendsSapling, nOutputsSapling, and nActionsOrchard 2^16 limit

* Apply suggestions from code review

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Removed not required #[macro_use]

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-18 15:06:07 -03:00
Deirdre Connolly eda83ebe0e
Async Halo2 verifier service (#2645)
* First pass at async Halo2 verification service

Stubs out a batch verifier for the future.
The dependencies for orchard, halo2, librustzcash, zcash_primitives, have
not been resolved.

* Halo2 verifier service and test

* Remove redundant conversion

* Test async halo2 verifier service with pre-computed Orchard shielded data test vectors

* Fix typo

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Assert future result is_ok() in Halo2 verifier test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Shorten tower::Service trait constraints for Halo2 verifier tests

* Remove commented out trait constraints

* .expect() vs .unwrap() to parse orchard::redpallas::VerificationKey

* Use .to_vec() for some test vectors

* Fix self-referential Display impl

* Fix deps

* Distinguish orchard vs zebra_chain::orchard imports

* Add test that halo2 verifier fails with malformed proof inputs

* Use thiserror for Halo2Error

* Use ZcashFoundation/orchard instead of dconnolly/orchard

* Add a link to the issue to remove the zfnd fork of orchard crate

* Update zebra-consensus/Cargo.toml

Co-authored-by: teor <teor@riseup.net>

* Add note

* Move artificial Orchard shielded data test vectors to zebra-test

* Align brackets

* Tidy some trait constraints and debug statements

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2021-11-17 13:26:15 +10:00
Dimitris Apostolou afb8b3d477
Fix typos (#3055)
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-12 19:30:22 +00:00
teor c0c00b3f0d
Simplify preallocate tests (#3032)
* Simplify preallocation tests using a test function

* Use prop_assert in proptests
2021-11-11 07:53:21 +10:00
Janito Vaqueiro Ferreira Filho 9e78a8af40
Refactor mempool spend conflict checks to increase performance (#2826)
* Add `HashSet`s to help spend conflict detection

Keep track of the spent transparent outpoints and the revealed
nullifiers.

Clippy complained that the `ActiveState` had variants with large size
differences, but that was expected, so I disabled that lint on that
`enum`.

* Clear the `HashSet`s when clearing the mempool

Clear them so that they remain consistent with the set of verified
transactions.

* Use `HashSet`s to check for spend conflicts

Store new outputs into its respective `HashSet`, and abort if a
duplicate output is found.

* Remove inserted outputs when aborting

Restore the `HashSet` to its previous state.

* Remove tracked outputs when removing a transaction

Keep the mempool storage in a consistent state when a transaction is
removed.

* Remove tracked outputs when evicting from mempool

Ensure eviction also keeps the tracked outputs consistent with the
verified transactions.

* Refactor to create a `VerifiedSet` helper type

Move the code to handle the output caches into the new type. Also move
the eviction code to make things a little simpler.

* Refactor to have a single `remove` method

Centralize the code that handles the removal of a transaction to avoid
mistakes.

* Move mempool size limiting back to `Storage`

Because the evicted transactions must be added to the rejected list.

* Remove leftover `dbg!` statement

Leftover from some temporary testing code.

Co-authored-by: teor <teor@riseup.net>

* Remove unnecessary `TODO`

It is more speculation than planning, so it doesn't add much value.

Co-authored-by: teor <teor@riseup.net>

* Fix typo in documentation

The verb should match the subject "transactions" which is plural.

Co-authored-by: teor <teor@riseup.net>

* Add a comment to warn about correctness

There's a subtle but important detail in the implementation that should
be made more visible to avoid mistakes in the future.

Co-authored-by: teor <teor@riseup.net>

* Remove outdated comment

Left-over from the attempt to move the eviction into the `VerifiedSet`.

* Improve comment explaining lint removal

Rewrite the comment explaining why the Clippy lint was ignored.

* Check for spend conflicts in `VerifiedSet`

Refactor to avoid API misuse.

* Test rejected transaction rollback

Using two transactions, perform the same test adding a conflict to both
of them to check if the second inserted transaction is properly
rejected. Then remove any conflicts from the second transaction and add
it again. That should work, because if it doesn't it means that when the
second transaction was rejected it left things it shouldn't in the
cache.

* Test removal of multiple transactions

When removing multiple transactions from the mempool storage, all of the
ones requested should be removed and any other transaction should be
still be there afterwards.

* Increase mempool size to 4, so that spend conflict tests work

If the mempool size is smaller than 4,
these tests don't fail on a trivial removal bug.
Because we need a minimum number of transactions in the mempool
to trigger the bug.

Also commit a proptest seed that fails on a trivial removal bug.
(This seed fails if we remove indexes in order,
because every index past the first removes the wrong transaction.)

* Summarise transaction data in proptest error output

* Summarise spend conflict field data in proptest error output

* Summarise multiple removal field data in proptest error output

And replace the very large proptest debug output with the new summary.

Co-authored-by: teor <teor@riseup.net>
2021-10-10 23:54:46 +00:00
Janito Vaqueiro Ferreira Filho 339fefb6e6
Update shared NU5 dependencies, set the NU5 testnet activation network upgrade parameters (#2825)
* Check return value of zcash_script_new_precomputed_tx

* Set the NU5 testnet activation height to 1_590_000

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update Nu5 constants to new values

* Update ZIP-244 test vectors for new branch ID

* Squashed commit of the following:

commit bdb120a249e3e889a913114a712505defdade1d4
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:54:01 2021 -0400

    Use pallas::Base::from_str_vartime() in sinsemilla tests

commit e99fa4925857840fa65ccfb4a076ec412e416576
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:45:24 2021 -0400

    Compiles

commit a5200181146bfd2aa1e09abea2caaa7a7ceb006e
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 10:15:17 2021 -0400

    Incomplete upgrade of deps

* Squashed commit of the following:

commit 8d1b76ec5626517817c3a4d9f3950acc90a359df
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 04:02:26 2021 +0000

    Update `zcash_script` to support V5 transactions

    Use a newer version of `zcash_script` that has been updated to support
    V5 transactions.

commit 371233628ae61e0c25d6ba8f31d9dba42823becb
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:06:20 2021 +0000

    Update Zcash dependencies

    Update some Zcash crates:

    - `halo2`
    - `incrementalmerkletree' (patch version)
    - `orchard` (patch version)
    - `zcash_history` (patch version)
    - `zcash_note_encryption` (patch version)
    - `zcash_primitives` (patch version)

    And also update the `group` dependency so that the code remains
    compatible.

commit de5cf1ec40c3fc08670fc971cdf3e65e13d9f4c7
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:04:13 2021 +0000

    Update error message assertion

    Use the updated message for the expected error variant.

* Update `zcash_script` to support V5 transactions

Use a newer version of `zcash_script` that has been updated to support
V5 transactions.

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-10-06 11:08:41 +10:00
teor e5f5ac9ce8
Fix or disable recent nightly clippy lints (#2817)
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-10-01 15:26:06 +00:00
Deirdre Connolly 679b4768df
Use complete addition instead of incomplete addition inside sinsemilla_commit (#2807) 2021-09-29 13:00:25 -04:00
Alfredo Garcia c3c302309a
Add orchard binding_verification_key (#2441)
* add orchard binding_verification_key

* Merge branch 'main' into issue2102

* Merge branch 'main' into issue2102

* fix the build

* Merge branch 'main' into issue2102

* Merge branch 'main' into issue2102

* Fix docs

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* Merge branch 'main' into issue2102

* rustfmt

* readd binding validation

* remove #2103 from the TODO list

* Merge branch 'main' into issue2102
2021-08-16 15:16:25 -04:00
Conrado Gouvea eac95bdf10
Cache note commitment tree roots (#2584)
* Cache note commitment tree roots

* Add comments to cached root fields

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2021-08-10 10:33:34 -03:00
teor f09f2a9022
Check remaining transaction value & make value balance signs match the spec (#2566)
* Make Amount arithmetic more generic

To modify generated amounts, we need some extra operations on `Amount`.

We also need to extend existing operations to both `NonNegative` and
`NegativeAllowed` amounts.

* Add a constrain method for ValueBalance

* Derive Eq for ValueBalance

* impl Neg for ValueBalance

* Make some Amount arithmetic expectations explicit

* Explain why we use i128 for multiplication

And expand the overflow error details.

* Expand Amount::sum error details

* Make amount::Error field order consistent

* Rename an amount::Error variant to Constraint, so it's clearer

* Add specific pool variants to ValueBalanceError

* Update coinbase remaining value consensus rule comment

This consensus rule was updated recently to include coinbase transactions,
but Zebra doesn't check block subsidy or miner fees yet.

* Add test methods for modifying transparent values and shielded value balances

* Temporarily set values and value balances to zero in proptests

In both generated chains and proptests that construct their own transactions.

Using zero values reduces value calculation and value check test coverage.
A future change will use non-zero values, and fix them so the check passes.

* Add extra fields to remaining transaction value errors

* Swap the transparent value balance sign to match shielded value balances

This makes the signs of all the chain value pools consistent.

* Use a NonNegative constraint for transparent values

This fix:
* makes the type signature match the consensus rules
* avoids having to write code to handle negative values

* Allocate total generated transaction input value to outputs

If there isn't enough input value for an output, set it to zero.

Temporarily reduce all generated values to avoid overflow.
(We'll remove this workaround when we calculate chain value balances.)

* Consistently use ValueBalanceError for ValueBalances

* Make the value balance signs match the spec

And rename and document methods so their signs are clearer.

* Convert amount::Errors to specific pool ValueBalanceErrors

* Move some error changes to the next PR

* Add extra info to remaining transaction value errors (#2585)

* Distinguish between overflow and negative remaining transaction value errors

And make some error types cloneable.

* Add methods for updating chain value pools (#2586)

* Move amount::test to amount::tests:vectors

* Make ValueBalance traits more consistent with Amount

- implement Add and Sub variants with Result and Assign
- derive Hash

* Clarify some comments and expects

* Create ValueBalance update methods for blocks and transactions

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-09 14:22:26 -03:00
Deirdre Connolly e719c46b1b
Track anchors and note commitment trees in zebra-state (#2458)
* Tidy chain Cargo.toml

* Organize imports

* Add method to get note commitments from all Actions in Orchard shielded data

* Add method to get note commitments from all JoinSplits in Sprout JoinSplitData

* Add Request and Response variants for awaiting anchors

* Add anchors and note commitment trees to finalized state db

* Add (From|Into)Disk impls for tree::Roots and stubs for NoteCommitmentTrees

* Track anchors and note commitment trees in Chain

Append note commitments to their trees when doing update_chain_state_with,
then use the resulting Sapling and Orchard roots to pass to history_tree, and add
new roots to the anchor sets.

* Handle errors when appending to note commitment trees

* Add comments explaining why note commitment are not removed from the tree in revert_chain_state_with

* Implementing note commitments in finalized state

* Finish serialization of Orchard tree; remove old tree when updating finalize state

* Add serialization and finalized state updates for Sprout and Sapling trees

* Partially handle trees in non-finalized state. Use Option for trees in Chain

* Rebuild trees when forking; change finalized state tree getters to not require height

* Pass empty trees to tests; use empty trees by default in Chain

* Also rebuild anchor sets when forking

* Use empty tree as default in finalized state tree getters (for now)

* Use HashMultiSet for anchors in order to make pop_root() work correctly

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS even more

* Apply suggestions from code review

* Add comments about order of note commitments and related methods/fields

* Don't use Option for trees

* Set DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES=1 and restore MAX_PARTIAL_CHAIN_BLOCKS

* Remove unneeded anchor set rebuilding in fork()

* Improve proptest formatting

* Add missing comparisons to eq_internal_state

* Renamed sprout::tree::NoteCommitmentTree::hash() to root()

* Improve comments

* Add asserts, add issues to TODOs

* Remove impl Default for Chain since it was only used by tests

* Improve documentation and assertions; add tree serialization tests

* Remove Sprout code, which will be moved to another branch

* Add todo! in Sprout tree append()

* Remove stub request, response *Anchor* handling for now

* Add test for validating Sapling note commitment tree using test blocks

* Increase database version (new columns added for note commitment trees and anchors)

* Update test to make sure the order of sapling_note_commitments() is being tested

* Improve comments and structure of the test

* Improve variable names again

* Rustfmt

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Conrado P. L. Gouvea <conradoplg@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-07-29 09:37:18 -04:00
Alfredo Garcia ee3c992ca6
Calculate the remaining value in the transparent transaction value pool (#2486)
* add value_balance methods to transparent and shielded

* add value_balance() to transaction

* check the remaining value consensus rule

* change error name

* fix doc and nitpick

* refactor value_balance() method for joinsplit

* changes to value_balance() of Inputs

* implement joinsplits() method(not working)

* remove created methods

* remove special case

* change return error in utilities

* move utils functions to transaction methods

* fix the docs

* simplify some code

* add constrains explicitly

* remove turbofish

* refactor some transaction methods

* fix value balance signs, add docs

* simplify some code

* avoid panic in consensus check

* add missing doc

* move remaining value balance check to the state

* make changes from the last review

Co-authored-by: teor <teor@riseup.net>
2021-07-29 13:49:36 +10:00
Deirdre Connolly 8a4add55f1
Fix Orchard implementation, refactor, and add more test vectors (#2445)
* Support incomplete Pallas addition, all the way down

* Check Orchard key derivation against test vectors (currently not passing)

* Fix up Orchard keys, notes, nullifiers, address, and their tests

* Fix Incoming Viewing Key generation

* Move around test vectors, impl From<SpendingKey> for FullViewingKey

* Add orchard test vectors module

* Pull in and use the Sinsemilla test vectors

* Test Pallas group hashes for Orchard with test vectors

* Move Orchard Arbitrary implementations to arbitrary.rs

* Improvements from code review

* Derive Eq instead of implementing it for SpendAuthorizingKey

* Dedupe Orchard NoteCommitment::extract_x; fix documentation

* Update zebra-chain/src/orchard/note.rs

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

Co-authored-by: Conrado P. L. Gouvea <conradoplg@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-07-16 13:52:35 +00:00
teor 049c92374e
clippy: remove a needless borrow (#2501) 2021-07-16 10:29:22 -03:00
Deirdre Connolly 48cf52735c
Calculate incremental note commitment trees (#2407)
* Make sapling::tree::NoteCommitmentTree incrementally updateable

* Make orchard::tree::NoteCommitmentTree incrementally updateable

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Changed to incrementalmerkletree implementation; update MerkleCRH^Orchard

* Improvements from review; organize file names (vectors.rs -> tree.rs)

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Conrado P. L. Gouvea <conradoplg@gmail.com>
2021-07-15 13:58:36 +00:00
Janito Vaqueiro Ferreira Filho 23fe2c2e94
Update panics by `todo!` and `unimplemented!` macros (#2471)
* Add panic message to `unimplemented!`

So that it is clear why the panic happened upon initial inspection. Also
include a reference to the mempool epic, so that it's easier to find the
issue that tracks the implementation of the missing code.

* Add panic message that references a tracking issue

Make it easy to find the relevant issue if the panic occurs.

* Remove incomplete and currently unnecessary code

The current implementation works, the commented out code was just a
previous improvement idea, which is now tracked by issue #2473.
2021-07-09 01:49:55 +00:00
Conrado Gouvea dd645e7e0c
Add ZIP-0244 TxId Digest support (#2129)
* Add ZIP-0244 TxId Digest support

* Apply suggestions from code review

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-07-06 09:58:22 -03:00
Deirdre Connolly bb974fd282
Orchard note commitment tree test vectors (#2384)
* Orchard note commitment tree and hash test vectors?

* Add failing sinsemilla test vector test

* Support incomplete Pallas addition, all the way down

* Fix sinsemilla sub function S(j), add note commitment tree empty root tests

* Clippy nightly lints

* allow(clippy::derive_hash_xor_eq) for orchard::tree

* Update zebra-chain/src/orchard/sinsemilla.rs

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-06-25 16:27:47 +00:00
Conrado Gouvea 9688811e35
Update multiple crates to ensure bitvec 0.22.3 is being used (#2351)
* Update multiple crates to ensure bitvec 0.22.3 is being used and avoid package conflicts

* Add documentation to zebra-chain::sapling to indicate that ZIP-216 rules are enforced by jubjub

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2021-06-23 13:16:22 +10:00
Alfredo Garcia 2291abc150
Stop panicking on invalid orchard nullifiers (#2267)
* stop panicking on invalid orchard nullifiers

* add context to error

* use `from_bytes_wide` for nullifiers in arbitrary

* orchard::Nullifier vec to array conversion is a bit clearer and simpler

Co-authored-by: teor <teor@riseup.net>
2021-06-15 02:29:19 +00:00
Alfredo Garcia 3dcd407d66
Security: Stop panicking on invalid reserved orchard::Flags bits (#2284)
* stop panicking in deserialize orchard flags

* make the test simpler

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-06-15 10:04:18 +10:00
teor 8ebb415e7c Clippy: remove needless borrows 2021-06-07 18:33:58 -04:00
teor 2f0f379a9e
Standardise clippy lints and require docs (#2238)
* Standardise lints across Zebra crates, and add missing docs

The only remaining module with missing docs is `zebra_test::command`

* Todo -> TODO

* Clarify what a transcript ErrorChecker does

Also change `Error` -> `BoxError`

* TransError -> ExpectedTranscriptError

* Output Descriptions -> Output descriptions
2021-06-04 08:48:40 +10:00
Alfredo Garcia 1685611592
Store orchard nullifiers into the state (#2185)
* add nullifier methods to orchard
* store orchard nullifiers
* bump database version
* update `IntoDisk`
* support V5 in `UpdateWith`
* add a test for finalized state
* Use the latest network upgrade in state proptests
2021-06-01 17:53:13 +10:00
teor f0c271bcfe
Doc: shielded data always contains at least one action (#2218)
Remove an incorrect part of a comment
2021-05-27 12:06:08 -03:00
Deirdre Connolly 7894cec814
Test Eq/PartialEq for orchard keys (#2187)
* Add ConstantTimeEq's for Orchard FullViewingKey and DiversifierKey and affirmatively test

* Fix orchard::keys doc comments with links to make them automatic links

* Exercise ConstantTimeEq for FullViewingKey with a cheap clone

* Allow some clippy lints to pass for somewhat contrived tests

Co-authored-by: teor <teor@riseup.net>
2021-05-27 07:46:05 +10:00
Alfredo Garcia c3820b44eb
Add action methods (#2199)
* add action methods

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-05-26 02:02:42 +00:00
teor d4f3ecd755
Remove redundant cmp_owned and comments (#2186)
* Remove redundant cmp_owned and comments

This is a cleanup after PR #2184.
2021-05-23 21:40:47 +00:00
Deirdre Connolly ad7a29517c
Fix clippy::cmp_owned for (sapling, orchard)::keys with `ConstantTimeEq` (#2184)
* Impl subtle::ConstantTimeEq for orchard SpendingKey, use that in Eq/PartialEq

* Use constant time comparisons for secret key data where applicable

This also makes Clippy happier so that we aren't creating types just to compare.

* Fix clippy::cmp_owned for orchard::keys Eq/PartialEq

By impl'ing ConstantTimeEq for those types where leaks of the value
would compromise access or privacy.

* Make clippy::cmp_owned happy for some sapling::keys
2021-05-22 09:40:49 -03:00