* lint: enable more clippy checks for bug-prone code
* fix(lint): stop denying lints, to avoid being excluded from Crater
Also categorise lints.
* lint: add some lints to the TODO list
* refactor(arithmetic): partial fixes for some integer arithmetic lints
* Document some weird lint behaviour
* Updating zebra-test to v1.0.0-beta.4
* Updating tower-fallback to v0.2.16
* Updating tower-batch to v0.2.20
* Updating zebra-chain to v1.0.0-beta.4
* Updating zebra-script to v1.0.0-beta.4
* Updating zebra-network to v1.0.0-beta.4
* Updating zebra-state to v1.0.0-beta.4
* Updating zebra-consensus to v1.0.0-beta.4
* Updating zebra-utils to v1.0.0-beta.4
* Updating zebrad to v1.0.0-beta.4
* Update Cargo.lock with updated crate versions
* Add section for 1.0.0-beta.4 in the CHANGELOG
* Update CHANGELOG.md
Co-authored-by: teor <teor@riseup.net>
* Update README.md
Co-authored-by: teor <teor@riseup.net>
* Update book/src/user/install.md
Co-authored-by: teor <teor@riseup.net>
* Update release description to mention Rust 2021
Update the changelog to mention that all crates now use Rust 2021
Edition.
* Elaborate on Section 3.6 documentation entry
Add the title of the section and mention that it's a section of the
Zcash protocol specification.
* Make changelog entries consistent
Use the same format for entries related to consensus rule documentation.
Co-authored-by: Marek <mail@marek.onl>
* Update README about validated consensus rules
Zebra now validates all documented consensus rules.
* Add changelog entry for newly merged PR
Describe the security fix in the changelog.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
The 2021 edition activates the v2 cargo resolver.
This broke implicit feature resolution,
but only when `zebra-chain`'s dev or bench targets were compiled by themselves.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Move standard lints into .cargo/config.toml
* Ignore "wrong self convention" in a futures-based trait
This lint might only trigger on beta or nightly at the moment.
* Warn if future incompatibile code is added to Zebra
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Refactor so that RetryLimit::Future is std::marker::Sync
* Make the syncer future std::marker::Send by spawning tips futures
* Download synced blocks in chain order, not HashSet order
* Improve MockService failure messages
* Add closure-based responses to the MockService API
* Move MockChainTip to zebra-chain
* Add a MockChainTipSender type alias
* Support MockChainTip in ChainSync and its downloader
* Add syncer tests for obtain tips, extend tips, and wrong block hashes
* Add block too high tests for obtain tips and extend tips
* Add syncer tests for duplicate FindBlocks response hashes
* Allow longer request delays for mocked services in syncer tests
* Ensure that sapling::keys::TransmissionKey jubjub point is always in the prime order group
* Adjust TransmissionKey check; add AuthorizingKey check
* Move ValueCommitment small order check to deserialization
* Apply suggestions from code review
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Use is_torsion_free() instead of is_identity() and is_prime_order()
* Add EphemeralPublicKey small order check on instantiation; remove old checks
* Use VerificationKey instead of VerificationKeyBytes; fix tests
* Use ValidatingKey instead of VerificationKeyBytes for rk
* Reject identity when creating an Orchard EphemeralPublicKey
* Make documentation more consistent, use generator in tests
* s/JubJub/Jubjub/
* Fix zebra-consensus tests (insert_fake_orchard_shielded_data)
* Create NotSmallOrderValueCommitment, since intermediate values can be the identity
* Clarify documentation
* rustdoc
* Tidy rustdoc
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
* Create a strategy for block heights after Sapling
Provides an arbitrary network (mainnet or testnet) and a block height
between the Sapling activation height on that network and the maximum
block height.
* Create a helper function to select block heights
Allows generating block heights inside a range using a scale factor
between 0 and 1.
* Allow specifying the outpoint index for mock UTXOs
Avoid creating multiple transparent transfers in the same transaction
with the same source UTXO, which would lead to a double spend.
* Create helper function to mock multiple transfers
Given relative block height scale factors, create a mock transparent
transfer for each one of them.
Also add a constant that serves as a guideline for the maximum number of
transparent transfers to mock.
* Create helper function to sanitize tx. version
Make sure the arbitrary transaction version is valid for the network
(testnet or mainnet) at the specified block height.
* Create `mock_transparent_transaction` helper func.
Creates a V4 or V5 mock transaction that only includes transparent
inputs and outputs.
* Create helper function for transaction validation
Performs the actual tested action of verifying a transaction. It sets up
the verifier and uses it to obtain the verification result.
* Test if zero lock time means unlocked
Generate arbitrary transactions with zero lock time, and check that they
are accepted by the transaction verifier.
* Allow changing the sequence number of an input
Add a setter method for a `transparent::Input`'s sequence number. This
setter is only available for testing.
* Test if sequence numbers can disable lock time
Create arbitrary transactions and set the sequence numbers of all of its
inputs to `u32::MAX` to see if that disables the lock time and the
transactions are accepted by the verifier.
* Test block height lock times
Make sure that the transaction verifier rejects transactions that are
still locked at a certain block height.
* Test block time lock times
Test that the transaction verifier rejects a transaction that is
validated at a block time that's before the transaction's lock time.
* Test unlocking by block height
Test that transactions unlocked at an earlier block height are accepted
by the transaction verifier.
* Test transactions unlocked by the block time
Test that transactions that were unlocked at a previous block time are
accepted by the transaction verifier.
* Fix an incorrect method comment
Co-authored-by: teor <teor@riseup.net>
* Tweak crawler timings so peers are more likely to be available
* Tweak min peer connection interval so we try all peers
* Let other tasks run between fanouts, so we're more likely to choose different peers
* Let other tasks run between retries, so we're more likely to choose different peers
* Let other tasks run after peer crawler DemandDrop
This makes it more likely that peers will become ready.
* Integrate JoinSplit verifier with transaction verifier
* Add test with malformed Groth16 Output proof
* Use TryFrom instead of From in ItemWrapper to correctly propagate malformed proof errors
* Simplify by removing ItemWrapper and directly TryFrom into Item
* Fix existing tests to work with JoinSplit validation
* Apply suggestions from code review
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
* Do prelim checking of Sprout anchors in non-finalized state
Does not check intra-transaction interstitial states yet
* Populate sprout anchors to allow other state tests to pass
* Preliminary interstitial sprout note commitment tree anchor checks implementation
* Make sure only prior anchors are checked in the same transaction
* Add tests
* Refactor a comment
* Refactor rustdoc
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Use the first `JoinSplit`s from mainnet
* Print debug messages
* Use correct blocks for the tests
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* fix parse_coinbase_height()
* move tests and create test for parse_coinbase_height()
* add a coinbase height round trip prop test
* fix range
Co-authored-by: teor <teor@riseup.net>
* extend examples in test
* add more round trip testing
* extend the range of test
Co-authored-by: teor <teor@riseup.net>
* add test for single byte
Co-authored-by: teor <teor@riseup.net>
* Stop checking the entire AddressBook for each connection attempt
* Stop redundant peer time checks within the address book
* Stop calling `Instant::now` 3 times for each address book update
* Only get the time once each time an address book method is called
* Update outdated comment
* Use an OrderedMap to efficiently store address book peers
* Add address book order tests
Zebra's latest beta continues implementing zero-knowledge proof and note commitment tree validation. In this release, we have finished implementing transaction header, transaction amount, and Zebra-specific NU5 validation. (NU5 mainnet validation is waiting on an `orchard` crate update, and some consensus parameter updates.)
We also fix a number of security issues that could pose a local denial of service risk, or make it easier for an attacker to make a node follow a false chain.
As of this release, Zebra will automatically download and cache the Sprout and Sapling Groth16 circuit parameters. The cache uses around 1 GB of disk space. These cached parameters are shared across all Zebra and `zcashd` instances run by the same user.
See CHANGELOG.md for the full list of changes in this release.
* Add Transaction::sprout_joinsplits()
* Add Anchor variants to ValidateContextError
* Make Chain anchor collections pub(crate)
* tracing::instrument several methods in state
* Add contains_*_anchors methods to FinalizedState
* Add check::anchors module and function
* Verify that anchors_refer_to_earlier_treestates in when updating chains in non-finalized state
* Update zebra-state/src/service/check/anchors.rs
Co-authored-by: teor <teor@riseup.net>
* Add anchors() to sapling::ShieldedData
* Add sapling_anchors() to Transaction
* Use Transaction::sapling_anchors() in the anchors_refer_to_earlier_treestates() check
* Whoops, itertools
* Add a comment for improvement
Co-authored-by: teor <teor@riseup.net>
* Add & use a cfg(test) method on FinalizedState to prep test state with anchors to allow other tests to pass contextual checks
* Allow test nullifier checks to pass by populating anchor sets, allowing test anchor checks to pass
* Add mainnet block 419202 and its sapling note commitment tree root to test vectors
* Test sapling anchor verification using the first few Sapling blocks data
* Correct comment
* assert_eq instead of assert(bool)
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Update zebra-state/src/service/non_finalized_state.rs
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Download and load Sprout parameters using zcash_proofs
Also update some librustzcash dependencies, to avoid duplicate dependencies.
* Update upstream orchard to avoid a compilation error
* Skip librustzcash batch refactor for now, to avoid compilation errors
* Change the cache ID, so we actually cache Sprout
* Move existing file checks into zcash_proofs
* Add a 1 hour timeout to parameter file downloads
* Give other tasks priority, before spawning the download task
* Update to the latest version of our modified librustzcash fork
* Change the cache key for Sprout
* Add 40 minutes to CI timeouts for occasional sprout downloads
* Update to zcash_proofs with split downloads
* Check file sizes to help debug parameter load failures in zcash_proofs
* Start the second download once the first has finished in zcash_proofs
* Document the parameter download task
* Stop hashing existing files twice
* validate consensus rule: negative fee not allowed
* fix a test TODO
* fix imports
* move import back
* fix panic text
* join consensus rule check code
* match assertion better in tests
* fix test
* fix consensus rule validation
* remove panics
* Delete a TODO
Co-authored-by: teor <teor@riseup.net>
* Create a `LockTime::unlocked` helper constructor
Returns a `LockTime` that is unlocked at the genesis block.
* Return `Option<LockTime>` from `lock_time` method
Prepare to return `None` for when a transaction has its lock time
disabled.
* Return `None` instead of zero `LockTime`
Because a zero lock time means that the transaction was unlocked at the
genesis block, so it was never actually locked.
* Rephrase zero lock time check comment
Clarify that the check is not redundant, and is necessary for the
genesis transaction.
Co-authored-by: teor <teor@riseup.net>
* Add a `transparent::Input::sequence` getter method
Retrieve a transparent input's sequence number.
* Check if lock time is enabled by a sequence number
Validate the consensus rule that the lock time is only enabled if at
least one transparent input has a value different from `u32::MAX` as its
sequence number.
* Add more Zcash specific details to comment
Explain the Zcash specific lock time behaviors.
Co-authored-by: teor <teor@riseup.net>
* Add `time` field to `Request::Block` variant
The block time to use to check if the transaction was unlocked and
allowed to be included in the block.
* Add `Request::block_time` getter
Returns the block time for the block that owns the transaction being
validated or the current time plus a tolerance for mempool transactions.
* Validate transaction lock times
If they are enabled by a transaction's transparent input sequence
numbers, make sure that they are in the past.
* Add comments with consensus rule parts
Make it easier to map what part of the consensus rule each match arm is
responsible for.
Co-authored-by: teor <teor@riseup.net>
* add testnet test blocks around nu5
* validate coinbase expiration height
* change const name and doc
Co-authored-by: teor <teor@riseup.net>
* change commit location
Co-authored-by: teor <teor@riseup.net>
* use pre Nu5 rules when there is no activation height
* add sapling final root to nu5 test vectors
* fix tests
Co-authored-by: teor <teor@riseup.net>
* Implement incremental note commitment Merkle tree for Sprout
* Add tests for Sprout note commitment tree
* Remove the `Arbitrary` attribute
* Reverse the vector of empty roots
* Add more tests
* Refactor rustdoc
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Refactor rustdoc
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* rustdoc
* Rustdoc
* rustdoc links
* Oops, need the trait in scope to use it
* Avoid accessing the wrapped hash directly
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* rustfmt
* Add typing
* Avoid accessing the wrapped hash directly
* Implement incremental note commitment Merkle tree for Sprout
* Add tests for Sprout note commitment tree
* Remove the `Arbitrary` attribute
* Reverse the vector of empty roots
* Add more tests
* Refactor rustdoc
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Refactor rustdoc
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* rustdoc
* Rustdoc
* rustdoc links
* Oops, need the trait in scope to use it
* Avoid accessing the wrapped hash directly
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* rustfmt
* Add typing
* Avoid accessing the wrapped hash directly
* Add Overwinter final roots (test vectors)
* Test sprout note commitments trees on Overwinter blocks
* Add new test vectors
* Finish the tests for the note commitment trees
* Make the wrapped hash in `Root` private
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
* First pass at async Halo2 verification service
Stubs out a batch verifier for the future.
The dependencies for orchard, halo2, librustzcash, zcash_primitives, have
not been resolved.
* Halo2 verifier service and test
* Remove redundant conversion
* Test async halo2 verifier service with pre-computed Orchard shielded data test vectors
* Fix typo
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Assert future result is_ok() in Halo2 verifier test
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Shorten tower::Service trait constraints for Halo2 verifier tests
* Remove commented out trait constraints
* .expect() vs .unwrap() to parse orchard::redpallas::VerificationKey
* Use .to_vec() for some test vectors
* Fix self-referential Display impl
* Fix deps
* Distinguish orchard vs zebra_chain::orchard imports
* Add test that halo2 verifier fails with malformed proof inputs
* Use thiserror for Halo2Error
* Use ZcashFoundation/orchard instead of dconnolly/orchard
* Add a link to the issue to remove the zfnd fork of orchard crate
* Update zebra-consensus/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* Add note
* Move artificial Orchard shielded data test vectors to zebra-test
* Align brackets
* Tidy some trait constraints and debug statements
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: teor <teor@riseup.net>
* Upgrade aes and fpe
* Upgrade bellman, bls12_381, jubjub to latest
* Upgrade x25519-dalek to 1.2.0 and curve25519-dalek to 3.2.0 in the Cargo.lock
* Skip outdated hdrhistogram rather than its dependencies
Co-authored-by: teor <teor@riseup.net>
* Cleanup a function that calls zcash_script
* Remove zebra_test::prelude macros that conflict with the Rust prelude
* Add sigops count support to zebra-script
* Check MAX_BLOCK_SIGOPS in the block verifier
* Test MAX_BLOCK_SIGOPS on generated and historic blocks
* Add SAFETY comments for all unsafe zebra-script code
* Explain where the consensus rule comes from
* Remove unused pretty_assertions dependency
* Allow large test block generation functions with the proptest-impl feature
* Replace `as` with `try_into` for integer conversions in unsafe code
* Expand SAFETY comments
* Revert "Remove commented-out code"
This reverts commit 9e69777925f103ee11e5940bba95b896c828839b.
* Implement deserialization for `addrv2` messages
* Limit addr and addrv2 messages to MAX_ADDRS_IN_MESSAGE
* Clarify address version comments
* Minor cleanups and fixes
* Add preallocation tests for AddrV2
* Add serialization tests for AddrV2
* Use prop_assert in AddrV2 proptests
* Use a generic utility method for deserializing IP addresses in `addrv2`
* Document the purpose of a conversion to MetaAddr
* Fix a comment typo, and clarify that comment
* Clarify the unsupported AddrV2 network ID error and enum variant names
```sh
fastmod AddrV2UnimplementedError UnsupportedAddrV2NetworkIdError zebra-network
fastmod Unimplemented Unsupported zebra-network
```
* Fix and clarify unsupported AddrV2 comments
* Replace `panic!` with `unreachable!`
* Clarify a comment about skipping a length check in a test
* Remove a redundant test
* Basic addr (v1) and addrv2 deserialization tests
* Test deserialized IPv4 and IPv6 values in addr messages
* Remove redundant io::Cursor
* Add comments with expected values of address test vectors
* Add a `Duration32::from_days` constructor
Make it simpler to construct a `Duration32` representing a certain
number of days.
* Add `MetaAddr::was_not_recently_seen` method
A helper method to check if a peer was never seen before or if it was
last seen a long time ago. This will be one of the conditions to
consider a peer as unreachable.
* Add `MetaAddr::is_probably_unreachable` method
A helper method to check if a peer should be considered unreachable. It
is considered unreachable if recent connection attempts have failed and
it was not recently seen.
If a peer is considered unreachable, Zebra shouldn't attempt to connect
to it again.
* Do not keep trying to connect to unreachable peer
A peer is probably unreachable if it was last seen a long time ago and
if it's last connection attempt failed.
* Test `was_not_recently_seen`
Redo the calculation on arbitrary `MetaAddr`s.
* Test `is_probably_unreachable`
Redo the calculation on arbitrary `MetaAddr`s.
* Test if probably unreachable peers are ignored
Given an `AddressBook` with a list of arbitrary `MetaAddr`s, check that
none of the peers listed for a reconnection is probably unreachable.
* Rename unit test to improve clarity
Remove the double negative from the name.
Co-authored-by: teor <teor@riseup.net>
* Rename constant to `MAX_RECENT_PEER_AGE`
Make the purpose of the constant clearer.
Co-authored-by: teor <teor@riseup.net>
* Rename method to `last_seen_is_recent`
Remove the double negative from the name.
* Rename method to `is_probably_reachable`
Avoid having to negate the result of the method in security critical
filter.
* Move check into `is_ready_for_connection_attempt`
Make sure the check is used in any place that requires a peer that's
ready for a connection attempt.
* Improve test documention
Describe the goal of the test better.
Co-authored-by: teor <teor@riseup.net>
* Improve `is_probably_reachable` documentation
List the conditions as bullet points.
Co-authored-by: teor <teor@riseup.net>
* Document what happens when peers have no last seen time
Co-authored-by: teor <teor@riseup.net>
* Implement addr v1 serialization using a separate AddrV1 type
* Remove commented-out code
* Split the address serialization code into modules
* Reorder v1 and in_version fields in serialization order
* Fix a missed search-and-replace
* Explain conversion to MetaAddr
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Add CompactSize64 and CompactSizeMessage types
But don't remove read_compactsize and write_compactsize yet.
* Fix CompactSize capitalisation
```sh
fastmod compactSize CompactSize zebra* book
fastmod compactsize CompactSize zebra* book
```
* Make CompactSize patterns consistent with integer lengths
* Replace unwrap_err with asserting is_err
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Replace a panic with an assertion
* Make generic serialization use CompactSizeMessage
* Fix type inference and borrow-checker errors
* Doctest fixes
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
teor
Janito Vaqueiro Ferreira Filho
Marek
dependabot[bot]
Deirdre Connolly
Alfredo Garcia
Conrado Gouvea
Pili Guerra
Dimitris Apostolou