Commit Graph

107 Commits

Author SHA1 Message Date
teor cef146edbd
lint(clippy): warn on manual printing to stdout or stderr (#3767)
Most logging should use `tracing::trace!()` or `tracing::debug!()` instead.
2022-03-08 09:14:15 +00:00
teor 744aca9d45
7. test(database): snapshot raw RocksDB column family data (#3630)
* refactor(state): split database access into modules by Zebra types

Also split the genesis block check from the genesis note commitment trees.

* test(db): snapshot column family names

* fix(db): assert that the default column family is empty on open and close

* feat(test): apply `cargo insta` settings to all tests

* doc(db/test): improve test docs

* test(db): snapshot column family data for the empty state

* refactor(db/test): split out the raw database snapshot

* test(db): snapshot raw database for blocks 0-2

* test(db): initial serialized snapshot data for raw database

* test(db): tweak snapshot file names

* test(db): rename snapshots for consistency

* test(db): store empty column families in a single snapshot

* test(db): simplify snapshot files by combining empty snapshots

* doc(db/test): put comment in a better place

* refactor(db): fastmod assert_default_is_empty assert_default_cf_is_empty

* doc(test): explain when insta settings are needed

* fix(state/test): use the network to initialize the state

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2022-03-02 02:44:39 +00:00
teor 729535cf25
fix(test): check for zebrad test output in the correct order (#3643)
The mempool is only activated once, so we must check for that log first.
After mempool activation, the stop regex is logged at least once.
(It might be logged before as well, but we can't rely on that.)

When checking that the mempool didn't activate,
wait for the `zebrad` command to exit,
then check the entire log.
2022-02-25 23:36:20 +00:00
teor 499ae89c80
T2. Add isolated Tor connection API, but don't enable it by default (#3303)
* Add arti as a zebra-network dependency

* Add a method for isolated anonymised Tor connections to a specific hostname

* Add tests for isolated tor connections

* Use a shared tor client instance for all isolated connections

* Silence a spurious tor warning in tests

* Make tor support optional, activate it via a new "tor" feature

* Extra Cargo.lock changes

* fastmod AsyncReadWrite PeerTransport zebra*

* Remove unnecessary PeerTransport generics

* Refactor common test code into a function

* Don't drop the stream until the end of the test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-25 01:46:31 +00:00
teor 00aa5d96a3
Consolidate standard lints into a cargo config file (#3386)
* Move standard lints into .cargo/config.toml

* Ignore "wrong self convention" in a futures-based trait

This lint might only trigger on beta or nightly at the moment.

* Warn if future incompatibile code is added to Zebra

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-24 16:25:06 +00:00
Janito Vaqueiro Ferreira Filho ec207cfa95
Ignore unexpected block responses to fix error cascade when synchronizing blocks (#3374)
* Refactor setup of `Connection` test vectors

Add a `new_test_connection` helper function to create a `Connection`
instance that's ready for testing.

* Check that no inbound requests are sent

Return the mock inbound service from `new_test_connection` and assert
that no requests were sent to it in any test.

* Replace `&mut Vec<u8>` with an `mpsc` channel

Make it easier to run the connection task in the background, i.e.,
remove any lifetime constraints from the borrowed buffer so that
`Connection` is `'static`.

It's now also easier to assert on individual messages sent from the
`Connection` instance.

* Make `MockServiceBuilder::finish` public

Allow test functions to be generic when creating a `MockService`, so
that caller functions actually determine if the type of `MockService`
assertions.

* Move `new_test_connection` to parent module

Make it more generic so that it can be used later in property tests as
well.

* Derive `Eq` and `PartialEq` for network `Response`

Allow intercepted `Response` instances to be easily compared in tests.

* Test block request cancel causes an error cascade

This is the scenario that caused the block synchronizer to reset every
few minutes, which made it considerably slower.

* Ignore unexpected block responses

It's likely that it's just a response for a previously cancelled block
request.
2022-01-20 08:14:16 +00:00
Deirdre Connolly 89b0403582
Enforce Rust edition 2021 (#3332)
* Rust edition 2021: zebra-network, cargo fix --edition and clippy --fix

* Rust edition 2021: zebra-chain, cargo fix --edition

* Rust edition 2021: tower-batch, cargo fix --edition

* Rust edition 2021: tower-fallback, cargo fix --edition

* Rust edition 2021: zebra-client, cargo fix --edition

* Rust edition 2021: zebra-consensus, cargo fix --edition

* Rust edition 2021: zebra-rpc, cargo fix --edition

* Rust edition 2021: zebra-state, cargo fix --edition

* Rust edition 2021: zebra-state, cargo fix --edition

* Rust edition 2021: zebra-test, cargo fix --edition

* Rust edition 2021: zebra-utils, cargo fix --edition

* Rust edition 2021: zebrad, cargo fix --edition

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-14 12:10:18 +00:00
teor d076b999f3
Fix syncer download order and add sync tests (#3168)
* Refactor so that RetryLimit::Future is std::marker::Sync

* Make the syncer future std::marker::Send by spawning tips futures

* Download synced blocks in chain order, not HashSet order

* Improve MockService failure messages

* Add closure-based responses to the MockService API

* Move MockChainTip to zebra-chain

* Add a MockChainTipSender type alias

* Support MockChainTip in ChainSync and its downloader

* Add syncer tests for obtain tips, extend tips, and wrong block hashes

* Add block too high tests for obtain tips and extend tips

* Add syncer tests for duplicate FindBlocks response hashes

* Allow longer request delays for mocked services in syncer tests
2022-01-11 14:11:35 -03:00
Janito Vaqueiro Ferreira Filho b71833292d
Use `MockedClientHandle` in other tests (#3241)
* Move `MockedClientHandle` to `peer` module

It's more closely related to a `Client` than the `PeerSet`, and this
prepares it to be used by other tests.

* Rename `MockedClientHandle` to `ClientTestHarness`

Reduce confusion, and clarify that the client is not mocked.

Co-authored-by: teor <teor@riseup.net>

* Add clarification to `mock_peers` documentation

Explicitly say how the generated data is returned.

* Rename method to `wants_connection_heartbeats`

The `Client` service only represents one direction of a connection, so
`is_connected` is not the exact term.

Co-authored-by: teor <teor@riseup.net>

* Mock `Client` instead of `LoadTrackedClient`

Move where the conversion from mocked `Client` to mocked
`LoadTrackedClient` in order to make the test helper more easily used by
other tests.

* Use `ClientTestHarness` in `initialize` tests

Replace the boilerplate code to create a fake `Client` instance with
usages of the `ClientTestHarness` constructor.

* Allow receiving requests from `Client` instance

Create a helper type to wrap the result, to make it easier to assert on
specific events after trying to receive a request.

* Allow inspecting the current error in the slot

Share the `ErrorSlot` between the `Client` and the handle, so that the
handle can be used to inspect the contents of the `ErrorSlot`.

* Allow placing an error into the `ErrorSlot`

Assuming it is initially empty. If it already has an error, the code
will panic.

* Allow gracefully closing the request receiver

Close the endpoint with the appropriate call to the `close()` method.

* Allow dropping the request receiver endpoint

Forcefully closes the endpoint.

* Rename field to `client_request_receiver`

Also rename the related methods to include
`outbound_client_request_receiver` to make it more precise.

Co-authored-by: teor <teor@riseup.net>

* Allow dropping the heartbeat shutdown receiver

Allows the `Client` to detect that the channel has been closed.

* Rename fn. to `drop_heartbeat_shutdown_receiver`

Make it clear that it affects the heartbeat task.

Co-authored-by: teor <teor@riseup.net>

* Move `NowOrLater` into a new `now-or-later` crate

Make it easily accessible to other crates.

* Add `IsReady` extension trait for `Service`

Simplifies checking if a service is immediately ready to be called.

* Add extension method to check for readiness error

Checks if the `Service` isn't immediately ready because a call to
`ready` immediately returns an error.

* Rename method to `is_failed`

Avoid negated method names.

Co-authored-by: teor <teor@riseup.net>

* Add a `IsReady::is_pending` extension method

Checks if a `Service` is not ready to be called.

* Use `ClientTestHarness` in `Client` test vectors

Reduce repeated code and try to improve readability.

* Create a new `ClientTestHarnessBuilder` type

A builder to create test `Client` instances using mock data which can be
tracked and manipulated through a `ClientTestHarness`.

* Allow configuring the `Client`'s mocked version

Add a `with_version` builder method.

* Use `ClientTestHarnessBuilder` in `PeerVersions`

Use the builder to set the peer version, so that the `version` parameter
can be removed from the constructor later.

* Use a default mock version where possible

Reduce noise when setting up the harness for tests that don't really
care about the remote peer version.

* Remove `Version` parameter from the `build` method

The `with_version` builder method should be used instead.

* Fix some typos and outdated info in the release checklist

* Add extra client tests for zero and multiple readiness checks (#3273)

And document existing tests.

* Replace `NowOrLater` with `futures::poll!` (#3272)

* Replace NowOrLater with the futures::poll! macro in zebrad

* Replace NowOrLater with the futures::poll! macro in zebra-test

* Remove the now-or-later crate

* remove unused imports

* rustfmt

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-12-22 06:13:26 +10:00
teor ea01e305ed
Silence expected anchor errors in tests (#3237) 2021-12-15 22:20:53 +00:00
Deirdre Connolly b973b7a622
Checking of Sprout anchors in non-finalized state (#3123)
* Do prelim checking of Sprout anchors in non-finalized state

Does not check intra-transaction interstitial states yet

* Populate sprout anchors to allow other state tests to pass

* Preliminary interstitial sprout note commitment tree anchor checks implementation

* Make sure only prior anchors are checked in the same transaction

* Add tests

* Refactor a comment

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Use the first `JoinSplit`s from mainnet

* Print debug messages

* Use correct blocks for the tests

Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-12-09 16:50:26 +00:00
Janito Vaqueiro Ferreira Filho 1f756fcc81
Add `zebra_test::init_async` helper function (#3169)
* Use a single-thread shared Tokio runtime

This allows it to pause the time and more closely resembles the
environment that's set by default for asynchronous tests.

* Add a `zebra_test::init_async` helper function

Calls `zebra_test::init` but also constructs a single-thread Tokio
runtime and returns it. This makes it simpler to initialize asynchronous
tests that can't use the `#[tokio::test]` attribute.

* Replace usages of `Runtime::new` in tests

Use the new `zebra_test::init_async()` helper function instead.

* Replace `runtime::Builder::new_current_thread()`

Use the new `zebra_test::init_async()` helper function instead.

* Replace `runtime::Builder::new_multi_thread()`

Use the new `zebra_test::init_async()` helper function instead. The test
with the change doesn't necessarily have to use a multi-thread runtime.
2021-12-09 00:18:17 +00:00
Deirdre Connolly e6ffe374d4
Validate sapling, orchard anchors (#3084)
* Add Transaction::sprout_joinsplits()

* Add Anchor variants to ValidateContextError

* Make Chain anchor collections pub(crate)

* tracing::instrument several methods in state

* Add contains_*_anchors methods to FinalizedState

* Add check::anchors module and function

* Verify that anchors_refer_to_earlier_treestates in when updating chains in non-finalized state

* Update zebra-state/src/service/check/anchors.rs

Co-authored-by: teor <teor@riseup.net>

* Add anchors() to sapling::ShieldedData

* Add sapling_anchors() to Transaction

* Use Transaction::sapling_anchors() in the anchors_refer_to_earlier_treestates() check

* Whoops, itertools

* Add a comment for improvement

Co-authored-by: teor <teor@riseup.net>

* Add & use a cfg(test) method on FinalizedState to prep test state with anchors to allow other tests to pass contextual checks

* Allow test nullifier checks to pass by populating anchor sets, allowing test anchor checks to pass

* Add mainnet block 419202 and its sapling note commitment tree root to test vectors

* Test sapling anchor verification using the first few Sapling blocks data

* Correct comment

* assert_eq instead of assert(bool)

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Update zebra-state/src/service/non_finalized_state.rs

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-11-30 16:05:35 +00:00
Alfredo Garcia dbd49a3f00
Validate coinbase expiration height (#3082)
* add testnet test blocks around nu5

* validate coinbase expiration height

* change const name and doc

Co-authored-by: teor <teor@riseup.net>

* change commit location

Co-authored-by: teor <teor@riseup.net>

* use pre Nu5 rules when there is no activation height

* add sapling final root to nu5 test vectors

* fix tests

Co-authored-by: teor <teor@riseup.net>
2021-11-23 05:17:05 +00:00
Marek 8963007397
Sprout note commitment trees (#3051)
* Implement incremental note commitment Merkle tree for Sprout

* Add tests for Sprout note commitment tree

* Remove the `Arbitrary` attribute

* Reverse the vector of empty roots

* Add more tests

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustdoc

* Rustdoc

* rustdoc links

* Oops, need the trait in scope to use it

* Avoid accessing the wrapped hash directly

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustfmt

* Add typing

* Avoid accessing the wrapped hash directly

* Implement incremental note commitment Merkle tree for Sprout

* Add tests for Sprout note commitment tree

* Remove the `Arbitrary` attribute

* Reverse the vector of empty roots

* Add more tests

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustdoc

* Rustdoc

* rustdoc links

* Oops, need the trait in scope to use it

* Avoid accessing the wrapped hash directly

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustfmt

* Add typing

* Avoid accessing the wrapped hash directly

* Add Overwinter final roots (test vectors)

* Test sprout note commitments trees on Overwinter blocks

* Add new test vectors

* Finish the tests for the note commitment trees

* Make the wrapped hash in `Root` private

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2021-11-18 23:05:52 +00:00
Deirdre Connolly eda83ebe0e
Async Halo2 verifier service (#2645)
* First pass at async Halo2 verification service

Stubs out a batch verifier for the future.
The dependencies for orchard, halo2, librustzcash, zcash_primitives, have
not been resolved.

* Halo2 verifier service and test

* Remove redundant conversion

* Test async halo2 verifier service with pre-computed Orchard shielded data test vectors

* Fix typo

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Assert future result is_ok() in Halo2 verifier test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Shorten tower::Service trait constraints for Halo2 verifier tests

* Remove commented out trait constraints

* .expect() vs .unwrap() to parse orchard::redpallas::VerificationKey

* Use .to_vec() for some test vectors

* Fix self-referential Display impl

* Fix deps

* Distinguish orchard vs zebra_chain::orchard imports

* Add test that halo2 verifier fails with malformed proof inputs

* Use thiserror for Halo2Error

* Use ZcashFoundation/orchard instead of dconnolly/orchard

* Add a link to the issue to remove the zfnd fork of orchard crate

* Update zebra-consensus/Cargo.toml

Co-authored-by: teor <teor@riseup.net>

* Add note

* Move artificial Orchard shielded data test vectors to zebra-test

* Align brackets

* Tidy some trait constraints and debug statements

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2021-11-17 13:26:15 +10:00
teor 1df3bdb089
Check MAX_BLOCK_SIGOPS in the block verifier (#3049)
* Cleanup a function that calls zcash_script

* Remove zebra_test::prelude macros that conflict with the Rust prelude

* Add sigops count support to zebra-script

* Check MAX_BLOCK_SIGOPS in the block verifier

* Test MAX_BLOCK_SIGOPS on generated and historic blocks

* Add SAFETY comments for all unsafe zebra-script code

* Explain where the consensus rule comes from

* Remove unused pretty_assertions dependency

* Allow large test block generation functions with the proptest-impl feature

* Replace `as` with `try_into` for integer conversions in unsafe code

* Expand SAFETY comments
2021-11-15 20:55:32 +00:00
Dimitris Apostolou afb8b3d477
Fix typos (#3055)
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-12 19:30:22 +00:00
teor d6f3b3dc9a
Parse received addrv2 messages (#3022)
* Revert "Remove commented-out code"

This reverts commit 9e69777925f103ee11e5940bba95b896c828839b.

* Implement deserialization for `addrv2` messages

* Limit addr and addrv2 messages to MAX_ADDRS_IN_MESSAGE

* Clarify address version comments

* Minor cleanups and fixes

* Add preallocation tests for AddrV2

* Add serialization tests for AddrV2

* Use prop_assert in AddrV2 proptests

* Use a generic utility method for deserializing IP addresses in `addrv2`

* Document the purpose of a conversion to MetaAddr

* Fix a comment typo, and clarify that comment

* Clarify the unsupported AddrV2 network ID error and enum variant names

```sh
fastmod AddrV2UnimplementedError UnsupportedAddrV2NetworkIdError zebra-network
fastmod Unimplemented Unsupported zebra-network
```

* Fix and clarify unsupported AddrV2 comments

* Replace `panic!` with `unreachable!`

* Clarify a comment about skipping a length check in a test

* Remove a redundant test

* Basic addr (v1) and addrv2 deserialization tests

* Test deserialized IPv4 and IPv6 values in addr messages

* Remove redundant io::Cursor

* Add comments with expected values of address test vectors
2021-11-12 00:25:23 +00:00
Conrado Gouvea 6570ebeeb8
ZIP 212: validate Sapling and Orchard output of coinbase transactions (#3029)
* Part of ZIP 212: validate Sapling and Orchard output of coinbase transactions

* Add Orchard test vector

* Revert accidentally deleted link

* Apply suggestions from code review

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Use height from loop

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Fix formatting

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-11-11 22:18:37 +00:00
Janito Vaqueiro Ferreira Filho 0960e4fb0b
Update to Tokio 1.13.0 (#2994)
* Update `tower` to version `0.4.9`

Update to latest version to add support for Tokio version 1.

* Replace usage of `ServiceExt::ready_and`

It was deprecated in favor of `ServiceExt::ready`.

* Update Tokio dependency to version `1.13.0`

This will break the build because the code isn't ready for the update,
but future commits will fix the issues.

* Replace import of `tokio::stream::StreamExt`

Use `futures::stream::StreamExt` instead, because newer versions of
Tokio don't have the `stream` feature.

* Use `IntervalStream` in `zebra-network`

In newer versions of Tokio `Interval` doesn't implement `Stream`, so the
wrapper types from `tokio-stream` have to be used instead.

* Use `IntervalStream` in `inventory_registry`

In newer versions of Tokio the `Interval` type doesn't implement
`Stream`, so `tokio_stream::wrappers::IntervalStream` has to be used
instead.

* Use `BroadcastStream` in `inventory_registry`

In newer versions of Tokio `broadcast::Receiver` doesn't implement
`Stream`, so `tokio_stream::wrappers::BroadcastStream` instead. This
also requires changing the error type that is used.

* Handle `Semaphore::acquire` error in `tower-batch`

Newer versions of Tokio can return an error if the semaphore is closed.
This shouldn't happen in `tower-batch` because the semaphore is never
closed.

* Handle `Semaphore::acquire` error in `zebrad` test

On newer versions of Tokio `Semaphore::acquire` can return an error if
the semaphore is closed. This shouldn't happen in the test because the
semaphore is never closed.

* Update some `zebra-network` dependencies

Use versions compatible with Tokio version 1.

* Upgrade Hyper to version 0.14

Use a version that supports Tokio version 1.

* Update `metrics` dependency to version 0.17

And also update the `metrics-exporter-prometheus` to version 0.6.1.
These updates are to make sure Tokio 1 is supported.

* Use `f64` as the histogram data type

`u64` isn't supported as the histogram data type in newer versions of
`metrics`.

* Update the initialization of the metrics component

Make it compatible with the new version of `metrics`.

* Simplify build version counter

Remove all constants and use the new `metrics::incement_counter!` macro.

* Change metrics output line to match on

The snapshot string isn't included in the newer version of
`metrics-exporter-prometheus`.

* Update `sentry` to version 0.23.0

Use a version compatible with Tokio version 1.

* Remove usage of `TracingIntegration`

This seems to not be available from `sentry-tracing` anymore, so it
needs to be replaced.

* Add sentry layer to tracing initialization

This seems like the replacement for `TracingIntegration`.

* Remove unnecessary conversion

Suggested by a Clippy lint.

* Update Cargo lock file

Apply all of the updates to dependencies.

* Ban duplicate tokio dependencies

Also ban git sources for tokio dependencies.

* Stop allowing sentry-tracing git repository in `deny.toml`

* Allow remaining duplicates after the tokio upgrade

* Use C: drive for CI build output on Windows

GitHub Actions uses a Windows image with two disk drives, and the
default D: drive is smaller than the C: drive. Zebra currently uses a
lot of space to build, so it has to use the C: drive to avoid CI build
failures because of insufficient space.

Co-authored-by: teor <teor@riseup.net>
2021-11-02 18:46:57 +00:00
teor 3db5ecb80e
Keep reading buffered test process output after it has exited (#2923) 2021-10-20 21:40:39 -04:00
Marek 002c533ea8
Return transaction fee (#2876)
* Get the transaction fee from utxos

* Return the transaction fee from the verifier

* Avoid calculating the fee for coinbase transactions

Coinbase transactions don't have fees. In case of a coinbase transaction, the
verifier returns a zero fee.

* Update the result obtained by `Downloads`
2021-10-15 07:15:10 +10:00
Janito Vaqueiro Ferreira Filho 339fefb6e6
Update shared NU5 dependencies, set the NU5 testnet activation network upgrade parameters (#2825)
* Check return value of zcash_script_new_precomputed_tx

* Set the NU5 testnet activation height to 1_590_000

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update Nu5 constants to new values

* Update ZIP-244 test vectors for new branch ID

* Squashed commit of the following:

commit bdb120a249e3e889a913114a712505defdade1d4
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:54:01 2021 -0400

    Use pallas::Base::from_str_vartime() in sinsemilla tests

commit e99fa4925857840fa65ccfb4a076ec412e416576
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:45:24 2021 -0400

    Compiles

commit a5200181146bfd2aa1e09abea2caaa7a7ceb006e
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 10:15:17 2021 -0400

    Incomplete upgrade of deps

* Squashed commit of the following:

commit 8d1b76ec5626517817c3a4d9f3950acc90a359df
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 04:02:26 2021 +0000

    Update `zcash_script` to support V5 transactions

    Use a newer version of `zcash_script` that has been updated to support
    V5 transactions.

commit 371233628ae61e0c25d6ba8f31d9dba42823becb
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:06:20 2021 +0000

    Update Zcash dependencies

    Update some Zcash crates:

    - `halo2`
    - `incrementalmerkletree' (patch version)
    - `orchard` (patch version)
    - `zcash_history` (patch version)
    - `zcash_note_encryption` (patch version)
    - `zcash_primitives` (patch version)

    And also update the `group` dependency so that the code remains
    compatible.

commit de5cf1ec40c3fc08670fc971cdf3e65e13d9f4c7
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:04:13 2021 +0000

    Update error message assertion

    Use the updated message for the expected error variant.

* Update `zcash_script` to support V5 transactions

Use a newer version of `zcash_script` that has been updated to support
V5 transactions.

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-10-06 11:08:41 +10:00
Janito Vaqueiro Ferreira Filho 175ac99a42
Improve `MockService` ergonomics for sending error responses (#2810)
* Remove `std::error::Error` constraint

The generic `Error` type parameter doesn't have to implement the
standard `Error` trait, so having that constraint only unnecessarily
limits the types that can be used as an error for the service.

* Implement automatic conversion into `Error` type

Make the usage a little more ergonomic by not having to write manual
error type conversions.

* Add a documentation test for an error response

Show an example that responds to a call to the service with an error.

Co-authored-by: teor <teor@riseup.net>
2021-09-30 02:48:26 +00:00
Janito Vaqueiro Ferreira Filho b714b2b3b6
Create a helper `MockService` type to help with writing tests that use mock `tower::Service`s (#2748)
* Implement initial service mocking helpers

Adds a [`MockService`] type, which can be configured and built for usage
in unit tests or proptests. The mocked service can then be used to
intercept requests and respond indivdiually to them.

* Use `MockService in the `mempool::Crawler` test

Refactor it to remove the helper mock function, and use the new
`MockService` helper type.

* Use `MockService` in `CandidateSet` test vectors

Refactor to remove the manual mocking of the peer set service.

* Panic if a response is not sent by `MockService`

Change the current semantics to require all `MockService` usages to
respond to every intercepted request.

A `must_use` attribute was added to the `ResponseSender` so that the
compiler can warn when this doesn't happen.

* Allow generic error types in `MockService`

Replace the hard-coded `BoxError` as the `Service`'s error type with a
generic type parameter. This allows mocking services in locations that
require specific error types.

* Add a `ResponseSender::request` getter

Allow inspecting the request again before responding, and using
information from the request in the response.

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-09-21 17:44:59 +00:00
Conrado Gouvea 40e350c342
Always compute sighash with librustzcash (#2469)
* Always use librustzcash for sighash and remove old sighash code

Also added ZIP-143 test vectors

* Remove librustzcash_sighash test that is no longer needed
2021-07-09 09:55:08 +10:00
Conrado Gouvea 1c919fbabb
Fix bug in sighash for coinbase transactions (#2459) 2021-07-08 16:05:40 -03:00
Conrado Gouvea fdfa3cbdc6
Add ZIP-244 signature hash support (#2165)
* ZIP-244 sighash implementation using librustzcash

* ZIP-244: fix sighash test; add roundtrip test; update vectors

* Improvements from review; renamed sighash::Hash to SigHash
2021-07-07 08:27:10 +10:00
Conrado Gouvea dd645e7e0c
Add ZIP-0244 TxId Digest support (#2129)
* Add ZIP-0244 TxId Digest support

* Apply suggestions from code review

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-07-06 09:58:22 -03:00
Janito Vaqueiro Ferreira Filho f33923f12f
Create a shared Tokio runtime for tests (#2397)
* Add a `zebra_test::RUNTIME` shared runtime

Create a lazily instantiated Tokio runtime that can be shared by tests.

* Split tests that require a shared runtime

Split two tests that were previously in one because of the need to share
a single Tokio runtime. With the `zebra_test::RUNTIME`, they can now
share the runtime without having to be a single test.
2021-07-01 10:30:31 +10:00
teor 9cb7ee4d0e
Release Blocker? Disable IPv6 tests when $ZEBRA_SKIP_IPV6_TESTS is set (#2405)
* Disable IPv6 tests when $ZEBRA_SKIP_IPV6_TESTS is set

This allows users to disable IPv6 tests in environments where IPv6 is not
configured.

* Add network test env var constants

* Replace env strings with constants

fastmod '"ZEBRA_SKIP_NETWORK_TESTS"' zebra_test::net::ZEBRA_SKIP_NETWORK_TESTS
fastmod '"ZEBRA_SKIP_IPV6_TESTS"' zebra_test::net::ZEBRA_SKIP_IPV6_TESTS

* Add functions to skip network tests

* Replace test network env var checks with test function

fastmod --fixed-strings 'env::var_os(zebra_test::net::ZEBRA_SKIP_NETWORK_TESTS).is_some()' 'zebra_test::net::zebra_skip_network_tests()'
fastmod --fixed-strings 'env::var_os(zebra_test::net::ZEBRA_SKIP_IPV6_TESTS).is_some()' 'zebra_test::net::zebra_skip_ipv6_tests()'

* Remove redundant logging and use statements
2021-06-29 11:20:32 +10:00
teor 0d5e5bec3c
Improve docs and panic messages for zebra_test::command (#2406) 2021-06-28 10:35:56 -03:00
teor bcd5f2c50d
Gossip dynamic local listener ports to peers (#2277)
* Gossip dynamically allocated listener ports to peers

Previously, Zebra would either gossip port `0`, which is invalid, or skip
gossiping its own dynamically allocated listener port.

* Improve "no configured peers" warning

And downgrade from error to warning, because inbound-only nodes are a
valid use case.

* Move random_known_port to zebra-test

* Add tests for dynamic local listener ports and the AddressBook

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-06-23 07:59:06 +10:00
teor 56ef08e385 Rewrite acceptance test matching
- Add a custom semver match for `zebrad` versions
- Prefer "line contains string" matches, so tests ignore minor changes
- Escape regex meta-characters when a literal string match is intended
- Rename test functions so they are more precise
- Rewrite match internals to remove duplicate code and enable custom matches
- Document match functions
2021-06-10 22:46:33 -04:00
Alfredo Garcia 96a1b661f0
Rate limit initial genesis block download retries, Credit: Equilibrium (#2255)
* implement and test a rate limit in `request_genesis()`
* add `request_genesis_is_rate_limited` test to sync
* add ensure_timeouts constraint for GENESIS_TIMEOUT_RETRY
* Suppress expected warning logs in zebrad tests

Co-authored-by: teor <teor@riseup.net>
2021-06-09 23:39:51 +00:00
teor c453fbf6f6
Add final sapling root test vectors (#2243)
* Add sapling final root test vectors

Also tidy some formatting and imports

* Doc: final sapling roots can be duplicated

* Reverse the byte order of final sapling root test vectors

This makes the test vectors match the byte order in the block header,
rather than the zcashd RPC responses.

* Ignore pre-sapling block header commitments

Previously, Zebra expected this reserved field to be all zeroes,
but some mainnet and testnet blocks had other values.

* Test structural and semantic validation of the block commitment field

History roots are excluded from these tests, because they require
contextual validation.
2021-06-04 10:31:47 -03:00
teor 2f0f379a9e
Standardise clippy lints and require docs (#2238)
* Standardise lints across Zebra crates, and add missing docs

The only remaining module with missing docs is `zebra_test::command`

* Todo -> TODO

* Clarify what a transcript ErrorChecker does

Also change `Error` -> `BoxError`

* TransError -> ExpectedTranscriptError

* Output Descriptions -> Output descriptions
2021-06-04 08:48:40 +10:00
teor 227757ae68 Clippy nightly: replace a "while let" with a "for" loop 2021-05-24 22:04:54 -04:00
teor 9c0c2f0848 Add Testnet Post-Halving block test vectors 2021-03-16 12:39:07 -04:00
teor 8444b78e01 Add Mainnet Canopy block test vectors
Including one block that just happened to be 72 kB, in binary format.
2021-03-16 12:39:07 -04:00
teor e1002ce2ce Test checkpoints on mainnet and testnet 2021-03-04 08:06:31 +10:00
teor 895bb43ead Clippy: Fix inconsistent struct member orders lint 2021-03-01 23:31:18 -05:00
teor d4915c18e7
Fix inverted was_killed logic (#1779)
Also improve the error messages and code structure.
2021-02-20 06:26:00 +10:00
teor b0bc4a79c9 Disable conflict failure cleanup on macOS 2021-02-20 06:22:11 +10:00
teor c51fd688ee Skip node2.is_running() on Windows
`node2.is_running()` can return `true` on Windows, even though `node2`
has logged a panic. This cleanup code only runs if `node2` fails to panic
and exit as expected. So it's ok for us to skip it.

See #1781 for details.
2021-02-19 18:03:07 +10:00
teor 631fe22422 Fix conflict test node termination
On Windows, if a process is killed after it is dead, it returns `true`
for `was_killed`. Instead, check if the process is running before killing
it.

Also make the section where processes are running as short as possible,
and include context for both processes in every error.
2021-02-19 18:03:07 +10:00
teor bcb9cead88
Kill running acceptance test nodes on error (#1770)
And show the output from those nodes.

These changes help us diagnose errors that happen while one or more
acceptance test nodes are running.
2021-02-19 06:46:27 +10:00
Alfredo Garcia 4b34482264
Add hints to port conflict and lock file panics (#1535)
* add hint for port error
* add issue filter for port panic
* add lock file hint
* add metrics endpoint port conflict hint
* add hint for tracing endpoint port conflict
* add acceptance test for resource conflics
* Split out common conflict test code into a function
* Add state, metrics, and tracing conflict tests

* Add a full set of stderr acceptance test functions

This change makes the stdout and stderr acceptance test interfaces
identical.

* move Zcash listener opening
* add todo about hint for disk full
* add constant for lock file
* match path in state cache
* don't match windows cache path

* Use Display for state path logs

Avoids weird escaping on Windows when using Debug

* Add Windows conflict error messages

* Turn PORT_IN_USE_ERROR into a regex

And add another alternative Windows-specific port error

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Jane Lusby <jane@zfnd.org>
2021-01-29 22:36:33 +10:00
teor 49e6150427 Use collect() to avoid a lint 2021-01-19 11:02:20 -05:00