* Put `tor` behind its own `--cfg` flag
* Bump x25519-dalek from 1.2.0 to 2.0.0-pre.1
* Bump zcash_proofs to 0.8.0
* Update `deny.toml`
* Update `Cargo.lock`
* Fix missing doc link warning
* Mention a ticket that tracks the dep. conflict
* Update `.codespellrc`
* Disable the other tor dependency
* Clarify a comment
* Add Tor x25519-dalek issue to README
* Update Cargo.lock
Co-authored-by: teor <teor@riseup.net>
* Move version into a ConnectionInfo struct
* Add negotiated version to ConnectionInfo
Part of this change was generated using:
```
fastmod --fixed-strings ".version(" ".remote_version(" zebra-network
```
* Add the peer address to ConnectionInfo, add ConnectionInfo to Connection
* Return a Client instance from connect_isolated_* functions
This allows library users to access client ConnectionInfo.
* Add and improve debug formatting
* Add peer services and user agent to ConnectionInfo
* Export the Client type, and fix up a zebrad test
* Export types used by the public API
* Split VersionMessage into its own struct
* Use VersionMessage in ConnectionInfo
* Add a public API test for ConnectionInfo
* Wrap ConnectionInfo in an Arc
* Fix some doc links
* Actually wait between initial peer connections
* Add a missing span to initial handshake tasks
* Forward handshake panics to the calling task
* Clarify a handshake comment
* Wrap the entire handshake in a timeout, not just some messages
* Actually delay spawning initial connections, so we don't flood the network
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Fix clippy::let_and_return
* Increase lightwalletd test timeouts for zebrad slowness
* Add a `zebrad_update_sync()` test, that update syncs Zebra without lightwalletd
* Run the zebrad-update-sync test in CI
* Add extra zebrad time to workaround lightwalletd bugs
* Fix the syntax of links in comments
* Fix a mistake in the docs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove unnecessary angle brackets from a link
* Revert the changes for links that serve as references
* Revert "Revert the changes for links that serve as references"
This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.
* Remove `<` `>` from links that serve as references
This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.
* Don't use `<` `>` in normal comments
* Don't use `<` `>` for normal comments
* Revert changes for comments starting with `//`
* Fix some warnings produced by `cargo doc`
* Fix some rustdoc warnings
* Fix some warnings
* Refactor some changes
* Fix some rustdoc warnings
* Fix some rustdoc warnings
* Resolve various TODOs
Co-authored-by: teor <teor@riseup.net>
* Fix some unresolved links
* Allow links to private items
* Fix some unresolved links
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Move peer address validation into its own module
* Add a network parameter to AddressBook and some MetaAddr methods
* Reject invalid initial peers, and connect to them in preferred order
* Reject Flux/ZelCash and misconfigured Zcash peers
* Prefer canonical Zcash ports
* Make peer_preference into a struct method
* Prefer peer addresses with canonical ports for outbound connections
* Also ignore the Zcash regtest port
* Document where field and variant order is required for correctness
* Use the correct peer count
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Fix the syntax of links in comments
* Fix a mistake in the docs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove unnecessary angle brackets from a link
* Revert the changes for links that serve as references
* Revert "Revert the changes for links that serve as references"
This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.
* Remove `<` `>` from links that serve as references
This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.
* Don't use `<` `>` in normal comments
* Don't use `<` `>` for normal comments
* Revert changes for comments starting with `//`
* Fix some warnings produced by `cargo doc`
* Fix some rustdoc warnings
* Fix some warnings
* Refactor some changes
* Fix some rustdoc warnings
* Fix some rustdoc warnings
* Resolve various TODOs
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Fix the syntax of links in comments
* Fix a mistake in the docs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove unnecessary angle brackets from a link
* Revert the changes for links that serve as references
* Revert "Revert the changes for links that serve as references"
This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.
* Remove `<` `>` from links that serve as references
This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.
* Don't use `<` `>` in normal comments
* Don't use `<` `>` for normal comments
* Revert changes for comments starting with `//`
* Fix some warnings produced by `cargo doc`
* Fix some rustdoc warnings
* Fix some warnings
* Refactor some changes
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Upgrade tracing and related dependencies
```sh
cargo upgrade --workspace
tracing-error
tracing-subscrber
color-eyre
tracing-flame
tracing-journald
sentry
sentry-tracing
metrics
metrics-exporter-prometheus
reqwest
```
* Update duplicate dependency checks
* Enable the tracing/env-filter feature
* Fix type inference for metrics
Manual changes, plus:
```sh
fastmod "as _" "as f64"
```
* Tidy up some unrelated test code
* Update metrics-exporter-prometheus API
And make unused dependencies optional.
* Adjust test regexes to new tracing format
Also fix some regex bugs, and refactor to simplify.
* Disable color-eyre span traces and track caller in release builds
* Add a feature that enables extra debugging in release builds
* Clean up some redundant features
* Increase a test timeout
* Fix the syntax of links in comments
* Fix a mistake in the docs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove unnecessary angle brackets from a link
* Revert the changes for links that serve as references
* Revert "Revert the changes for links that serve as references"
This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.
* Remove `<` `>` from links that serve as references
This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.
* Don't use `<` `>` in normal comments
* Don't use `<` `>` for normal comments
* Revert changes for comments starting with `//`
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* clippy: unused import on non-linux platforms
* Fix some instances of clippy::derive_partial_eq_without_eq
* Move a deref to fix clippy::significant_drop_in_scrutinee
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* fix(network): split synthetic NotFoundRegistry from message NotFoundResponse
* docs(network): Improve `notfound` message documentation
* refactor(network): Rename MustUseOneshotSender to MustUseClientResponseSender
```
fastmod MustUseOneshotSender MustUseClientResponseSender zebra*
```
* docs(network): fix a comment typo
* refactor(network): remove generics from MustUseClientResponseSender
* refactor(network): add an inventory collector to Client, but don't use it yet
* feat(network): register missing peer responses as missing inventory
We register this missing inventory based on peer responses,
or connection errors or timeouts.
Inbound message inventory tracking requires peers to send `notfound` messages.
But `zcashd` skips `notfound` for blocks, so we can't rely on peer messages.
This missing inventory tracking works regardless of peer `notfound` messages.
* refactor(network): rename ResponseStatus to InventoryResponse
```sh
fastmod ResponseStatus InventoryResponse zebra*
```
* refactor(network): rename InventoryStatus::inner() to to_inner()
* fix(network): remove a redundant runtime.enter() in a test
* doc(network): the exact time used to filter outbound peers doesn't matter
* fix(network): handle block requests slightly more efficiently
* doc(network): fix a typo
* fmt(network): `cargo fmt` after rename ResponseStatus to InventoryResponse
* doc(test): clarify some test comments
* test(network): test synthetic notfound from connection errors and peer inventory routing
* test(network): improve inbound test diagnostics
* feat(network): add a proptest-impl feature to zebra-network
* feat(network): add a test-only connect_isolated_with_inbound function
* test(network): allow a response on the isolated peer test connection
* test(network): fix failures in test synthetic notfound
* test(network): Simplify SharedPeerError test assertions
* test(network): test synthetic notfound from partially successful requests
* test(network): MissingInventoryCollector ignores local NotFoundRegistry errors
* fix(network): decrease the inventory rotation interval
This stops us waiting 3-4 sync resets (4 minutes) before we retry a missing block.
Now we wait 1-2 sync resets (2 minutes), which is still a reasonable rate limit.
This should speed up syncing near the tip, and on testnet.
* fmt(network): cargo fmt --all
* cleanup(network): remove unnecessary allow(dead_code)
* cleanup(network): stop importing the whole sync module into tests
* doc(network): clarify syncer inventory retry constraint
* doc(network): add a TODO for a fix to ensure API behaviour remains consistent
* doc(network): fix a function doc typo
* doc(network): clarify how we handle peers that don't send `notfound`
* docs(network): clarify a test comment
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* refactor(network): rename Advertised to Available
```sh
fastmod Advertised Available zebra*
fastmod advertised available zebra*
```
* refactor(network): allow different available and missing types inside an InventoryStatus
And rename it to ResponseStatus.
Split the methods between ResponseStatus and an InventoryStatus alias.
* refactor(network): add a block_hash convenience method to InventoryHash
* test(network): improve failure logs for connection tests
* fix(inbound): move address sanitization into the response future
* feat(network): send notfound when Zebra doesn't have a block or transaction
* doc(network): move module docs to the top of each module
This makes them more likely to get updated when the module changes.
* fix(network): stop sending unsupported missing inventory types to the registry
* test(network): inbound messages are forwarded to the registry
* test(inbound): test Peers requests to the inbound service, directly and via TCP
* test(network): notfound block responses are sent by the inbound service
* test(network): notfound tx responses are sent by the inbound service
* test(network): increase sync test mock service timeout
The code that these tests use hasn't actually changed much,
and they are only failing on some platforms (coverage, macOS).
So it seems like the extra concurrent inbound tests have pushed them
past their time limit.
(Perhaps due to TCP system calls, or extra serialization work.)
* doc(network): fix typo
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* test(network): remove unnecessary multi-threaded runtime from tests
This prevents `MockService<zebra_state>` timeouts
in the `sync_block_too_high_extend_tips` test,
at the cost of reducing coverage of different execution orders.
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Remove redundant documentation
The documentation was exactly the same as the documentation from the
trait.
* Calculate a mock time block delta for tests
Simulate a block being added to the chain with a random block time based
on the previous block time and the target spacing time.
* Add a `time` field to `ChainTipBlock`
Store the block time so that it's ready for a future chain that allows
obtaining the chain tip's block time.
* Add `ChainTip::best_tip_block_time` method
Allow obtaining the bes chain tip's block time.
* Add method to obtain both height and block time
Prevent any data races by returning both values so that they refer to
the same chain tip.
* Add `NetworkUpgrade::all_target_spacings` method
Returns all the target spacings defined for a network.
* Create a `NetworkChainTipEstimator` helper type
Isolate the code to calculate the height estimation in a new type, so
that it's easier to understand and doesn't decrease the readability of
the `chain_tip.rs` file.
* Add `ChainTip::estimate_network_chain_tip_height`
This is more of an extension method than a trait method. It uses the
`NetworkChainTipHeightEstimator` to actually perform the estimation, but
obtains the initial information from the current best chain tip.
* Fix typo in documentation
There was an extra closing bracket in the summary line.
* Refactor `MockChainTipSender` into a separate type
Prepare to allow mocking the block time of the best tip as well as the
block height.
* Allow sending mock best tip block times
Add a separate `watch` channel to send the best tip block times from a
`MockChainTipSender` to a `MockChainTip`.
The `best_tip_height_and_block_time` implementation will only return a
value if there's a height and a block time value for the best tip.
* Fix off-by-one height estimation error
Use Euclidean division to force the division result to round down
instead of rounding towards zero. This fixes an off-by-one error when
estimating a height that is lower than the current height, because the
fractionary result was being discarded, and it should have forced the
height to go one block back.
* Fix panics on local times very far in the past
Detect situations that might cause the block height estimate to
underflow, and return the genesis height instead.
* Fix another off-by-one height estimation error
The implementation of `chrono::Duration::num_seconds` adds one to the
number of seconds if it's negative. This breaks the division
calculation, so it has to be compensated for.
* Test network chain tip height estimation
Generate pairs of block heights and check that it's possible to estimate
the larger height from the smaller height and a displaced time
difference.
* feat(network): send notfound messages to the inventory registry
* refactor(network): move the inventory filter into an async function
* feat(network): avoid routing requests to peers that are missing inventory
* test(network): advertised routing is independent of numeric address value
* test(network): peer set routes requests to peers not missing that inventory
* test(network): peer set fails requests if all ready peers are missing that inventory
* fix(clippy): needless-borrow in the peer set
* fix(lint): remove redundant trailing commas in macro calls
There is no clippy lint for this, maybe because some macros
are sensitive to trailing commas.
(But not the ones changed in this commit.)
* test(network): check the exact number of inventory peers
* doc(network): explain why we ignore inventory send failures
* docs(network): explain why a channel error is ignored
* feat(network): create an API for registering missing inventory, but don't use it yet
* feat(constraint): implement AtLeastOne::iter_mut()
* refactor(network): add InventoryStatus::marker() method to remove associated data
* fix(network): prefer current inventory, and missing inventory statuses
* fix(network): if an inventory rotation is missed, delay future rotations
* fix(network): don't immediately rotate a new empty inventory registry
* fix(network): assert that only expected inventory variants are stored in the registry
* test(network): add a basic empty inventory registry test
Also adds an inventory registry update future,
which makes it easier to call from an async context.
* refactor(network): add a convenience API for new InventoryChanges
* feat(network): improve inventory registry logging and metrics
* test(network): make sure advertised and missing inventory is correctly registered
* test(network): check that missing inventory is preferred over advertised
* test(network): check that current inventory is preferred over previous
* test(network): check peer set routes inv requests to advertised peers
* refactor(network): make the InventoryChange API more flexible
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* fix(network): add a send timeout to outbound peer messages
* test(network): test peer send and receive timeouts
And the equivalent success cases:
- spawn the run loop with no messages
- spawn the run loop and send and receive a message
* test(network): check for specific error types in the tests
And add an outbound error test that doesn't expect a response.
* test(network): use bounded fake peer connection channels
This lets us actually trigger send timeouts in the tests.
* refactor(network): rename some confusing types and variables
fastmod peer_inbound_tx peer_tx zebra*
fastmod peer_inbound_rx peer_rx zebra*
fastmod ClientSendTimeout ConnectionSendTimeout zebra*
fastmod ClientReceiveTimeout ConnectionReceiveTimeout zebra*
* doc(network test): explain the purpose of each peer connection test vector
* Add arti as a zebra-network dependency
* Add a method for isolated anonymised Tor connections to a specific hostname
* Add tests for isolated tor connections
* Use a shared tor client instance for all isolated connections
* Silence a spurious tor warning in tests
* Make tor support optional, activate it via a new "tor" feature
* Extra Cargo.lock changes
* fastmod AsyncReadWrite PeerTransport zebra*
* Remove unnecessary PeerTransport generics
* Refactor common test code into a function
* Don't drop the stream until the end of the test
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Refactor setup of `Connection` test vectors
Add a `new_test_connection` helper function to create a `Connection`
instance that's ready for testing.
* Check that no inbound requests are sent
Return the mock inbound service from `new_test_connection` and assert
that no requests were sent to it in any test.
* Replace `&mut Vec<u8>` with an `mpsc` channel
Make it easier to run the connection task in the background, i.e.,
remove any lifetime constraints from the borrowed buffer so that
`Connection` is `'static`.
It's now also easier to assert on individual messages sent from the
`Connection` instance.
* Make `MockServiceBuilder::finish` public
Allow test functions to be generic when creating a `MockService`, so
that caller functions actually determine if the type of `MockService`
assertions.
* Move `new_test_connection` to parent module
Make it more generic so that it can be used later in property tests as
well.
* Derive `Eq` and `PartialEq` for network `Response`
Allow intercepted `Response` instances to be easily compared in tests.
* Test block request cancel causes an error cascade
This is the scenario that caused the block synchronizer to reset every
few minutes, which made it considerably slower.
* Ignore unexpected block responses
It's likely that it's just a response for a previously cancelled block
request.
* Make handshakes generic over AsyncRead + AsyncWrite
* Simplify connect_isolated using ServiceExt::map_err and BoxError
* Move isolated network tests to their own module
* Improve isolated TCP connection tests
* Add an in-memory connection test that uses AsyncReadWrite
* Support connect_isolated on testnet
* Add a wrapper function for isolated TCP connections to an IP address
* Run test tasks for a while, and clean up after them
* Upgrade Zebra dependencies to be compatible with arti, but don't add arti yet
* Fix deny.toml
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Refactor so that RetryLimit::Future is std::marker::Sync
* Make the syncer future std::marker::Send by spawning tips futures
* Download synced blocks in chain order, not HashSet order
* Improve MockService failure messages
* Add closure-based responses to the MockService API
* Move MockChainTip to zebra-chain
* Add a MockChainTipSender type alias
* Support MockChainTip in ChainSync and its downloader
* Add syncer tests for obtain tips, extend tips, and wrong block hashes
* Add block too high tests for obtain tips and extend tips
* Add syncer tests for duplicate FindBlocks response hashes
* Allow longer request delays for mocked services in syncer tests
* Stop holding completed messages until the next inbound message
* Add more info to network message block download debug logs
* Simplify address metrics logs
* Try handling inbound messages as responses, then try as a new request
* Improve address book logging
* Fix a race between the first heartbeat and getaddr requests
* Temporarily reduce the getaddr fanout to 1
* Update metrics when exiting the Connection run loop
* Downgrade some debug logs to trace
* Refactor to create heartbeat sender function
Move the code that's part of the heartbeat task into a separate helper
function.
* Move `Client` initialization down
Keep it closer to where it's actually used, and make it easier to add
new fields to `Client` for the connection and heartbeat tasks.
* Add background task handles to `Client` type
Prepare it to be able to check for panics or errors from the background
tasks.
* Add dummy background tasks to `ClientTestHarness`
Spawn simple timeout tasks as mock connection and heartbeat tasks.
* Fix `PeerSet` tests that use `ClientTestHarness`
Building a `ClientTestHarness` requires a Tokio runtime to be set up, so
the calls were moved into the `async` block.
* Refactor to create `set_task_exited_error`
Make the code reusable for both background tasks.
* Check heartbeat task for errors
Periodically poll it to check if the task has unexpectedly stopped.
* Check if connection background task has stopped
The client service should stop if the connection background task has
exited, because then it's not able to receive any replies.
* Allow aborting mocked `Client` background tasks
Wrap the background tasks in `Abortable`, so that they can be aborted
through the `ClientTestHarness`.
* Test if stopped connection task is detected
Check that stopping the background connection task is something that the
`Client` instance detects and handles correctly.
* Test if stopped heartbeat task is detected
Check that stopping the background heartbeat task is something that the
`Client` instance detects and handles correctly.
* Allow setting custom background tasks
Will be used later to create background tasks that panic.
* Test if `Client` handles panics in connection task
Use a mock background connection task that panics immediately, and check
that the `Client` handles it gracefully.
* Test if `Client` handles panics in heartbeat task
Use a mock background heartbeat task that panics immediately, and check
that the `Client` handles it gracefully.
* Change ticket referenced by `TODO`
The previously linked issue was a broad plan to improve Zebra's shutdown
behavior, while the new issue is more specific, and can be scheduled
sooner.
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: teor <teor@riseup.net>
* Move `MockedClientHandle` to `peer` module
It's more closely related to a `Client` than the `PeerSet`, and this
prepares it to be used by other tests.
* Rename `MockedClientHandle` to `ClientTestHarness`
Reduce confusion, and clarify that the client is not mocked.
Co-authored-by: teor <teor@riseup.net>
* Add clarification to `mock_peers` documentation
Explicitly say how the generated data is returned.
* Rename method to `wants_connection_heartbeats`
The `Client` service only represents one direction of a connection, so
`is_connected` is not the exact term.
Co-authored-by: teor <teor@riseup.net>
* Mock `Client` instead of `LoadTrackedClient`
Move where the conversion from mocked `Client` to mocked
`LoadTrackedClient` in order to make the test helper more easily used by
other tests.
* Use `ClientTestHarness` in `initialize` tests
Replace the boilerplate code to create a fake `Client` instance with
usages of the `ClientTestHarness` constructor.
* Allow receiving requests from `Client` instance
Create a helper type to wrap the result, to make it easier to assert on
specific events after trying to receive a request.
* Allow inspecting the current error in the slot
Share the `ErrorSlot` between the `Client` and the handle, so that the
handle can be used to inspect the contents of the `ErrorSlot`.
* Allow placing an error into the `ErrorSlot`
Assuming it is initially empty. If it already has an error, the code
will panic.
* Allow gracefully closing the request receiver
Close the endpoint with the appropriate call to the `close()` method.
* Allow dropping the request receiver endpoint
Forcefully closes the endpoint.
* Rename field to `client_request_receiver`
Also rename the related methods to include
`outbound_client_request_receiver` to make it more precise.
Co-authored-by: teor <teor@riseup.net>
* Allow dropping the heartbeat shutdown receiver
Allows the `Client` to detect that the channel has been closed.
* Rename fn. to `drop_heartbeat_shutdown_receiver`
Make it clear that it affects the heartbeat task.
Co-authored-by: teor <teor@riseup.net>
* Move `NowOrLater` into a new `now-or-later` crate
Make it easily accessible to other crates.
* Add `IsReady` extension trait for `Service`
Simplifies checking if a service is immediately ready to be called.
* Add extension method to check for readiness error
Checks if the `Service` isn't immediately ready because a call to
`ready` immediately returns an error.
* Rename method to `is_failed`
Avoid negated method names.
Co-authored-by: teor <teor@riseup.net>
* Add a `IsReady::is_pending` extension method
Checks if a `Service` is not ready to be called.
* Use `ClientTestHarness` in `Client` test vectors
Reduce repeated code and try to improve readability.
* Create a new `ClientTestHarnessBuilder` type
A builder to create test `Client` instances using mock data which can be
tracked and manipulated through a `ClientTestHarness`.
* Allow configuring the `Client`'s mocked version
Add a `with_version` builder method.
* Use `ClientTestHarnessBuilder` in `PeerVersions`
Use the builder to set the peer version, so that the `version` parameter
can be removed from the constructor later.
* Use a default mock version where possible
Reduce noise when setting up the harness for tests that don't really
care about the remote peer version.
* Remove `Version` parameter from the `build` method
The `with_version` builder method should be used instead.
* Fix some typos and outdated info in the release checklist
* Add extra client tests for zero and multiple readiness checks (#3273)
And document existing tests.
* Replace `NowOrLater` with `futures::poll!` (#3272)
* Replace NowOrLater with the futures::poll! macro in zebrad
* Replace NowOrLater with the futures::poll! macro in zebra-test
* Remove the now-or-later crate
* remove unused imports
* rustfmt
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Revert "Stop ignoring some completed Responses"
This reverts commit 0383562e1098ee2b49a4b5dd1b37646e6512782f from PR #3120,
but keeps the metrics and logging changes since that commit.
* Document why the request handling needs to happen in this order
* Justify that the ErrorSlot Mutex is deadlock-safe
* Document cancellation safety in the async RFC
* Document task starvation in the async RFC
Co-authored-by: Marek <mail@marek.onl>
* Tweak crawler timings so peers are more likely to be available
* Tweak min peer connection interval so we try all peers
* Let other tasks run between fanouts, so we're more likely to choose different peers
* Let other tasks run between retries, so we're more likely to choose different peers
* Let other tasks run after peer crawler DemandDrop
This makes it more likely that peers will become ready.
* Spawn the address book updater on a blocking thread
* Spawn CandidateSet address book operations on blocking threads
* Replace the PeerSet address book with a metrics watch channel
* Fix comment
* Await spawned address book tasks
* Run the address book update tasks concurrently (except for the mutex)
* Explain an internal-only method better
* Fix a typo
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Drop peer services if their cancel handles are dropped
* Exit the client task if the heartbeat task exits
* Allow multiple errors on a connection without panicking
* Explain why we don't need to send an error when the request is cancelled
* Document connection fields
* Make sure connections don't hang due to spurious timer or channel usage
* Actually shut down the client when the heartbeat task exits
* Add tests for unready services
* Close all senders to peer when `Client` is dropped
* Return a Client error if the error slot has an error
* Add tests for peer Client service errors
* Make Client drop and error cleanups consistent
* Use a ClientDropped error when the Client struct is dropped
* Test channel and error state in peer Client tests
* Move all Connection cleanup into a single method
* Add tests for Connection
* fix typo in comment
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Use a named CancelHeartbeatTask unit struct for the channel type
* Prefer cancel handles in selects, if both are ready
* Fix message metrics to just show the command name
* Add metrics for internal requests and responses
* Add internal requests and responses to the messages dashboard
* Add a canceled metric, and peer addresses to request and response metrics
* Add a canceled messages graph
* Add connection state metrics for currently open connections
* Fix the connection state graph with new metrics
* Always send an error before dropping pending responses
* Move error detail logging into `fail_with`
* Delete an unused timer future
* Make error strings in metrics less verbose
* Downgrade some error logs to info
* Remove a redundant expect
* Avoid unnecessary allocations for connection state metrics
* Fix missed updates to mempool and block gossip metrics
* Replace usage of `discover::Change` with a tuple
Remove the assumption that a `Remove` variant would never be created
with type changes that allow the compiler to guarantee that assumption.
* Add a `version` field to the `Client` type
Keep track of the peer's reported protocol version.
* Create `LoadTrackedClient` type
A `peer::Client` type wrapper that implements `Load`. This helps with
the creation of a client service that has extra peer information to be
accessed without having to send requests.
* Use `LoadTrackedClient` in `initialize`
Ensure that `PeerSet` receives `LoadTrackedClient`s so that it will be
able to query the peer's protocol version later on.
* Require `LoadTrackedClient` in `PeerSet`
Replace the generic type with a concrete `LoadTrackedClient` so that we
can query its version.
* Create `MinimumPeerVersion` helper type
A type to track the current minimum protocol version for connected
peers based on the current block height.
* Use `MinimumPeerVersion` in handshakes
Keep the code to obtain the current minimum peer protocol version in a
central place.
* Add a `MinimumPeerVersion` instance to `PeerSet`
Prepare it to be able to disconnect from outdated peers based on the
current minimum supported peer protocol version.
* Disconnect from ready services for outdated peers
When the minimum peer protocol version is detected to have changed
(because of a network upgrade), remove all ready services of peers that
became outdated.
* Cancel added unready services of outdated peers
Only add an unready service if it's for a peer that has a supported
protocol version. Otherwise, add it but drop the cancel handle so that
the `UnreadyService` can execute and detect that it was cancelled.
* Avoid adding ready services for outdated peers
If a service becomes ready but it's for a connection to an outdated
peer, drop it.
* Improve comment inside `crawl_and_dial`
Describe an edge case that is also handled but was not explicit.
Co-authored-by: teor <teor@riseup.net>
* Test if calculated minimum peer version is correct
Given an arbitrary best chain tip height, check that the calculated
minimum peer protocol version is the expected value.
* Test if minimum version changes with chain tip
Apply an arbitrary list of chain tip height updates and check that for
each update the minimum peer version is calculated correctly.
* Test minimum peer version changed reports
Simulate a series of best chain tip height updates, and check for
minimum peer version updates at least once between them. Changes should
only be reported once.
* Create a `MockedClientHandle` helper type
Used to create and then track a mock `Client` instance.
* Add `MinimumPeerVersion::with_mock_chain_tip`
An extension method useful for tests, that contains some shared
boilerplate code.
* Bias arbitrary `Version`s to be in valid range
Give a 50% chance for an arbitrary `Version` to be in the range of
previously used values the Zcash network.
* Create a `PeerVersions` helper type
Helps with the creation of mocked client services with arbitrary
protocol versions.
* Create a `PeerSetGuard` helper type
An auxiliary type to a `PeerSet` instance created for testing. It keeps
track of any dummy endpoints of channels created and passed to the
`PeerSet` instance.
* Create a `PeerSetBuilder` helper type
Helps to reduce the code when preparing a `PeerSet` test instance.
* Test if outdated peers are rejected by `PeerSet`
Simulate a set of discovered peers being sent to the `PeerSet`. Ensure
that only up-to-date peers are kept by the `PeerSet` and that outdated
peers are dropped.
* Create `BlockHeightPairAcrossNetworkUpgrades` type
A helper type that allows the creation of arbitrary block height pairs,
where one value is before and the other is at or after the activation
height of an arbitrary network upgrade.
* Test if peers are dropped as they become outdated
Simulate a network upgrade, and check that peers that become outdated
are dropped by the `PeerSet`.
* Remove dbg! macros
Co-authored-by: teor <teor@riseup.net>
dependabot[bot]
Marek
teor
Alfredo Garcia
Conrado Gouvea
Janito Vaqueiro Ferreira Filho