Commit Graph

75 Commits

Author SHA1 Message Date
Alfredo Garcia 37595c4b32
Mempool support for transaction expiration (#2774)
* mempool - support transaction expiration

* use `LatestChainTip` instead of state call

* clippy

* remove spawn task

* remove non needed async from function

* remove return value

* add a `expiry_height_mut()` method to `Transaction` for testing purposes

* fix `remove_expired_transactions()`

* add a `mempool_transaction_expiration()` test

* tidy cleanup to `expiry_height()`

* improve docs

* fix the build

* try fix macos build

* extend tests

* add doc to function

* clippy

* fix build

* start tests at block two
2021-09-29 16:52:44 +00:00
Conrado Gouvea c6878d9b63
Cancel download and verify tasks when the mempool is deactivated (#2764)
* Cancel download and verify tasks when the mempool is deactivated

* Refactor enable/disable logic to use a state enum

* Add helper test functions to enable/disable the mempool

* Add documentation about errors on service calls

* Improvements from review

* Improve documentation

* Fix bug in test

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2021-09-29 09:06:40 +10:00
Janito Vaqueiro Ferreira Filho 11b77afec7
Refactor mempool tests (#2771)
* Remove return of redundant vector length

An attempt to improve readability a bit by not returning a tuple with a
value that can be obtained from a single return type.

* Refactor `unmined_transactions_in_blocks`

Use a more functional style to try to make it a bit clearer.

* Use ranges in `unmined_transactions_in_blocks`

Allow a finer control over the block range to extract the transactions
from.

* Refactor mempool test code to improve clarity

It was previously not clear that only the first genesis transaction was
being used. The remaining transactions in the genesis block were
discarded and then readded later.

* Replace `oneshot` with `call`

Remove a redundant internal `ready_and` call.

* Return an `impl Iterator` instead of a `Vec<_>`

Remove unnecessary deserializations and heap allocations.

* Refactor `mempool_storage_basic_for_network` test

Make the separation between the transactions expected to be in the
mempool and those expected to be rejected clearer.

* Replace `Iterator` with `DoubleEndedIterator`

Allow getting the last transaction more easily.

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-09-23 13:54:14 +00:00
Alfredo Garcia 56636c85fc
Add missing tests for mempool inbound requests (#2769)
* Use `MockService` in inbound test

Refactor the `mempool_requsets_for_transactions` test so that it uses a
`MockService` instead of the `mock_peer_set` function.

* Use `MockService` in the basic mempool test

Refactor the `mempool_service_basic` test so that it uses a
`MockService` instead of the `mock_peer_set` helper function.

* Remove the `mock_peer_set` helper function

It is not used anymore, since the usages were replaced with
`MockService`s.

* add tests for mempool inbound requests

* Use MockService for transaction verifier

* Refactor creation of mock `peer_set`

Use the same style as the mock transaction verifier.

* Derive `Eq` for `zebra_network::Request`

Make it easy to use the `MockService::expect_request` method.

* Return mocked peer set service from `setup`

Allow it to be used to respond to requests.

* Add bindings for the transaction used for testing

Allow them to be moved into futures later.

* Respond to transaction download request

Make sure that the test transaction appears to the mempool as if it had
been downloaded by the peer set service.

* Assert that no unexpected requests were received

Check that the mempool doesn't send unexpected requests to the peer set
service.

* add tests for mempool inbound requests

* Use MockService for transaction verifier

* add missing `expect_no_requests` to `mempool_advertise_transaction_ids` test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-09-23 10:17:06 -03:00
Janito Vaqueiro Ferreira Filho 1f1bf2ec4d
Replace `mock_peer_set` function with `MockService` (#2790)
* Use `MockService` in inbound test

Refactor the `mempool_requsets_for_transactions` test so that it uses a
`MockService` instead of the `mock_peer_set` function.

* Use `MockService` in the basic mempool test

Refactor the `mempool_service_basic` test so that it uses a
`MockService` instead of the `mock_peer_set` helper function.

* Remove the `mock_peer_set` helper function

It is not used anymore, since the usages were replaced with
`MockService`s.

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-09-22 15:44:13 +00:00
Marek 061ad55144
Sneak chain_tip_change into mempool (#2785)
* Pass ChainTipChange to the mempool

* Fix nits
2021-09-21 17:06:52 +00:00
Conrado Gouvea 957e12e4ca
Pass sync_status to mempool (#2754)
* Pass sync_status to mempool

* Update zebrad/src/components/mempool.rs

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Remove enabled flag for now; will be handled in #2723

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-09-15 22:13:29 +00:00
Conrado Gouvea 8825a52bb8
Move transaction download and verify stream into the mempool service (#2741)
* Move transaction dowloader and verifier into the mempool service

* add test for `Storage::contains_rejected()`

* Rename DownloadAndVerify->Queue; move should_download_or_verify() to previous impl

* GossipedTx -> Gossip

* Revamp error handling

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-09-13 16:28:07 -04:00
Conrado Gouvea a2993e8df0
Skip download and verification if the transaction is already in the mempool or state (#2718)
* Check if tx already exists in mempool or state before downloading

* Reorder checks

* Add rejected test; refactor into separate function

* Wrap mempool in buffered service

* Rename RejectedTransactionsById -> RejectedTransactionsIds

* Add RejectedTransactionIds response; fix request name

* Organize imports

* add a test for Storage::rejected_transactions

* add test for mempool `Request::RejectedTransactionIds`

* change buffer size to 1 in the test

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-09-08 18:51:17 +00:00
Alfredo Garcia 65e308d2e1
Respond to inbound `TransactionsById` with mempool content (#2725)
* reply to inbound `TransactionsById` requests

* apply style/redability suggestions and fix typo

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-09-06 22:55:17 +00:00
Alfredo Garcia 9c220afdc8
Reply to `Request::MempoolTransactionIds` with mempool content (#2720)
* reply to `Request::MempoolTransactionIds`

* remove boilerplate

* get storage from mempool with a method

* change panic message

* try fix for mac

* use normal init instead of init_tests for state service

* newline

* rustfmt

* fix test build
2021-09-02 13:42:31 +00:00
Conrado Gouvea 1ccb2de7c7
Add transaction downloader and verifier (#2679)
* Add transaction downloader

* Changed mempool downloader to be like inbound

* Verifier working (logs result)

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Fix coinbase check for mempool, improve is_coinbase() docs

* Change other downloads.rs docs to reflect the mempool downloads.rs changes

* Change TIMEOUTs to downloads.rs; add docs

* Renamed is_coinbase() to has_valid_coinbase_transaction_inputs() and contains_coinbase_input() to has_any_coinbase_inputs(); reorder checks

* Validate network upgrade for V4 transactions; check before computing sighash (for V5 too)

* Add block_ prefix to downloads and verifier

* Update zebra-consensus/src/transaction.rs

Co-authored-by: teor <teor@riseup.net>

* Add consensus doc; add more Block prefixes

Co-authored-by: teor <teor@riseup.net>
2021-09-02 00:06:20 +00:00
teor 74e155ff9f
Spelling: gossipped -> gossiped (#2119) 2021-05-07 13:01:11 +02:00
teor 306fa88214 Document the correctness of Poll::Pending wakeups 2021-03-27 08:55:49 -04:00
teor 829a6f11c5 Document the behaviour of the `select!` macro 2021-03-27 08:55:49 -04:00
teor c6358b157c Reduce inbound concurrency to limit memory usage
Inbound malicious blocks can use a large amount of RAM when
deserialized. Limit inbound concurrency, so that the total amount
of RAM remains small.
2021-03-11 06:24:46 -05:00
teor 0b76352468
Document a state_contains bug (#1715)
* Document a state_contains bug in the syncer and Inbound
2021-02-10 09:05:14 +10:00
teor 21b0360114 Limit concurrent inbound gossipped block requests
Uses the "load shed directly" design pattern from #1618.
2021-01-29 11:02:26 +10:00
teor 92d95d4be5 Refactor inbound members into a consistent order
And add download comments
2021-01-13 20:46:25 -05:00
Henry de Valence fa02b266ca clippy 2020-11-25 10:55:44 -08:00
Henry de Valence de8415dcb1 tidy spans 2020-11-25 10:55:44 -08:00
Henry de Valence 77bf327b07 fix errors (2) 2020-11-25 10:55:44 -08:00
Henry de Valence 527f4d39ed fix errors 2020-11-25 10:55:44 -08:00
Henry de Valence e645e3bf0c remove async 2020-11-25 10:55:44 -08:00
Alfredo Garcia 486e55104a create Downloads for Inbound 2020-11-25 10:55:44 -08:00