Commit Graph

3 Commits

Author SHA1 Message Date
teor cf4b2f7a67
feat(verify): Concurrently verify proof and signature batches (#4776)
* Initialize the rayon threadpool with a new config for CPU-bound threads

* Verify proofs and signatures on the rayon thread pool

* Only spawn one concurrent batch per verifier, for now

* Allow tower-batch to queue multiple batches

* Fix up a potentially incorrect comment

* Rename some variables for concurrent batches

* Spawn multiple batches concurrently, without any limits

* Simplify batch worker loop using OptionFuture

* Clear pending batches once they finish

* Stop accepting new items when we're at the concurrent batch limit

* Fail queued requests on drop

* Move pending_items and the batch timer into the worker struct

* Add worker fields to batch trace logs

* Run docker tests on PR series

* During full verification, process 20 blocks concurrently

* Remove an outdated comment about yielding to other tasks
2022-07-18 08:43:29 +10:00
teor 9b9cd55097
fix(batch): Improve batch verifier async, correctness, and performance (#4750)
* Use a new channel for each batch

* Prefer the batch timer if there are also new batch requests

* Allow other tasks to run after each batch

* Label each batch worker with the verifier's type

* Rename Handle to ErrorHandle, and fix up some docs

* Check batch worker tasks for panics and task termination

* Use tokio's PollSemaphore instead of an outdated Semaphore impl

* Run all verifier cryptography on a blocking thread

Also use a new verifier channel for each batch.

* Make flush and drop behaviour consistent for all verifiers

* Partly fix an incorrect NU5 test

* Switch batch tests to the multi-threaded runtime

* Export all verifier primitive modules from zebra-consensus

* Remove outdated test code in tower-batch

* Use a watch channel to send batch verifier results

* Use spawn_blocking for batch fallback verifiers

* Spawn cryptography batches onto blocking tokio threads

* Use smaller batches for halo2

* Minor tower-batch cleanups

* Fix doc link in zebra-test

* Drop previous permit before acquiring another to avoid a deadlock edge case
2022-07-18 08:41:18 +10:00
Deirdre Connolly ff299781c1
Merge pull request #2318 from ZcashFoundation/redpallas-verifier
RedPallas async verifier service
2021-07-02 02:50:23 -04:00