Go to file
teor 343c5e68d4
change(state): Write finalized blocks to the state in a separate thread, to avoid network and RPC hangs (#5134)
* Add a new block commit task and channels, that don't do anything yet

* Add last_block_hash_sent to the state service, to avoid database accesses

* Update last_block_hash_sent regardless of commit errors

* Rename a field to StateService.max_queued_finalized_height

* Commit finalized blocks to the state in a separate task

* Check for panics in the block write task

* Wait for the block commit task in tests, and check for errors

* Always run a proptest that sleeps once

* Add extra debugging to state shutdowns

* Work around a RocksDB shutdown bug

* Close the finalized block channel when we're finished with it

* Only reset state queue once per error

* Update some TODOs

* Add a module doc comment

* Drop channels and check for closed channels in the block commit task

* Close state channels and tasks on drop

* Remove some duplicate fields across StateService and ReadStateService

* Try tweaking the shutdown steps

* Update and clarify some comments

* Clarify another comment

* Don't try to cancel RocksDB background work on drop

* Fix up some comments

* Remove some duplicate code

* Remove redundant workarounds for shutdown issues

* Remode a redundant channel close in the block commit task

* Remove a mistaken `!force` shutdown condition

* Remove duplicate force-shutdown code and explain it better

* Improve RPC error logging

* Wait for chain tip updates in the RPC tests

* Wait 2 seconds for chain tip updates before skipping them

* Remove an unnecessary block_in_place()

* Fix some test error messages that were changed by earlier fixes

* Expand some comments, fix typos

Co-authored-by: Marek <mail@marek.onl>

* Actually drop children of failed blocks

* Explain why we drop descendants of failed blocks

* Clarify a comment

* Wait for chain tip updates in a failing test on macOS

* Clean duplicate finalized blocks when the non-finalized state activates

* Send an error when receiving a duplicate finalized block

* Update checkpoint block behaviour, document its consensus rule

* Wait for chain tip changes in inbound_block_height_lookahead_limit test

* Wait for the genesis block to commit in the fake peer set mempool tests

* Disable unreliable mempool verification check in the send transaction test

* Appease rustfmt

* Use clear_finalized_block_queue() everywhere that blocks are dropped

* Document how Finalized and NonFinalized clones are different

* Use the same check as commit_finalized() for finalized block heights

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Marek <mail@marek.onl>
2022-09-28 16:09:56 +00:00
.cargo lint(clippy): add `unwrap_in_result` lint (#4667) 2022-06-28 06:22:07 +00:00
.github Use the best cache for docker images (#5282) 2022-09-28 13:14:29 +00:00
book List temporary CI errors that should be retried (#5281) 2022-09-28 13:14:39 +00:00
docker fix(ci): Build zebrad with Rust 1.63 to avoid Zcash parameter download hangs (#5251) 2022-09-26 12:37:23 +00:00
grafana Update transaction verification Grafana dashboard to show all shielded pool sigs, proofs, nullifiers (#4585) 2022-06-09 15:08:57 +00:00
tower-batch build(deps): bump tracing from 0.1.35 to 0.1.36 (#5273) 2022-09-28 03:17:49 +00:00
tower-fallback build(deps): bump tracing from 0.1.35 to 0.1.36 (#5273) 2022-09-28 03:17:49 +00:00
zebra-chain build(deps): bump rand_core from 0.6.3 to 0.6.4 (#5175) 2022-09-28 06:04:08 +00:00
zebra-client Repoint zebra image links to our new zfnd.org site for now (#3949) 2022-03-27 23:42:47 +00:00
zebra-consensus build(deps): bump tracing from 0.1.35 to 0.1.36 (#5273) 2022-09-28 03:17:49 +00:00
zebra-network build(deps): bump tracing from 0.1.35 to 0.1.36 (#5273) 2022-09-28 03:17:49 +00:00
zebra-node-services chore: Release Zebra 1.0.0-beta.15 (#5194) 2022-09-19 21:21:27 +00:00
zebra-rpc change(state): Write finalized blocks to the state in a separate thread, to avoid network and RPC hangs (#5134) 2022-09-28 16:09:56 +00:00
zebra-script chore: Release Zebra 1.0.0-beta.15 (#5194) 2022-09-19 21:21:27 +00:00
zebra-state change(state): Write finalized blocks to the state in a separate thread, to avoid network and RPC hangs (#5134) 2022-09-28 16:09:56 +00:00
zebra-test build(deps): bump tracing from 0.1.35 to 0.1.36 (#5273) 2022-09-28 03:17:49 +00:00
zebra-utils build(deps): bump color-eyre from 0.6.1 to 0.6.2 (#5220) 2022-09-26 22:23:59 +00:00
zebrad change(state): Write finalized blocks to the state in a separate thread, to avoid network and RPC hangs (#5134) 2022-09-28 16:09:56 +00:00
.codespellrc feat(ci): add a codespell linting action (#4482) 2022-05-26 00:27:44 +00:00
.dockerignore fix(build): avoid docker cache contamination and invalidation (#4254) 2022-05-03 10:13:28 -04:00
.gitignore fix: improve file and directories to be ignored by git and Docker (#3399) 2022-01-26 06:22:51 +00:00
CHANGELOG.md chore: Release Zebra 1.0.0-beta.15 (#5194) 2022-09-19 21:21:27 +00:00
CODE_OF_CONDUCT.md CODE_OF_CONDUCT.md (#1097) 2021-03-25 10:54:08 +01:00
CONTRIBUTING.md docs: fix typo (#3877) 2022-03-22 22:01:35 -04:00
Cargo.lock build(deps): bump rand_core from 0.6.3 to 0.6.4 (#5175) 2022-09-28 06:04:08 +00:00
Cargo.toml change(deps): upgrade zcash_script and zcash dependencies (#4926) 2022-09-02 07:24:29 +00:00
LICENSE-APACHE Add copyright marks on each license 2019-11-14 11:50:49 -08:00
LICENSE-MIT Add copyright marks on each license 2019-11-14 11:50:49 -08:00
README.md chore: Release Zebra 1.0.0-beta.15 (#5194) 2022-09-19 21:21:27 +00:00
SECURITY.md Explicitly allow unencrypted disclosures for alpha releases (#2127) 2021-05-11 14:41:33 +02:00
clippy.toml Apply clippy fixes 2020-02-05 12:42:32 -08:00
codecov.yml Re-enable code coverage comments on PRs (#3246) 2021-12-22 00:52:08 +00:00
deny.toml fix(doc): Update deny.toml developer docs and file comments (#5151) 2022-09-14 19:52:32 +00:00
firebase.json Configure redirect for firebase hosting 2020-01-16 18:38:16 -05:00
katex-header.html Add KaTeX to rendered docs. (#832) 2020-08-05 17:34:30 -07:00
prometheus.yaml Tell Prometheus to scrape more aggressively 2020-02-14 20:14:05 -05:00

README.md

Zebra logotype


CI Docker CI OSes Continuous Delivery Coverage codecov Build docs Build lightwalletd Build Zcash Params

License

Contents

About

Zebra is the Zcash Foundation's independent, consensus-compatible implementation of a Zcash node, currently under development. It can be used to join the Zcash peer-to-peer network, which helps keeping Zcash working by validating and broadcasting transactions, and maintaining the Zcash blockchain state in a distributed manner.

Zcash is a cryptocurrency designed to preserve the user's privacy. If you just want to send and receive Zcash then you don't need to use Zebra directly. You can download a Zcash wallet application which will handle that for you.

Please join us on Discord if you'd like to find out more or get involved!

Using Zebra

You would want to run Zebra if you want to contribute to the Zcash network: the more nodes are run, the more reliable the network will be in terms of speed and resistance to denial of service attacks, for example.

Zebra aims to be faster, more secure, and more easily extensible than other Zcash implementations.

Beta Releases

Every few weeks, we release a new Zebra beta release.

Zebra's network stack is interoperable with zcashd, and Zebra implements all the features required to reach Zcash network consensus.

Currently, Zebra validates all of the Zcash consensus rules for the NU5 network upgrade.

But it may not validate any:

  • Undocumented rules derived from Bitcoin
  • Undocumented network protocol requirements

Getting Started

Building zebrad requires Rust, libclang, and a C++ compiler.

Build and Run Instructions

zebrad is still under development, so there is no supported packaging or install mechanism. To run zebrad, follow the instructions to compile zebrad for your platform:

  1. Install cargo and rustc.
    • Zebra is tested with the latest stable Rust version. Earlier versions are not supported or tested. Any Zebra release can remove support for older Rust versions, without any notice. (Rust 1.59 and earlier are definitely not supported, due to missing features.)
  2. Install Zebra's build dependencies:
    • libclang: the libclang, libclang-dev, llvm, or llvm-dev packages, depending on your package manager
    • clang or another C++ compiler: g++, Xcode, or MSVC
  3. Run cargo install --locked --git https://github.com/ZcashFoundation/zebra --tag v1.0.0-beta.15 zebrad
  4. Run zebrad start (see Running Zebra for more information)

For more detailed instructions, refer to the documentation.

Configuring JSON-RPC for lightwalletd

To use zebrad as a lightwalletd backend, give it this ~/.config/zebrad.toml:

[rpc]
# listen for RPC queries on localhost
listen_addr = '127.0.0.1:8232'

# automatically use multiple CPU threads
parallel_cpu_threads = 0

WARNING: This config allows multiple Zebra instances to share the same RPC port. See the RPC config documentation for details.

Optional Features

For performance reasons, some debugging and monitoring features are disabled in release builds.

You can enable these features using:

cargo install --features=<name> ...

System Requirements

The recommended requirements for compiling and running zebrad are:

  • 4 CPU cores
  • 16 GB RAM
  • 300 GB available disk space for building binaries and storing cached chain state
  • 100 Mbps network connection, with 300 GB of uploads and downloads per month

We continuously test that our builds and tests pass on:

The latest GitHub Runners for:

  • macOS
  • Ubuntu

Docker:

  • Debian Bullseye

Zebra's tests can take over an hour, depending on your machine. We're working on making them faster.

zebrad might build and run fine on smaller and slower systems - we haven't tested its exact limits yet.

For more detailed requirements, refer to the documentation.

Memory Troubleshooting

If Zebra's build runs out of RAM, try setting: export CARGO_BUILD_JOBS=2

If Zebra's tests timeout or run out of RAM, try running: cargo test -- --test-threads=2

(cargo uses all the processor cores on your machine by default.)

macOS Test Troubleshooting

Some of Zebra's tests deliberately cause errors that make Zebra panic. macOS records these panics as crash reports.

If you are seeing "Crash Reporter" dialogs during Zebra tests, you can disable them using this Terminal.app command:

defaults write com.apple.CrashReporter DialogType none

Network Ports and Data Usage

By default, Zebra uses the following inbound TCP listener ports:

  • 8233 on Mainnet
  • 18233 on Testnet

Zebra needs some peers which have a round-trip latency of 2 seconds or less. If this is a problem for you, please open a ticket.

zebrad's typical mainnet network usage is:

  • Initial sync: 100 GB download, we expect the initial download to grow to hundreds of gigabytes over time
  • Ongoing updates: 10 MB - 10 GB upload and download per day, depending on user-created transaction size and peer requests

Zebra performs an initial sync every time its internal database version changes, so some version upgrades might require a full download of the whole chain.

For more detailed information, refer to the documentation.

Network Troubleshooting

Some of Zebra's tests download Zcash blocks, so they might be unreliable depending on your network connection. You can set ZEBRA_SKIP_NETWORK_TESTS=1 to skip the network tests.

Zebra may be unreliable on Testnet, and under less-than-perfect network conditions. See our roadmap for details.

Disk Usage

Zebra uses around 100 GB of space for cached mainnet data, and 10 GB of space for cached testnet data. We expect disk usage to grow over time, so we recommend reserving at least 300 GB for mainnet nodes.

Zebra's database cleans up outdated data periodically, and when Zebra is shut down and restarted.

Disk Troubleshooting

Zebra's state commits changes using RocksDB database transactions.

If you forcibly terminate Zebra, or it panics, any incomplete changes will be rolled back the next time it starts.

So Zebra's state should always be valid, unless your OS or disk hardware is corrupting data.

Known Issues

There are a few bugs in Zebra that we're still working on fixing:

  • No Windows support #3801
    • We used to test with Windows Server 2019, but not anymore; see issue for details

Future Work

Performance and Reliability:

  • Reliable syncing under poor network conditions
  • Additional batch verification
  • Performance tuning

Currently, the following features are out of scope:

  • Optional Zcash network protocol messages
  • Consensus rules removed before Canopy activation (Zebra checkpoints on Canopy activation)

Documentation

The Zebra website contains user documentation, such as how to run or configure Zebra, set up metrics integrations, etc., as well as developer documentation, such as design documents. We also render API documentation for the external API of our crates, as well as internal documentation for private APIs.

Security

Zebra has a responsible disclosure policy, which we encourage security researchers to follow.

License

Zebra is distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT.