We modeled a Bitcoin `headers` message as being a list of block headers.
However, the actual data structure is slightly different: it's a list of (block
header, transaction count) pairs. This caused zcashd to reject our headers
messages.
To fix this, introduce a new `CountedHeader` struct with a `block::Header` and
transaction count `usize`, then thread it through the inbound service and the
state.
I tested this locally by running Zebra with these changes and inspecting a
trace-level log of the span of a peer connection that requested a nontrivial
headers packet from us, and verified that it did not reject our message.
The `CoinbaseData` parses the block height separately from the rest of the
free-form coinbase data. However, it had two bugs:
1. It did not require that the height was canonically encoded;
2. Its canonical encoding was incorrect relative to the BIP34-inherited encoding.
This meant that we computed some transaction hashes incorrectly, because when
we re-serialized the coinbase transaction, we would canonically serialize the
coinbase transaction (using the incorrect definition of canonical, bug 2). And
we didn't notice that the wrong definition of canonical encoding was being used
because we accepted what we thought were non-canonically encoded heights.
The relevant rules are here: 877212414a/src/script/script.h (L307-L346)
This commit changes the encoding to reject non-canonically encoded heights, and
to match the correct encoding rules. We check that at least one
non-canonically encoded height is correctly rejected using a new test vector.
The database format increments because we saved a bunch of wrongly encoded blocks.
This discrepancy was originally noticed by @teor2345, who pointed out that a
previous version of the block 202 test vector (now preserved as "bad block
202") did not match the block from zcashd.
* Implement Expanded to Compact Difficulty
* Implement Arbitrary for CompactDifficulty
Remove the derive, and generate values from random block
hashes.
* Implement Arbitrary for ExpandedDifficulty and Work
* Use Arbitrary for CompactDifficulty in Arbitrary for Block
* Test difficulty on all block test vectors
And cleanup some duplicate test code
* Round-trip tests for compact test cases
* Round-trip tests for compact difficulty in block test vectors
* Make Add for Work return PartialCumulativeWork
Remove AddAssign for Work
Rewrite a proptest using Sub for PartialCumulativeWork
Use Arbitrary for Work
* Add roundtrip work sum tests
* Add roundtrip comparison difficulty tests
* Add failing proptest cases due to test bugs
* Use Some(_) rather than _.into()
* Reduce visibility of difficulty type inner values
* Split work and other difficulty proptests
This change makes sure that rejected work values don't disable property
tests on other types.
* Reverse displayed endianness of transaction and block hashes
* fix zebra-checkpoints utility for new hash order
* Stop using "zebrad revhex" in zebrad-hash-lookup
* Rebuild checkpoint lists in new hash order
This change also adds additional checkpoints to the end of each list.
* Replace TransactionHash with transaction::Hash
This change should have been made in #905, but we missed Debug impls
and some docs.
Co-authored-by: Ramana Venkata <vramana@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
* add general and founders reward subsidy modules
* validate founders reward
* Use funding streams after Canopy on testnet
ZIP-1014 only applies to mainnet, where Canopy is at the first halving.
On testnet, Canopy is before the first halving, and the dev fund rules
apply from Canopy. (See ZIP-214.)
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* pass all test vectors through current subsidy validation
* Add testnet and halving subsidy tests
* add subsidy validation error tests
* rename block validation methods
* add network to block verifier
* add amount operators
* Implement Ord, Eq, and Hash for Amount
* Implement Add<Height> for Height
And make the existing Height operators do range checks.
* Apply operator suggestions
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* implement most of the chain functions
* implement fork
* fix outpoint handling in Chain struct
* update expect for work
* split utxo into two sets
* update the Chain definition
* remove allow attribute in zebra-state/lib.rs
* merge ChainSet type into MemoryState
* Add error messages to asserts
* export proptest impls for use in downstream crates
* add testjob for disabled feature in zebra-chain
* try to fix github actions syntax
* add module doc comment
* update RFC for utxos
* add missing header
* working proptest for Chain
* propagate back results over channel
* Start updating RFC to match changes
* implement queued block pruning
* and now it syncs wooo!
* remove empty modules
* setup config for proptests
* re-enable missing_docs lint
* update RFC to match changes in impl
* add documentation
* use more explicit variable names
* Begin work on RFC5 implementation
* I think this is necessary
* holy shit supertrait implemented via subtrait
* implement most of the chain functions
* change to slightly better name
* implement fork
* fix outpoint handling in Chain struct
* update expect for work
* resolve review comment
* split utxo into two sets
* update the Chain definition
* just a little more
* update comment
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* apply changes from code review
* remove allow attribute in zebra-state/lib.rs
* Update zebra-state/src/memory_state.rs
Co-authored-by: teor <teor@riseup.net>
* merge ChainSet type into MemoryState
* rename state impl types
* Add error messages to asserts
* checkpoint so I can split off arbitrary changes into a PR
* export proptest impls for use in downstream crates
* add testjob for disabled feature in zebra-chain
* run rustfmt
* try to fix github actions syntax
* differentiate name
* prove that github action tests zebra-chain build without features
* revert change from last commit now that test is running
* remove accidentally introduced newline
* checkpoint
* add module doc comment
* update RFC for utxos
* add missing header
* working proptest for Chain
* apply change from chain impl PR
* setup config for proptests
* Update zebra-chain/src/block/arbitrary.rs
Co-authored-by: teor <teor@riseup.net>
* run rustfmt
Co-authored-by: teor <teor@riseup.net>
To avoid confusion with TESTNET_BLOCKS.
Automated search and replace, made using this script:
sed -i 's/TEST_BLOCKS/BLOCKS/' \
$(grep -r TEST_BLOCKS zebra* | cut -d: -f1 | sort -u)
* Move block test vectors into separate files
* Refactor block test vectors into their own module
* Remove some duplicate test vector data
* Add mainnet block test vectors for each network upgrade
* Add testnet block test vectors for each network upgrade
* Update the TEST_BLOCKS list
* Add basic tests using TEST_BLOCKS
* Ignore shell.nix
* export proptest impls for use in downstream crates
* add testjob for disabled feature in zebra-chain
* run rustfmt
* try to fix github actions syntax
* differentiate name
* prove that github action tests zebra-chain build without features
* revert change from last commit now that test is running
* remove accidentally introduced newline
* Update .github/workflows/ci.yml
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
This is the first in a sequence of changes that change the block:: items
to not include Block as a prefix in their name, in accordance with the
Rust API guidelines.
This Merkle tree is the SHA256d one used only for including transactions
in a block, so it should be kept there in order to not be confused with
other Merkle trees (like the note commitment trees).
It's not accurate to call it a LightClientRootHash, because it's not
always a root has for a light client -- sometimes it's a different kind
of root hash.
This moves the tests::generate module into the block tests. Because
this whole set of changes is just focused on reorganization, the
generate code is unchanged, but in the future, the code should be
rewritten as a collection of proptest strategies.
This extracts the `difficulty` module from `block` and the
`equihash_solution` module from the crate root. The PoW calculations
are significantly more complicated than the other block code and pretty
dissimilar from it, so it makes more sense to create a common proof of
work module.
The `EquihashSolution` and `EQUIHASH_SOLUTION_SIZE` are renamed to
`equihash::Solution` and `equihash::SOLUTION_SIZE` and imported that
way, except in `block/header.rs`, to avoid a conflict with the
`equihash` crate. In the future it would be better to encapsulate the
equihash solution check into the `equihash::Solution` type so that
callers only need to import our `work::equihash`.
The test organization leaves a little to be desired but I think that
this can be improved as we fill out the proof of work implementation.
This extracts the SHA256d code from being split across two modules and puts it
in one module, under serialization.
The code is unchanged except for three deleted tests:
* `sha256d_flush` in `sha256d_writer` (not a meaningful test);
* `transactionhash_debug` (constructs an invalid transaction hash, and the
behavior is tested in the next test);
* `decode_state_debug` (we do not need to test the Debug output of
DecodeState);
* feature: Implement CompactDifficulty to Work
* fix: Test Work on all difficulty test cases
* fix: Add Bitcoin test vectors for difficulty
* feature: Cumulative Work
* impl Add, AddAssign for Work
* derive Default for Work
* test cumulative work
* fix: comments
* fix: More comments
* Fix comment links
* Remove unused import
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Henry de Valence
teor
Alfredo Garcia
Jane Lusby
Ramana Venkata
Deirdre Connolly