Commit Graph

117 Commits

Author SHA1 Message Date
likho 9024519f28 WIP: Remove transparent transactions 2024-04-22 17:52:12 +02:00
idky137 4579722833
change(chain): Remove `Copy` trait impl from `Network` (#8354)
* removed derive copy from network, address and ledgerstate

* changed is_max_block_time_enforced to accept ref

* changed NetworkUpgrade::Current to accept ref

* changed NetworkUpgrade::Next to accept ref

* changed NetworkUpgrade::IsActivationHeight to accept ref

* changed NetworkUpgrade::TargetSpacingForHeight to accept ref

* changed NetworkUpgrade::TargetSpacings to accept ref

* changed NetworkUpgrade::MinimumDifficultySpacing_forHeight to accept ref

* changed NetworkUpgrade::IsTestnetMinDifficultyBlock to accept ref

* changed NetworkUpgrade::AveragingWindowTimespanForHeight to accept ref

* changed NetworkUpgrade::ActivationHeight to accept ref

* changed sapling_activation_height to accept ref

* fixed lifetime for target_spacings

* fixed sapling_activation_height

* changed transaction_to_fake_v5 and fake_v5_transactions_for_network to accept ref to network

* changed Input::vec_strategy to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_history.rs to accept ref to network

* changed functions in zebra-chain/src/history_tree.rs to accept ref to network

* changed functions in zebra-chain/src/history_tree.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/address.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/viewing_key* to accept ref to network

* changed functions in zebra-chain/src/transparent/address.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_primitives.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_note_encryption.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/history_tree* to accept ref to network

* changed functions in zebra-chain/src/block* to accept ref to network

* fixed errors in zebra-chain::parameters::network

* fixed errors in zebra-chain::parameters::network

* fixed errors in zebra-chain

* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref

* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref

* fixed errors in zebra-chain/src/block/arbitrary.rs

* finished fixing errors in zebra-chain - all crate tests pass

* changed functions in zebra-state::service::finalized_state to accept &Network

* changed functions in zebra-state::service::non_finalized_state to accept &Network

* zebra-state tests run but fail with overflow error

* zebra-state tests all pass

* converted zebra-network -- all crate tests pass

* applied all requested changes from review

* converted zebra-consensus -- all crate tests pass

* converted zebra-scan -- all crate tests pass

* converted zebra-rpc -- all crate tests pass

* converted zebra-grpc -- all crate tests pass

* converted zebrad -- all crate tests pass

* applied all requested changes from review

* fixed all clippy errors

* fixed build error in zebrad/src/components/mempool/crawler.rs
2024-03-19 20:45:27 +00:00
teor 7e7f989545
Fix new nightly clippy and rustc lints (#7860) 2023-10-27 19:35:57 +00:00
teor cc61bd50b9
change(state): Add note subtree indexes for new and existing blocks (#7437)
* Copy the add_subtrees upgrade from the original branch

* Copy the database write changes in shielded.rs from the original branch

* Copy the tree API changes from the original branch

* Simplify subtree APIs to avoid exposing frontiers

* Fix a dead code warning by re-using existing methods

* Use mpsc::Receiver<CancelFormatChange> in the subtree upgrade

* Run the subtree upgrade on startup

* Bump the database format version to 25.2.0

* Fix a confusing 'upgrade complete' log

* Clarify some comments and error messages

* Simplify prev_tree unwrap to avoid an (impossible?) concurrency bug

* Use separate subtree writing functions

* Use common note commitment list code

* Fix subtree completion condition and add asserts

* Simplify subtree API and avoid exposing Address

* Fix API compatibility when Arcs are removed

* Log when each subtree is added

* If a format change is cancelled, don't mark the database as upgraded or do format checks

* Log subtree progress about once every two minutes

* Adds a state validity check for subtrees upgrade

* Orchard is faster, decrease log interval

* Clarify subtree index docs

* Move a log to the correct location

* Refactor subtree upgrade to remove duplicate inverted loop conditions

* updates subtree state validity check

* Add a subtree format check when there is no upgrade

* Fix an off-by-one error with the final subtree check

* Use error-level logs for database format checks

* Skip format checks in tests that create invalid formats

* fix state validity test

* Add a concurrency comment to subtree by height methods

* Add individual subtree state methods: reverts removing these methods in an earlier PR

* fastmod "subtrees_by_index" "subtree_list_by_index_for_rpc"

---------

Co-authored-by: arya2 <aryasolhi@gmail.com>
2023-09-05 16:52:06 +00:00
Alfredo Garcia 17bd7884ea
fix(build): add elasticsearch feature to block serialize (#6709)
* add elasticsearch feature to block serialize

* fix for elastic feature

* add zebra-chain elasticsearch dep to zebrad
2023-06-14 19:01:39 +00:00
teor d3ce022ecc
change(utils): Add a direct connection mode to zebra-checkpoints (#6516)
* Rename variables so it's clearer what they do

* Fully document zebra-checkpoints arguments

* Remove some outdated references to zcashd

* Add a json-conversion feature for converting JSON to valid Heights

* Simplify zebra-checkpoints code using conversion methods

* Track the last checkpoint height rather than the height gap

* Move all the CLI-specific code into a single function

* Remove cfg(feature) from the test RPC client API

* Move the RpcRequestClient into zebra-node-services

* Fix the case of RpcRequestClient

* Add transport and addr arguments to zebra-checkpoints

* Make zebra-checkpoints support both CLI and direct JSON-RPC connections

* Fix RpcRequestClient compilation

* Add a suggestion for zcashd authentication failures

* Document required features

* Handle differences in CLI & direct parameter and response formats

* Replace a custom function with an existing dependency function

* Add a checkpoint list test for minimum height gaps
2023-04-26 23:35:53 +00:00
Marek 2a48d4cf25
change(chain): Refactor the handling of height differences (#6330)
* Unify the `impl`s of `Sub` and `Add` for `Height`

* Adjust tests for `Height` subtraction

* Use `Height` instead of `i32`

* Use `block:Height` in RPC tests

* Use `let .. else` statement

Co-authored-by: Arya <aryasolhi@gmail.com>

* Update zebra-consensus/src/block/subsidy/general.rs

* Refactor the handling of height differences

* Remove a redundant comment

* Update zebrad/src/components/sync/progress.rs

Co-authored-by: Arya <aryasolhi@gmail.com>

* Update progress.rs

* impl TryFrom<u32> for Height

* Make some test assertions clearer

* Refactor estimate_up_to()

* Restore a comment that was accidentally removed

* Document when estimate_distance_to_network_chain_tip() returns None

* Change HeightDiff to i64 and make Height.sub(Height) return HeightDiff (no Option)

* Update chain tip estimates for HeightDiff i64

* Update subsidy for HeightDiff i64

* Fix some height calculation test edge cases

* Fix the funding stream interval calculation

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2023-03-29 23:06:31 +00:00
teor 8c451968ee
fix(diagnostics): Hex-encode debug format of commitments, nonces, and nullifiers (#5960)
* Format commitments and nonces as hex

* Format Sprout and Sapling nullifiers as hex when debugging

* Format Sprout commitments as hex when debugging

* Format redpallas keys as hex when debugging

* Update code that we're going to delete in the next PR anyway
2023-01-17 13:57:22 +00:00
teor 26ff1b4fb0
fix(rust): Make Zebra build with the latest nightly Rust (#5738)
* Remove an unused async track_caller which will soon become a warning

* Explicitly drop unused futures

* Work around a compiler panic (ICE) with flat_map()

https://github.com/rust-lang/rust/issues/105044

* Remove a redundant into_iter()

* allow(clippy::needless_collect)
2022-11-30 07:03:24 +00:00
Alfredo Garcia 4ccd0741b6
feature(rpc): add fixed values to getblocktemplate response (#5558)
* add fixed values to getblocktemplate rpc call response

* suggestion: Avoid new uses of lazy_static (#5559)

* Avoid using lazy_static

* Add some missing documentation

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* minor fixes

* move docs to struct

* add fixed values to coinbase tx

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2022-11-08 22:55:11 +00:00
teor cbb3232769
Only fetch block headers from the database to answer headers requests (#4792) 2022-07-22 09:15:22 +10:00
teor 42ef8846b3
fix(clippy): resolve or disable new nightly clippy lints (#4739)
* Fix new dead_code lints in generated pin-project code

* Fix and ignore new needless_borrow lints
2022-07-05 11:04:47 +10:00
Alfredo Garcia 97fb85dca9
lint(clippy): add `unwrap_in_result` lint (#4667)
* `unwrap_in_result` in zebra-chain crate

* `unwrap_in_result` in zebra-script crate

* `unwrap_in_result` in zebra-state crate

* `unwrap_in_result` in zebra-consensus crate

* `unwrap_in_result` in zebra-test crate

* `unwrap_in_result` in zebra-network crate

* `unwrap_in_result` in zebra-rpc crate

* `unwrap_in_result` in zebrad crate

* rustfmt

* revert `?` and add exceptions

* explain some panics better

* move some lint positions

* replace a panic with error

* Fix rustfmt?

Co-authored-by: teor <teor@riseup.net>
2022-06-28 06:22:07 +00:00
Marek b8b35f8da9
fix(doc): Fix various doc warnings, part 1 (#4514)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

* Fix some warnings produced by `cargo doc`

* Fix some rustdoc warnings

* Fix some warnings

* Refactor some changes

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-06-02 15:07:35 +00:00
Marek 6f896ef5a5
fix(doc): Fix the syntax of links in comments (#4494)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fab453e7d3559a5d474e0879183b9bfb.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620ae1a2140760ae7ea379deecb4b583c.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-05-30 20:12:11 +00:00
Marek 7c726b246d
feat(rpc): Implement `z_gettreestate` RPC (#3990)
* Impl the elementary structure of the `z_gettreestate` RPC

* Fix merging bugs

* Fix a merge bug

* Fix a merge bug

* Move a derive attribute

Co-authored-by: teor <teor@riseup.net>

* Clarify the support of negative heights

* Add Orchard note commitment trees to the response

* Add the time to the response

* Finalize the `z_gettreestate` RPC

* Add a note that verified blocks have coinbase height

* Refactor `from_str` for `HashOrHeight`

* Fix a mistake in the docs

Co-authored-by: teor <teor@riseup.net>

* Clarify request types

Co-authored-by: teor <teor@riseup.net>

* Simplify `hash_or_height` conversion to height

Co-authored-by: teor <teor@riseup.net>

* Add a TODO about optimization

Co-authored-by: teor <teor@riseup.net>

* Add a doc comment

* Make sure Sapling & Orchard trees don't get mixed up

* Serialize Sapling commitment trees

* Refactor some comments

* Serialize Orchard commitment trees

* Serialize block heights

* Simplify the serialization of commitment trees

* Remove the block time from the RPC response

* Simplify the serialization of block heights

* Put Sapling & Orchard requests together

* Remove a redundant TODO

* Add block times to the RPC response

* Derive `Clone, Debug, Eq, PartialEq` for `GetTreestate`

Co-authored-by: teor <teor@riseup.net>

* Derive `Clone`, `Debug`, `Eq` and `PartialEq` for `SerializedTree`

* Document the fields of `GetTreestate`

* Skip the serialization of empty trees

This ensures compatibility with `zcashd` in the `z_gettreestate` RPC.

* Document the `impl` of `merkle_tree::Hashable` for nodes

* Make the structure of the JSON response consistent with `zcashd`

* Derive `Eq` for nodes

Co-authored-by: teor <teor@riseup.net>

* Convert Sapling commitment trees to a format compatible with zcashd

* Refactor the conversion of Sapling commitment trees

* Refactor some comments

* Refactor comments

* Add a description of the conversion

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Fix comment indenting

* Document the conversion between the dense and sparse formats

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2022-05-12 07:00:12 +00:00
teor e49c1d7034
feat(state): add transparent address indexes to the non-finalized state (#4022)
* Derive Hash for transparent address index types

* Expose some types used by transparent address indexes

* Add an empty transparent transfers type for transparent address indexes

* Update TransparentTransfers with created UTXOs

* Add spent transparent outputs to ContextuallyValidBlock

* Update TransparentTransfers with spent transparent outputs

* Ignore missing spent outputs, so that tests pass

* Remove empty TransparentTransfers after a spend revert

* Update TransparentTransfers with creating and spending transaction IDs

* Ignore duplicate created UTXOs, so that tests pass

* Add some TODO comments

* Remove accidental doctest formatting

* Add address transfers index accessor methods

* Use TransactionLocation in the non-finalized state

* Apply more address index assertions to production code

* Refactor deeply nested code and apply more assertions

* Return UTXOs in chain order

* Return transaction hashes in chain order

* Stop indexing each transparent output multiple times

* Run some more asserts during tests

* Tidy TODO comments

* Fix an incorrect assert condition

* Use OrderedUtxos so that spent UTXOs can be stored in chain order

* Update tests to use OrderedUtxos

* Update the index API for the getaddressutxos query

* Remove redundant arguments in tests

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-12 17:21:46 +00:00
teor b4deca2912
6. add(test): add more tests for height and transaction location serialization (#3879)
* Add round-trip test for OutputLocation disk serialization

* Make the transaction snapshot tests more accurate

Previously, we were showing the genesis transaction hash at height 0, index 0.

But its hash is actually not stored by location in the database,
because the genesis transaction is skipped due to a consensus rule.

* Update the transaction snapshot data

* Add history tree snapshot tests

At the current test heights, the history trees are all empty.

* Add the history tree snapshot data

* Update comments

* Simplify snapshot test code

* Make some serde::Serialize impls test-only, so we can hex-encode them

This should also speed up release compilation a bit.

* Add snapshot test code for UTXOs

* Add snapshot data for UTXOs
2022-03-18 20:30:16 +00:00
Alfredo Garcia 833560411f
feature(rpc): implement getblock api call (#3707)
* feature(rpc): start adding a `getblock` method

* fix(rpc): replace oneshot

* fix(rpc): replace a panic with error

* fix(rpc): fix test

* feature(rpc): add hex to response

* refactor(rpc): use generic instead of alias

* docs(rpc): improve docs for getblock method

* test(rpc): add a test for getblock method

* deps(rpc): remove non needed tower features

Co-authored-by: teor <teor@riseup.net>

* docs(rpc): add a note to getblock doc

* refactor(rpc): replace alias

* fix(rpc): use `zcash_serialize_to_vec()` instead of logging format

* tests(rpc): add network argument to `populated_state()`

* refactor(rpc): use an error for state service readiness

* fix(rpc): add parameter

* fix(rpc): clippy

* nit(rpc): remove new line from imports

* fix(rpc): remove commented code

* fix(rpc): simplify error

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Use a `SerializedBlock` type to help serializing blocks (#3725)

* Create a `SerializedBlock` helper type

Create a type that can be used as a byte slice, but is guaranteed to
represent a valid block.

* Use `into_iter` instead of `iter`

There's no need to borrow the elements, they can be moved out directly.
This will be necessary because `&Arc<T>` doesn't implement `Borrow<T>`,
so a `SerializedBlock` can't be built directly from an `&Arc<Block>`.

* Use `SerializedBlock` in `GetBlock`

Make the type stricter to avoid storing possibly invalid values. The
bytes are still serialized as a hexadecimal string, through the usage of
`hex`.

The `serde::Deserialize` can't be derived because `hex` requires the
type to also implement `FromHex`.

* feature(rpc): add suggestions from code review

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* tests(rpc): make sure mempool has no requests in get_block test

* fix(rpc): change height argument type in getblock method

* fix(rpc): rustfmt

* fix(rpc): replace panic

* fix(rpc): change getblock response

* fix(rpc): fix lightwalletd test

* tests(rpc): add a getblock error test

* fix(rpc): try another regex

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2022-03-10 01:12:41 +00:00
Alfredo Garcia 30ebab626a
docs: Transaction consensus rules: Header rules (#3456)
* refactor transaction header consensus rules

* add a note

* readd some docs

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-04 18:07:20 +00:00
teor 4ce6fbccc4
Fix new clippy lints in clippy nightly (#3176) 2021-12-09 14:19:14 +00:00
teor 1df3bdb089
Check MAX_BLOCK_SIGOPS in the block verifier (#3049)
* Cleanup a function that calls zcash_script

* Remove zebra_test::prelude macros that conflict with the Rust prelude

* Add sigops count support to zebra-script

* Check MAX_BLOCK_SIGOPS in the block verifier

* Test MAX_BLOCK_SIGOPS on generated and historic blocks

* Add SAFETY comments for all unsafe zebra-script code

* Explain where the consensus rule comes from

* Remove unused pretty_assertions dependency

* Allow large test block generation functions with the proptest-impl feature

* Replace `as` with `try_into` for integer conversions in unsafe code

* Expand SAFETY comments
2021-11-15 20:55:32 +00:00
Dimitris Apostolou afb8b3d477
Fix typos (#3055)
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-12 19:30:22 +00:00
Janito Vaqueiro Ferreira Filho 9e78a8af40
Refactor mempool spend conflict checks to increase performance (#2826)
* Add `HashSet`s to help spend conflict detection

Keep track of the spent transparent outpoints and the revealed
nullifiers.

Clippy complained that the `ActiveState` had variants with large size
differences, but that was expected, so I disabled that lint on that
`enum`.

* Clear the `HashSet`s when clearing the mempool

Clear them so that they remain consistent with the set of verified
transactions.

* Use `HashSet`s to check for spend conflicts

Store new outputs into its respective `HashSet`, and abort if a
duplicate output is found.

* Remove inserted outputs when aborting

Restore the `HashSet` to its previous state.

* Remove tracked outputs when removing a transaction

Keep the mempool storage in a consistent state when a transaction is
removed.

* Remove tracked outputs when evicting from mempool

Ensure eviction also keeps the tracked outputs consistent with the
verified transactions.

* Refactor to create a `VerifiedSet` helper type

Move the code to handle the output caches into the new type. Also move
the eviction code to make things a little simpler.

* Refactor to have a single `remove` method

Centralize the code that handles the removal of a transaction to avoid
mistakes.

* Move mempool size limiting back to `Storage`

Because the evicted transactions must be added to the rejected list.

* Remove leftover `dbg!` statement

Leftover from some temporary testing code.

Co-authored-by: teor <teor@riseup.net>

* Remove unnecessary `TODO`

It is more speculation than planning, so it doesn't add much value.

Co-authored-by: teor <teor@riseup.net>

* Fix typo in documentation

The verb should match the subject "transactions" which is plural.

Co-authored-by: teor <teor@riseup.net>

* Add a comment to warn about correctness

There's a subtle but important detail in the implementation that should
be made more visible to avoid mistakes in the future.

Co-authored-by: teor <teor@riseup.net>

* Remove outdated comment

Left-over from the attempt to move the eviction into the `VerifiedSet`.

* Improve comment explaining lint removal

Rewrite the comment explaining why the Clippy lint was ignored.

* Check for spend conflicts in `VerifiedSet`

Refactor to avoid API misuse.

* Test rejected transaction rollback

Using two transactions, perform the same test adding a conflict to both
of them to check if the second inserted transaction is properly
rejected. Then remove any conflicts from the second transaction and add
it again. That should work, because if it doesn't it means that when the
second transaction was rejected it left things it shouldn't in the
cache.

* Test removal of multiple transactions

When removing multiple transactions from the mempool storage, all of the
ones requested should be removed and any other transaction should be
still be there afterwards.

* Increase mempool size to 4, so that spend conflict tests work

If the mempool size is smaller than 4,
these tests don't fail on a trivial removal bug.
Because we need a minimum number of transactions in the mempool
to trigger the bug.

Also commit a proptest seed that fails on a trivial removal bug.
(This seed fails if we remove indexes in order,
because every index past the first removes the wrong transaction.)

* Summarise transaction data in proptest error output

* Summarise spend conflict field data in proptest error output

* Summarise multiple removal field data in proptest error output

And replace the very large proptest debug output with the new summary.

Co-authored-by: teor <teor@riseup.net>
2021-10-10 23:54:46 +00:00
Conrado Gouvea bc4194fcb9
ZIP-221/244 auth data commitment validation in checkpoint verifier (#2633)
* Add validation of ZIP-221 and ZIP-244 commitments

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Add auth commitment check in the finalized state

* Reset the verifier when comitting to state fails

* Add explanation comment

* Add test with fake activation heights

* Add generate_valid_commitments flag

* Enable fake activation heights using env var instead of feature

* Also update initial_tip_hash; refactor into progress_from_tip()

* Improve comments

* Add fake activation heights test to CI

* Fix bug that caused commitment trees to not match when generating partial arbitrary chains

* Add ChainHistoryBlockTxAuthCommitmentHash::from_commitments to organize and deduplicate code

* Remove stale comment, improve readability

* Allow overriding with PROPTEST_CASES

* partial_chain_strategy(): don't update note commitment trees when not needed; add comment

Co-authored-by: teor <teor@riseup.net>
2021-08-23 14:17:33 +00:00
Conrado Gouvea eadca72e75
Implement ZIP-244 authorizing data commitment (auth_digest) (#2547)
* Implement ZIP-244 authorizing data commitment (auth_digest)

* s/Merke/Merkle/

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Fix AuthDataRoot computation to use padded leaves; add tests

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-08-13 12:58:04 -04:00
teor f09f2a9022
Check remaining transaction value & make value balance signs match the spec (#2566)
* Make Amount arithmetic more generic

To modify generated amounts, we need some extra operations on `Amount`.

We also need to extend existing operations to both `NonNegative` and
`NegativeAllowed` amounts.

* Add a constrain method for ValueBalance

* Derive Eq for ValueBalance

* impl Neg for ValueBalance

* Make some Amount arithmetic expectations explicit

* Explain why we use i128 for multiplication

And expand the overflow error details.

* Expand Amount::sum error details

* Make amount::Error field order consistent

* Rename an amount::Error variant to Constraint, so it's clearer

* Add specific pool variants to ValueBalanceError

* Update coinbase remaining value consensus rule comment

This consensus rule was updated recently to include coinbase transactions,
but Zebra doesn't check block subsidy or miner fees yet.

* Add test methods for modifying transparent values and shielded value balances

* Temporarily set values and value balances to zero in proptests

In both generated chains and proptests that construct their own transactions.

Using zero values reduces value calculation and value check test coverage.
A future change will use non-zero values, and fix them so the check passes.

* Add extra fields to remaining transaction value errors

* Swap the transparent value balance sign to match shielded value balances

This makes the signs of all the chain value pools consistent.

* Use a NonNegative constraint for transparent values

This fix:
* makes the type signature match the consensus rules
* avoids having to write code to handle negative values

* Allocate total generated transaction input value to outputs

If there isn't enough input value for an output, set it to zero.

Temporarily reduce all generated values to avoid overflow.
(We'll remove this workaround when we calculate chain value balances.)

* Consistently use ValueBalanceError for ValueBalances

* Make the value balance signs match the spec

And rename and document methods so their signs are clearer.

* Convert amount::Errors to specific pool ValueBalanceErrors

* Move some error changes to the next PR

* Add extra info to remaining transaction value errors (#2585)

* Distinguish between overflow and negative remaining transaction value errors

And make some error types cloneable.

* Add methods for updating chain value pools (#2586)

* Move amount::test to amount::tests:vectors

* Make ValueBalance traits more consistent with Amount

- implement Add and Sub variants with Result and Assign
- derive Hash

* Clarify some comments and expects

* Create ValueBalance update methods for blocks and transactions

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-09 14:22:26 -03:00
Conrado Gouvea fe989e0758
ZIP-221: Add Orchard support to history tree (#2531)
* Add Orchard support to HistoryTree

* Handle network upgrades in HistoryTree

* Add additional methods to save/load HistoryTree

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Clarification of Entry documentation

* Improvements from code review

* Add HistoryTree tests

* Improved test comments and variable names based on feedback from #2458 on similar test

* Update zebra-chain/src/history_tree.rs

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Use type aliases for V1 and V2 history trees

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-08-03 18:33:51 +00:00
Alfredo Garcia 0325aa2517
Add value_balance() method to Block (#2546)
* add value_balance() method to Block

* Fix a comment typo

Co-authored-by: teor <teor@riseup.net>
2021-07-29 23:46:52 +00:00
teor 3d792f7195
Validate spends of transparent coinbase outputs (#2525)
* Validate transparent coinbase output maturity and shielding

- Add a CoinbaseSpendRestriction enum and Transaction method
- Validate transparent coinbase spends in non-finalized chains

* Don't use genesis created UTXOs for spends in generated block chains

* Refactor out a new_transaction_ordered_outputs function

* Add Transaction::outputs_mut for tests

* Generate valid transparent spends in arbitrary block chains

* When generating blocks, fixup the block contents, then the block hash

* Test that generated chains contain at least one transparent spend

* Make generated chains long enough for reliable tests

* Add transparent and shielded input and output methods to Transaction

* Split chain generation into 3 functions

* Test that unshielded and immature transparent coinbase spends fail

* Comment punctuation

* Clarify a comment

* Clarify probability calculation

* Test that shielded mature coinbase output spends succeed
2021-07-29 14:23:50 +10:00
teor 14d5abdb03
Add methods for getting block nullifiers (#2465)
* Add methods for getting block nullifiers

These methods will be used in a future PR to check for double-spends.

* Add doc links

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-07-09 02:54:57 +00:00
Alfredo Garcia 7638c43a7c
Move network_upgrade check into zebra-chain (#2354)
* move network_upgrade check into zebra-chain

* fix the errors

* rename function

* typo fix

* rename the check function

* make changes from last code review
2021-06-22 12:06:52 +10:00
Conrado Gouvea 5c0880810b
ZIP-221: integrate MMR tree from librustcash (without Orchard) (#2227)
* add zcash_history.rs with librustzcash Tree wrapper

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Apply changes from code review

* Update zebra-chain/src/primitives/zcash_history.rs

Co-authored-by: teor <teor@riseup.net>

* Apply changes from code review

* Add Entry struct; return Result where needed; add test

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* zcash_history: improve naming style with `inner`

* zcash_history: check if block has the correct network upgrade when adding to tree

* zcash_history: test improvements

* zcash_history: split Tree::new into new_from_block and new_from_cache

* zcash_history: move tests to their own file

* remove unneeded empty line in Cargo.toml

Co-authored-by: teor <teor@riseup.net>
2021-06-11 11:25:48 -03:00
teor 2f0f379a9e
Standardise clippy lints and require docs (#2238)
* Standardise lints across Zebra crates, and add missing docs

The only remaining module with missing docs is `zebra_test::command`

* Todo -> TODO

* Clarify what a transcript ErrorChecker does

Also change `Error` -> `BoxError`

* TransError -> ExpectedTranscriptError

* Output Descriptions -> Output descriptions
2021-06-04 08:48:40 +10:00
teor 0d8ffc367e Move LedgerState into the block module 2021-04-23 09:34:13 -04:00
Kirill Fomichev b636cf8ac1
Benchmark Block struct serialization code (#2018)
* Benchmark Block struct serialization code

* benchmark generated block

* add deserialize benchmark

* Fix block/tests.rs conditional module imports

* add large_single_transaction_block

* Fix name for deserialise benchmark

Co-authored-by: teor <teor@riseup.net>
2021-04-19 12:15:02 +00:00
teor 64662a758d
Move the preallocate tests into their own files (#1977)
* Move the preallocate tests into their own files

And move the MetaAddr proptest into its own file.

Also do some minor formatting and cleanups.

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2021-04-07 12:32:27 +10:00
teor 05b60db993
Clean up block commitment enum and parsing (#1978)
* Rename RootHash to Commitment based on ZIP-244

Interactive replace using:
```sh
fastmod RootHash Commitment
fastmod root_hash commitment
fastmod root_bytes commitment_bytes
git mv zebra-chain/src/block/root_hash.rs zebra-chain/src/block/commitment.rs
```

All replacements were accepted.

* rustfmt

* Comment and format cleanups after interactive replace

* Distinguish Sapling tree roots from other tree roots

* Add the NU5 BlockCommitmentsHash variant to block::Commitment

This change parses the hash, but does not perform validation.

* Validate reserved values in Block::commitment

- change Block::commitment to return a Result rather than an Option
- enforce the all-zeroes reserved value consensus rules
- change `PreSaplingReserved([u8; 32])` to `PreSaplingReserved`
- change `ChainHistoryActivationReserved([u8; 32])` to `ChainHistoryActivationReserved`
- update the function comments to describe when each variant is verified

* Fix comment whitespace
2021-04-06 20:19:28 +10:00
Preston Evans 0daaf582e2
Implement Trusted Vector Preallocation (#1920)
* Implement SafePreallocate. Resolves #1880

* Add proptests for SafePreallocate

* Apply suggestions from code review

Comments which did not include replacement code will be addressed in a follow-up commit.

Co-authored-by: teor <teor@riseup.net>

* Rename [Safe-> Trusted]Allocate. Add doc and tests

Add tests to show that the largest allowed vec under TrustedPreallocate
is small enough to fit in a Zcash block/message (depending on type).
Add doc comments to all TrustedPreallocate test cases.
Tighten bounds on max_trusted_alloc for some types.

Note - this commit does NOT include TrustedPreallocate
impls for JoinSplitData, String, and Script.
These impls will be added in a follow up commit

* Implement SafePreallocate. Resolves #1880

* Add proptests for SafePreallocate

* Apply suggestions from code review

Comments which did not include replacement code will be addressed in a follow-up commit.

Co-authored-by: teor <teor@riseup.net>

* Rename [Safe-> Trusted]Allocate. Add doc and tests

Add tests to show that the largest allowed vec under TrustedPreallocate
is small enough to fit in a Zcash block/message (depending on type).
Add doc comments to all TrustedPreallocate test cases.
Tighten bounds on max_trusted_alloc for some types.

Note - this commit does NOT include TrustedPreallocate
impls for JoinSplitData, String, and Script.
These impls will be added in a follow up commit

* Impl TrustedPreallocate for Joinsplit

* Impl ZcashDeserialize for Vec<u8>

* Arbitrary, TrustedPreallocate, Serialize, and tests for Spend<SharedAnchor>

Co-authored-by: teor <teor@riseup.net>
2021-04-06 09:49:42 +10:00
teor 29163cd0b4
Rename RootHash to Commitment based on ZIP-244 (#1957)
* Rename RootHash to Commitment based on ZIP-244

Interactive replace using:
```sh
fastmod RootHash Commitment
fastmod root_hash commitment
fastmod root_bytes commitment_bytes
git mv zebra-chain/src/block/root_hash.rs zebra-chain/src/block/commitment.rs
```

All replacements were accepted.

* rustfmt
2021-03-31 09:51:42 +10:00
teor 0eeb77a44c Remove unused import
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-03-22 11:54:31 -04:00
teor 609d70ae53 Stop untrusted preallocation during string deserialization
This is an easy memory denial of service attack.
2021-03-22 11:54:31 -04:00
teor f2bdebee07 Clippy: Use Option::map instead of a manual impl 2021-03-01 23:31:18 -05:00
Henry de Valence b449fe93b2 network: correct data modeling for headers messages
We modeled a Bitcoin `headers` message as being a list of block headers.
However, the actual data structure is slightly different: it's a list of (block
header, transaction count) pairs.  This caused zcashd to reject our headers
messages.

To fix this, introduce a new `CountedHeader` struct with a `block::Header` and
transaction count `usize`, then thread it through the inbound service and the
state.

I tested this locally by running Zebra with these changes and inspecting a
trace-level log of the span of a peer connection that requested a nontrivial
headers packet from us, and verified that it did not reject our message.
2020-12-02 10:24:31 -08:00
Alfredo Garcia 4544463059
Inbound `FindBlocks` and `FindHeaders` (#1347)
* implement inbound `FindBlocks`
* Handle inbound peer FindHeaders requests
* handle request before having any chain tip
* Split `find_chain_hashes` into smaller functions

Add a `max_len` argument to support `FindHeaders` requests.

Rewrite the hash collection code to use heights, so we can handle the
`stop` hash and "no intersection" cases correctly.

* Split state height functions into "any chain" and "best chain"
* Rename the best chain block method to `best_block`
* Move fmt utilities to zebra_chain::fmt
* Summarise Debug for some Message variants

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
2020-12-01 07:30:37 +10:00
teor c4630cd1f5 Improve error messages for header.time validation 2020-10-13 08:11:11 +10:00
Jane Lusby 855f9b5bcb
Implement MVP of NonFinalizedState and integrate it with the state service (#1101)
* implement most of the chain functions
* implement fork
* fix outpoint handling in Chain struct
* update expect for work
* split utxo into two sets
* update the Chain definition
* remove allow attribute in zebra-state/lib.rs
* merge ChainSet type into MemoryState
* Add error messages to asserts
* export proptest impls for use in downstream crates
* add testjob for disabled feature in zebra-chain
* try to fix github actions syntax
* add module doc comment
* update RFC for utxos
* add missing header
* working proptest for Chain
* propagate back results over channel
* Start updating RFC to match changes
* implement queued block pruning
* and now it syncs wooo!
* remove empty modules
* setup config for proptests
* re-enable missing_docs lint
* update RFC to match changes in impl
* add documentation
* use more explicit variable names
2020-10-08 13:07:32 +10:00
Jane Lusby 86ed13060f
Add tests for `Chain` implementation (#1093)
* Begin work on RFC5 implementation

* I think this is necessary

* holy shit supertrait implemented via subtrait

* implement most of the chain functions

* change to slightly better name

* implement fork

* fix outpoint handling in Chain struct

* update expect for work

* resolve review comment

* split utxo into two sets

* update the Chain definition

* just a little more

* update comment

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* apply changes from code review

* remove allow attribute in zebra-state/lib.rs

* Update zebra-state/src/memory_state.rs

Co-authored-by: teor <teor@riseup.net>

* merge ChainSet type into MemoryState

* rename state impl types

* Add error messages to asserts

* checkpoint so I can split off arbitrary changes into a PR

* export proptest impls for use in downstream crates

* add testjob for disabled feature in zebra-chain

* run rustfmt

* try to fix github actions syntax

* differentiate name

* prove that github action tests zebra-chain build without features

* revert change from last commit now that test is running

* remove accidentally introduced newline

* checkpoint

* add module doc comment

* update RFC for utxos

* add missing header

* working proptest for Chain

* apply change from chain impl PR

* setup config for proptests

* Update zebra-chain/src/block/arbitrary.rs

Co-authored-by: teor <teor@riseup.net>

* run rustfmt

Co-authored-by: teor <teor@riseup.net>
2020-10-02 15:51:51 -07:00
Jane Lusby 0b4e974c9e
export proptest impls for use in downstream crates (#1092)
* export proptest impls for use in downstream crates

* add testjob for disabled feature in zebra-chain

* run rustfmt

* try to fix github actions syntax

* differentiate name

* prove that github action tests zebra-chain build without features

* revert change from last commit now that test is running

* remove accidentally introduced newline

* Update .github/workflows/ci.yml

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2020-09-23 18:52:52 -07:00
Henry de Valence ca4a5ce30c chain: add Transaction::hash() method.
This makes Transaction and Block have a consistent API.
2020-09-04 17:08:41 -04:00