Commit Graph

209 Commits

Author SHA1 Message Date
Conrado Gouvea dff25473aa
refactor: document coinbase rules, refactor to ease understanding (#4056)
* refactor: document coinbase rules, refactor to ease understanding

* Update zebra-consensus/src/block/check.rs

Co-authored-by: teor <teor@riseup.net>

* remove no longer used contains_prevout_input()

* remove unused CoinbaseHasPrevOutInput

* update coinbase description in documentation

Co-authored-by: teor <teor@riseup.net>
2022-04-20 09:31:12 +00:00
Conrado Gouvea d79e71e969
change(nu5): use new V5 transaction script verification API (#3799)
* update librustzcash; adapt to new API

* add ticket reference for removing zcash_proofs duplicated dependencies

* update to new zcash_script V5 API

* use zp_tx shorthand

* update to Zcash 4.7.0 dependencies

* update protocol versions

* feat(rpc): Implement `getblockchaininfo` RPC method (#3891)

* Implement `getblockchaininfo` RPC method

* add a test for `get_blockchain_info`

* fix tohex/fromhex

* move comment

* Update lightwalletd acceptance test for getblockchaininfo RPC (#3914)

* change(rpc): Return getblockchaininfo network upgrades in height order (#3915)

* Update lightwalletd acceptance test for getblockchaininfo RPC

* Update some doc comments for network upgrades

* List network upgrades in order in the getblockchaininfo RPC

Also:
- Use a constant for the "missing consensus branch ID" RPC value
- Simplify fetching consensus branch IDs
- Make RPC type derives consistent
- Update RPC type documentation

* Make RPC type derives consistent

* Fix a confusing test comment

* get hashand height at the same time

* fix estimated_height

* fix lint

* add extra check

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix typo

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* split test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix(rpc): ignore an expected error in the RPC acceptance tests (#3961)

* Add ignored regexes to test command failure regex methods

* Ignore empty chain error in getblockchaininfo

We expect this error when zebrad starts up with an empty state.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Make sync error logs more user-friendly (#3944)

- use info level, there is nothing the user needs to do,
  particularly for a single error
- explain that the errors are temporary
- hide backtraces, because they look like crashes

* Update test.patch.yml with lightwalletd job (#3970)

* Update test.patch.yml with lightwalletd job

* Remove a workflow condition that will always be false

In general, patch workflows need the
opposite conditions to the original workflow.

But in this case, we know the result of the
condition will always be true, so we can just delete it.

Co-authored-by: teor <teor@riseup.net>

* fix(doc): Fix bugs in the lightwalletd database design (#3964)

* Re-order column families in design in dependency order

* Minor RFC design tweaks and fixes

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Repoint zebra image links to our new zfnd.org site for now (#3949)

* Repoint zebra image links to our new zfnd.org site for now

* Remove images/

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Fix typos (#3956)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* bump database version to trigger testnet rollback

* reduce minimum protocol version for now (will be changed later)

* update dependencies

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* update versions to match zcash 4.7.0

* deny.toml: update 'darling'

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Dimitris Apostolou <dimitris.apostolou@icloud.com>
2022-04-19 10:14:16 +10:00
Conrado Gouvea e7c0a78d4d
feat(rpc): add getrawtransaction (#3908)
* feat(rpc): add getrawtransaction

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Apply suggestions from code review

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* address review comments

* move SerializedTransaction to the right module

* Use try_into() instead of 'as'

* add proptests

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-24 09:45:37 +00:00
Conrado Gouvea b9640fbb55
fix(chain): make FromHex consistent with ToHex for tx/block hashes (#3893) 2022-03-17 20:24:48 +00:00
Janito Vaqueiro Ferreira Filho 0e0aefaa4e
Refactor `SentTransactionHash` to be a stricter type (#3706)
* Stub `sendrawtransaction` RPC method

Register the RPC method, and stub an implementation that currently just
panics. The method has a single `String` parameter with the hexadecimal
string of the raw transaction's bytes and returns a
`SentTransactionHash` wrapper type that's just a hexadecimal `String` of
the sent transaction's hash.

* Add mempool service instance to `RpcImpl`

Use a type parameter to represent the mempool service using the
interface defined by `zebra-node-services`.

* Update test vector to use a mock mempool service

Update the test to be compatible with the changes to `RpcImpl`. The mock
mempool service is expected to not be used during the test.

* Use a `tower::Buffer` for the mempool service

Make it simpler to send requests to the service in a concurrent manner.

* Return a `Future` from `send_raw_transaction`

Make the call asynchronous.

* Implement `sendrawtransaction` RPC

Deserialize the transaction and send it to be queued for verification
and subsequent inclusion in the mempool.

* Test if mempool receives sent raw transaction

Use a mock service as the mempool service and check that it receives a
sent raw transaction.

* Test using non-hexadecimal string parameter

The method should return an error.

* Test with bytes that fail deserialization

Check that the method returns an invalid parameters error if the input
can't be deserialized as a `Transaction`.

* Test if mempool errors are forwarded to caller

Mempool service errors should be sent back to the remote caller as
server errors.

* Test transactions rejected by the mempool service

Transactions that are rejected by the mempool service should result in
a server error being sent to the caller.

* Improve error message

Add the word "structurally" to make it clear that the issue is in the
transaction's deserialization.

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* Add note regarding missing `allowhighfees` param.

The parameter isn't supported yet because `lightwalletd` doesn't use it.

* Update the documentation to be consistent

Follow the convention adopted by the `get_info` RPC method.

* Implement `ToHex` and `FromHex` for `Hash`

Make it easier to generate hexadecimal strings from `transaction::Hash`
instances.

* Use `ToHex` in `Debug` and `Display`

Reduce repeated code.

* Refactor to add `bytes_in_display_order` method

Use it to remove repeated code and improve clarity a bit.

* Use `hex::serialize` to serialize transaction hash

Make the type stricter in its contents, while still serializing the
transaction has as a hexadecimal string.

* Simplify serialization attribute

Deserialization should also use `hex::deserialize`, so using the shorter
attribute makes things easier to read and more future proof.

* Update zebra-chain/src/transaction/hash.rs

* Remove unnecessary lifetime

The anonymous lifetime is automatically inferred by the compiler.

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-03-08 09:14:21 +00:00
Conrado Gouvea b7b62af2f2
Document consensus rules from 4.6 Action Descriptions (#3549)
* docs: document consensus rules from 4.6 Action Descriptions

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Marek <mail@marek.onl>
2022-02-21 23:49:32 +00:00
Conrado Gouvea ab03a376ef
docs: document consensus rules from 7.3 Spend Description Encoding and Consensus (#3575) 2022-02-19 02:27:09 +00:00
Alfredo Garcia 137ae4e041
refactor(anchorSapling): Change type to force consensus rule validation (#3544)
* change `anchorSapling` type

* implement PartialEq manually for clippy

* use `unique_by` in place of `sorted`

* replace panic with new error

* improve some serialize/deserialize calls for sapling anchors

* fix arbitrary for sapling::tree::Root

* remove dedup()

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-17 03:20:22 +00:00
Conrado Gouvea 495e1ec20f
docs: document consensus rules from 4.5 Output Descriptions (#3462)
* docs: document consensus rules from 4.4 Spend Descriptions

* docs: document consensus rules from 4.5 Output Descriptions

* cargo fmt

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-14 21:31:20 +00:00
Alfredo Garcia 9ede0fb52a
document transaction orchard types (#3507) 2022-02-11 03:16:35 +00:00
Alfredo Garcia 62750c8192
document transaction transparent types (#3498) 2022-02-11 03:16:29 +00:00
Alfredo Garcia c57339bbee
document transaction sprout types (#3499) 2022-02-11 03:16:22 +00:00
Alfredo Garcia 14882183c3
document transaction sapling types (#3501) 2022-02-11 03:16:16 +00:00
Marek 683b88c819
Support large block heights (#3401)
* Support large block heights

* Document consensus rules referring to expiry heights

* Refactor the docs

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Fix the formatting of an error message

* refactor: Simplify coinbase expiry code so the consensus rule is clear (#3408)

* Fix some outdated TODO comments

* refactor(coinbase expiry): Simplify the code so consensus rule is clear

* Fix the formatting of an error message

* Remove a redundant comment

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Marek <mail@marek.onl>

* Check the max expiry height at parse time

* Test that 2^31 - 1 is the last valid height

* Add tests for nExpiryHeight

* Add tests for expiry heights of V4 transactions

* Add tests for V5 transactions

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-11 00:32:57 +00:00
Alfredo Garcia 88aca26d30
docs(transaction encoding): header fields (#3491)
* document the `header` field

* document the `nVersionGroupId` field

* document the `nConsensusBranchId` field

* document the `lock_time` field

* document the `nExpiryHeight` field (and some missing `lock_time`)

* add missing note to `header` field in serialization

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-09 00:39:09 +00:00
Alfredo Garcia 29ad801a35
docs: Transaction consensus rules: Size rules (#3461)
* refactor transaction size consensus rules

* quote mssing consensus rule

* nit

Co-authored-by: teor <teor@riseup.net>

* move consensus rule doc

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-08 20:28:40 +00:00
Conrado Gouvea 3bbb8f0531
docs: document consensus rules from 4.4 Spend Descriptions (#3460) 2022-02-08 09:56:59 +00:00
Deirdre Connolly 3644171738
Use string literal to format unreachable message (#3479) 2022-02-08 05:56:32 +00:00
Alfredo Garcia 30ebab626a
docs: Transaction consensus rules: Header rules (#3456)
* refactor transaction header consensus rules

* add a note

* readd some docs

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-04 18:07:20 +00:00
Conrado Gouvea 494b7dc9f4
refactor: address comments from #3415: Prepare for changes in ZIP-244 (#3446)
* refactor: address comments from #3415

* Shorter `if let` match

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-01 06:24:08 +00:00
Conrado Gouvea f270fd2de6
Prepare for changes in ZIP-244 (#3415)
* Add all_previous_outputs; load UTXOs in transaction verifier

* Remove UTXO loading and returning from script.rs

* Don't pass state service to script verifier

* Remove output from is_valid()

* Refactor loading UTXOs to separate function

* Pass all_previous_output to sighash

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Create AwaitUtxo only when needed; formatting

* Add comments about output vectors in tests

* Change sighash() to receive reference and avoid cloning

* Expand comments

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-31 15:28:42 +00:00
Marek a1f4cec0de
Document the consensus rules for Section 3.6 (#3338) 2022-01-24 23:27:45 +00:00
Deirdre Connolly 9734ea1014
Check jubjub key correctness independent of redjubjub / jubjub (#3154)
* Ensure that sapling::keys::TransmissionKey jubjub point is always in the prime order group

* Adjust TransmissionKey check; add AuthorizingKey check

* Move ValueCommitment small order check to deserialization

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Use is_torsion_free() instead of is_identity() and is_prime_order()

* Add EphemeralPublicKey small order check on instantiation; remove old checks

* Use VerificationKey instead of VerificationKeyBytes; fix tests

* Use ValidatingKey instead of VerificationKeyBytes for rk

* Reject identity when creating an Orchard EphemeralPublicKey

* Make documentation more consistent, use generator in tests

* s/JubJub/Jubjub/

* Fix zebra-consensus tests (insert_fake_orchard_shielded_data)

* Create NotSmallOrderValueCommitment, since intermediate values can be the identity

* Clarify documentation

* rustdoc

* Tidy rustdoc

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
2021-12-22 15:10:34 +00:00
teor 4ce6fbccc4
Fix new clippy lints in clippy nightly (#3176) 2021-12-09 14:19:14 +00:00
Janito Vaqueiro Ferreira Filho ec2c980bb1
Validate transaction lock times (#3060)
* Create a `LockTime::unlocked` helper constructor

Returns a `LockTime` that is unlocked at the genesis block.

* Return `Option<LockTime>` from `lock_time` method

Prepare to return `None` for when a transaction has its lock time
disabled.

* Return `None` instead of zero `LockTime`

Because a zero lock time means that the transaction was unlocked at the
genesis block, so it was never actually locked.

* Rephrase zero lock time check comment

Clarify that the check is not redundant, and is necessary for the
genesis transaction.

Co-authored-by: teor <teor@riseup.net>

* Add a `transparent::Input::sequence` getter method

Retrieve a transparent input's sequence number.

* Check if lock time is enabled by a sequence number

Validate the consensus rule that the lock time is only enabled if at
least one transparent input has a value different from `u32::MAX` as its
sequence number.

* Add more Zcash specific details to comment

Explain the Zcash specific lock time behaviors.

Co-authored-by: teor <teor@riseup.net>

* Add `time` field to `Request::Block` variant

The block time to use to check if the transaction was unlocked and
allowed to be included in the block.

* Add `Request::block_time` getter

Returns the block time for the block that owns the transaction being
validated or the current time plus a tolerance for mempool transactions.

* Validate transaction lock times

If they are enabled by a transaction's transparent input sequence
numbers, make sure that they are in the past.

* Add comments with consensus rule parts

Make it easier to map what part of the consensus rule each match arm is
responsible for.

Co-authored-by: teor <teor@riseup.net>
2021-11-23 05:53:53 +00:00
Alfredo Garcia dbd49a3f00
Validate coinbase expiration height (#3082)
* add testnet test blocks around nu5

* validate coinbase expiration height

* change const name and doc

Co-authored-by: teor <teor@riseup.net>

* change commit location

Co-authored-by: teor <teor@riseup.net>

* use pre Nu5 rules when there is no activation height

* add sapling final root to nu5 test vectors

* fix tests

Co-authored-by: teor <teor@riseup.net>
2021-11-23 05:17:05 +00:00
Marek 8963007397
Sprout note commitment trees (#3051)
* Implement incremental note commitment Merkle tree for Sprout

* Add tests for Sprout note commitment tree

* Remove the `Arbitrary` attribute

* Reverse the vector of empty roots

* Add more tests

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustdoc

* Rustdoc

* rustdoc links

* Oops, need the trait in scope to use it

* Avoid accessing the wrapped hash directly

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustfmt

* Add typing

* Avoid accessing the wrapped hash directly

* Implement incremental note commitment Merkle tree for Sprout

* Add tests for Sprout note commitment tree

* Remove the `Arbitrary` attribute

* Reverse the vector of empty roots

* Add more tests

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Refactor rustdoc

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustdoc

* Rustdoc

* rustdoc links

* Oops, need the trait in scope to use it

* Avoid accessing the wrapped hash directly

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* rustfmt

* Add typing

* Avoid accessing the wrapped hash directly

* Add Overwinter final roots (test vectors)

* Test sprout note commitments trees on Overwinter blocks

* Add new test vectors

* Finish the tests for the note commitment trees

* Make the wrapped hash in `Root` private

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2021-11-18 23:05:52 +00:00
teor d6f3b3dc9a
Parse received addrv2 messages (#3022)
* Revert "Remove commented-out code"

This reverts commit 9e69777925f103ee11e5940bba95b896c828839b.

* Implement deserialization for `addrv2` messages

* Limit addr and addrv2 messages to MAX_ADDRS_IN_MESSAGE

* Clarify address version comments

* Minor cleanups and fixes

* Add preallocation tests for AddrV2

* Add serialization tests for AddrV2

* Use prop_assert in AddrV2 proptests

* Use a generic utility method for deserializing IP addresses in `addrv2`

* Document the purpose of a conversion to MetaAddr

* Fix a comment typo, and clarify that comment

* Clarify the unsupported AddrV2 network ID error and enum variant names

```sh
fastmod AddrV2UnimplementedError UnsupportedAddrV2NetworkIdError zebra-network
fastmod Unimplemented Unsupported zebra-network
```

* Fix and clarify unsupported AddrV2 comments

* Replace `panic!` with `unreachable!`

* Clarify a comment about skipping a length check in a test

* Remove a redundant test

* Basic addr (v1) and addrv2 deserialization tests

* Test deserialized IPv4 and IPv6 values in addr messages

* Remove redundant io::Cursor

* Add comments with expected values of address test vectors
2021-11-12 00:25:23 +00:00
teor c0c00b3f0d
Simplify preallocate tests (#3032)
* Simplify preallocation tests using a test function

* Use prop_assert in proptests
2021-11-11 07:53:21 +10:00
teor b1303ab8d7
Replace read_compactsize and write_compactsize with CompactSizeMessage (#3014)
* Replace read_compactsize and write_compactsize with CompactSizeMessage

* Add tests for CompactSize64

* Add compact size range and conversion tests
2021-11-05 15:24:24 -03:00
Deirdre Connolly 0381c2347b
ZIP-401: weighted random mempool eviction (#2889)
* ZIP-401 weighted random mempool eviction

* rename zcash.mempool.total_cost.bytes to zcash.mempool.cost.bytes

Co-authored-by: teor <teor@riseup.net>

* Remove duplicated lines

* Add cost() method to UnminedTx

Update serialization failure messages

* More docs quoting ZIP-401 rules

* Change mempool::Storage::new() to handle Copy-less HashMap, HashSet

* mempool: tidy cost types and evict_one()

* More consensus rule docs

* Refactor calculating mempool costs for Unmined transactions

* Add a note on asympotic performance of calculating weights of txs in mempool

* Bump test mempool / storage config to avoid weighted random cost limits

* Use mempool tx_cost_limit = u64::MAX for some tests

* Remove failing tests for now

* Allow(clippy::field-reassign-with-default) because of a move on a type that doesn't impl Copy

* Fix mistaken doctest formatting

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Increase test timeout for Windows builds

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-10-27 00:21:19 +00:00
Marek 2920d838ff
Limit transaction size in the mempool (#2917)
* Limit tx size

Zebra now limits the transaction size in the `zcash_deserialize()` method for
`Transaction`.

* Remove unused error variants (#2941)

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Limit tx size

Zebra now limits the transaction size in the `zcash_deserialize()` method for
`Transaction`.

* Test the tx deserialization limit

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-10-25 15:25:28 -03:00
teor 2d129414e0
Store the transaction fee in the mempool storage (#2885)
* Create a new VerifiedUnminedTx containing the miner fee

* Use VerifiedUnminedTx in mempool verification responses

And do a bunch of other cleanups.

* Use VerifiedUnminedTx in mempool download and verifier

* Use VerifiedUnminedTx in mempool storage and verified set

* Impl Display for VerifiedUnminedTx, and some convenience methods

* Use VerifiedUnminedTx in existing tests
2021-10-18 11:24:37 +10:00
teor 2f0926a8e4
Stop ignoring the mempool conflicting transaction reject list size limit (#2855)
* Limit the size of rejection lists when there is a spend conflict

Previously, `insert` would return early with an error,
and skip limiting the rejection list sizes.

* Use prop_assert macros in proptests, rather than assert
2021-10-12 10:35:50 +10:00
Janito Vaqueiro Ferreira Filho 9e78a8af40
Refactor mempool spend conflict checks to increase performance (#2826)
* Add `HashSet`s to help spend conflict detection

Keep track of the spent transparent outpoints and the revealed
nullifiers.

Clippy complained that the `ActiveState` had variants with large size
differences, but that was expected, so I disabled that lint on that
`enum`.

* Clear the `HashSet`s when clearing the mempool

Clear them so that they remain consistent with the set of verified
transactions.

* Use `HashSet`s to check for spend conflicts

Store new outputs into its respective `HashSet`, and abort if a
duplicate output is found.

* Remove inserted outputs when aborting

Restore the `HashSet` to its previous state.

* Remove tracked outputs when removing a transaction

Keep the mempool storage in a consistent state when a transaction is
removed.

* Remove tracked outputs when evicting from mempool

Ensure eviction also keeps the tracked outputs consistent with the
verified transactions.

* Refactor to create a `VerifiedSet` helper type

Move the code to handle the output caches into the new type. Also move
the eviction code to make things a little simpler.

* Refactor to have a single `remove` method

Centralize the code that handles the removal of a transaction to avoid
mistakes.

* Move mempool size limiting back to `Storage`

Because the evicted transactions must be added to the rejected list.

* Remove leftover `dbg!` statement

Leftover from some temporary testing code.

Co-authored-by: teor <teor@riseup.net>

* Remove unnecessary `TODO`

It is more speculation than planning, so it doesn't add much value.

Co-authored-by: teor <teor@riseup.net>

* Fix typo in documentation

The verb should match the subject "transactions" which is plural.

Co-authored-by: teor <teor@riseup.net>

* Add a comment to warn about correctness

There's a subtle but important detail in the implementation that should
be made more visible to avoid mistakes in the future.

Co-authored-by: teor <teor@riseup.net>

* Remove outdated comment

Left-over from the attempt to move the eviction into the `VerifiedSet`.

* Improve comment explaining lint removal

Rewrite the comment explaining why the Clippy lint was ignored.

* Check for spend conflicts in `VerifiedSet`

Refactor to avoid API misuse.

* Test rejected transaction rollback

Using two transactions, perform the same test adding a conflict to both
of them to check if the second inserted transaction is properly
rejected. Then remove any conflicts from the second transaction and add
it again. That should work, because if it doesn't it means that when the
second transaction was rejected it left things it shouldn't in the
cache.

* Test removal of multiple transactions

When removing multiple transactions from the mempool storage, all of the
ones requested should be removed and any other transaction should be
still be there afterwards.

* Increase mempool size to 4, so that spend conflict tests work

If the mempool size is smaller than 4,
these tests don't fail on a trivial removal bug.
Because we need a minimum number of transactions in the mempool
to trigger the bug.

Also commit a proptest seed that fails on a trivial removal bug.
(This seed fails if we remove indexes in order,
because every index past the first removes the wrong transaction.)

* Summarise transaction data in proptest error output

* Summarise spend conflict field data in proptest error output

* Summarise multiple removal field data in proptest error output

And replace the very large proptest debug output with the new summary.

Co-authored-by: teor <teor@riseup.net>
2021-10-10 23:54:46 +00:00
Alfredo Garcia f1718f5c92
Add `zcash_serialized_size()` to `ZcashSerialize` trait (#2824)
* add a zcash_serialized_size()

* add a size field to `UnminedTx`

* refactor zcash_serialized_size() to don't allocate RAM

* improve performance

Co-authored-by: teor <teor@riseup.net>

* clippy

Co-authored-by: teor <teor@riseup.net>
2021-10-06 22:40:11 +00:00
Conrado Gouvea 1ccb2de7c7
Add transaction downloader and verifier (#2679)
* Add transaction downloader

* Changed mempool downloader to be like inbound

* Verifier working (logs result)

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Fix coinbase check for mempool, improve is_coinbase() docs

* Change other downloads.rs docs to reflect the mempool downloads.rs changes

* Change TIMEOUTs to downloads.rs; add docs

* Renamed is_coinbase() to has_valid_coinbase_transaction_inputs() and contains_coinbase_input() to has_any_coinbase_inputs(); reorder checks

* Validate network upgrade for V4 transactions; check before computing sighash (for V5 too)

* Add block_ prefix to downloads and verifier

* Update zebra-consensus/src/transaction.rs

Co-authored-by: teor <teor@riseup.net>

* Add consensus doc; add more Block prefixes

Co-authored-by: teor <teor@riseup.net>
2021-09-02 00:06:20 +00:00
teor dcc0dcd26c
Document consensus rules for version group IDs (#2719)
* Document consensus rules for version group IDs

* Remove spaces at end of lines

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-09-01 17:44:46 +00:00
teor 2ed6679069
Use unmined types for transaction verifier mempool requests and responses (#2666)
* Add `Arc<Transaction>` conversions for Transaction IDs

* Use UnminedTxId as the transaction verifier response type

* Use UnminedTx in transaction verifier mempool requests

* Refactor is_mempool into a transaction verifier request method

* Order derives in alphabetical order

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-08-25 21:02:47 +00:00
teor d7eb01d7f0
Add value pool/balances to non-finalized state (#2656)
* add value balances to non finalized state

* fix 2 tests

* fix remaining constrain issues in tests

* extend value pool test to non finalized

* WIP: fix tests after adding value pools to non-finalized state (#2647)

* Update Chain::eq_internal_state with Chain.value_balance

Also increase the number of cases in its tests,
because they didn't detect this bug.

* Calculate the chain value pool change before `Chain::push`

Code
- store the chain value pool change in `ContextuallyValidBlock`
- convert `PreparedBlock` to `ContextuallyValidBlock` using `with_block_and_spent_utxos`
  (rather than `from` or `into`)
- replace `block_utxos` with `new_outputs` in `PreparedBlock`
- replace `block_utxos` with `chain_value_pool_change` in `ContextuallyValidBlock`

Tests
- create test methods for `PreparedBlock` and `ContextuallyValidBlock`
- use `test_with_zero_chain_pool_change` or `test_with_zero_spent_utxos`
  to make tests pass

* fix conflicts

* build `set_current_value_pool()` only for tests

* remove redundant cfgs

* change cfg of set_current_value_pool()

* Clarify some chain field documentation

* Fix bugs in the non-finalized chain value pool calculations

1. Only revert the chain value pool balances when the tip is popped.
   Don't modify them when the root is finalized.

2. Only update or revert the chain value pool balances once per block.
   (Previously, the block changes were multiplied by the number of *transactions*.)

And make corresponding changes to method names and documentation.

* Add extra proptests to try to identify value balance failures

* Simplify some transaction generation code

* Add extra debugging info to value balance errors

* Actually update non-finalized chain value pools in `UpdateWith`

Previously, we were dropping the updated value pools in the `Ok` result.
So the initial (finalized) chain value pool balances were never modified.

* Rename and document value balance add methods

The names and documentation of these methods were confusing.

* Create genesis-based proptests that check chain value pools

* Increase coverage for some test vectors

* Test each chain value balance calculation for blocks 0-10

* Make continuous blockchain test errors easier to debug

* Test the exact transparent pool values for the first few blocks

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-25 10:57:07 -03:00
teor c608260256
Support witnessed transaction IDs in zebra-network requests and responses (#2638)
* Rename internal network requests for wide transaction IDs

fastmod TransactionsByHash TransactionsById zebra*
fastmod AdvertiseTransactions AdvertiseTransactionIds zebra*
fastmod MempoolTransactions MempoolTransactionIds zebra*
fastmod TransactionHashes TransactionIds zebra*

* Update network transaction request/response comments

* Rename a transaction hash method for wide transaction IDs

fastmod transaction_hashes transaction_ids zebra-network

* Add UnminedTxId methods and conversions for InventoryHash

* Map WtxIds to unmined transaction network messages

Also, use UnminedTxId and UnminedTx in:
* Zebra's internal request and response format, and
* external Zcash network protocol messages.

* Enable WtxId mempool inventory tracking for peers

* Further clarify transaction IDs

* Use Witnessed rather than Wide for transaction IDs

And rename narrow to legacy when it only applies to v1-v4 transactions.
Otherwise, rename it to mined ID.

* Rename a missed binding
* Remove an incorrectly named binding

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-08-18 22:55:24 +00:00
teor 84c5f6189d
Create types for unmined transactions and their IDs (#2634)
* Create new types for unmined transactions and their IDs

* Add accessor methods for the parts of an unmined transaction ID
2021-08-18 15:52:42 +10:00
teor 6c86c8dd0d
Implement a WtxId struct, and use it in Zebra's external network protocol (#2618)
* Make the `AuthDigest` display order match transaction IDs

And derive `Hash`, just like transaction IDs.

Don't derive `serde` for now, because it's not needed.

* Move transaction::Hash test to tests module

* Add a simple AuthDigest display order test

* Add a WtxId type for wide transaction IDs

* Add conversions between transaction IDs and bytes

* Use the WtxId type in external network protocol messages
2021-08-16 21:26:08 +00:00
Conrado Gouvea eadca72e75
Implement ZIP-244 authorizing data commitment (auth_digest) (#2547)
* Implement ZIP-244 authorizing data commitment (auth_digest)

* s/Merke/Merkle/

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Fix AuthDataRoot computation to use padded leaves; add tests

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-08-13 12:58:04 -04:00
teor 76591ceeed
Generate test chains with valid chain value pools (#2597)
* Generate chains with valid chain value pool balances

* Move MAX_PARTIAL_CHAIN_BLOCKS to zebra-chain

* Fix generated value overflow based on the maximum number of values

And split it into its own method.

* Split fix_remaining_value into smaller methods

* Remove unused methods

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-12 12:38:16 +00:00
teor 298ececabe
Refactor value balances to support generated data modification (#2596)
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-10 14:42:02 +00:00
teor f09f2a9022
Check remaining transaction value & make value balance signs match the spec (#2566)
* Make Amount arithmetic more generic

To modify generated amounts, we need some extra operations on `Amount`.

We also need to extend existing operations to both `NonNegative` and
`NegativeAllowed` amounts.

* Add a constrain method for ValueBalance

* Derive Eq for ValueBalance

* impl Neg for ValueBalance

* Make some Amount arithmetic expectations explicit

* Explain why we use i128 for multiplication

And expand the overflow error details.

* Expand Amount::sum error details

* Make amount::Error field order consistent

* Rename an amount::Error variant to Constraint, so it's clearer

* Add specific pool variants to ValueBalanceError

* Update coinbase remaining value consensus rule comment

This consensus rule was updated recently to include coinbase transactions,
but Zebra doesn't check block subsidy or miner fees yet.

* Add test methods for modifying transparent values and shielded value balances

* Temporarily set values and value balances to zero in proptests

In both generated chains and proptests that construct their own transactions.

Using zero values reduces value calculation and value check test coverage.
A future change will use non-zero values, and fix them so the check passes.

* Add extra fields to remaining transaction value errors

* Swap the transparent value balance sign to match shielded value balances

This makes the signs of all the chain value pools consistent.

* Use a NonNegative constraint for transparent values

This fix:
* makes the type signature match the consensus rules
* avoids having to write code to handle negative values

* Allocate total generated transaction input value to outputs

If there isn't enough input value for an output, set it to zero.

Temporarily reduce all generated values to avoid overflow.
(We'll remove this workaround when we calculate chain value balances.)

* Consistently use ValueBalanceError for ValueBalances

* Make the value balance signs match the spec

And rename and document methods so their signs are clearer.

* Convert amount::Errors to specific pool ValueBalanceErrors

* Move some error changes to the next PR

* Add extra info to remaining transaction value errors (#2585)

* Distinguish between overflow and negative remaining transaction value errors

And make some error types cloneable.

* Add methods for updating chain value pools (#2586)

* Move amount::test to amount::tests:vectors

* Make ValueBalance traits more consistent with Amount

- implement Add and Sub variants with Result and Assign
- derive Hash

* Clarify some comments and expects

* Create ValueBalance update methods for blocks and transactions

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-09 14:22:26 -03:00
teor 8f4c3b09ea
Add test methods for modifying orchard shielded data and joinsplits (#2580)
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-07 13:23:32 +00:00
teor faae1c08bf
Stop using prop_filter_map to produce valid sapling shielded data (#2579)
This improves proptest results in CI and locally.

Proptests should be faster, because they are not discarding 1/16 results.

Failures should be minimised more often, improving failure logs,
and generating proptest seeds locally and in CI.

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-08-06 17:32:35 +00:00
teor 4f96a4bb40
Increase coverage for generated chains and proptests (#2540)
* Make legacy chain limit clearer

That way, it doesn't get confused with the coinbase maturity limit.

* Allow 1-5 transactions in each generated block, not always 5

* rustfmt

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-07-30 09:22:10 +10:00