* upgrade docs for new lightwalletd repo zebra is using now
* Update book/src/user/lightwalletd.md
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* add the basics
* add some docs, move code
* upgrade compact formats to https://github.com/zcash/lightwalletd/blob/v0.4.15/walletrpc/compact_formats.proto
* add a test for in sync chain
* test changing to ecc lightwalletd
* revert change of lightwalletd repo (already merged to main)
* add debug log to see whats going on with the test
* change log to tracing::info
* remove log line
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* De-duplicate app_version and user_agent code, rename to build_version
* Make RPC testnet flag forward-compatible with additional testnets
* Fix RPC tests with new argument
* Use "modified" rather than "dirty" for uncommitted changes in build metadata
* Split the vergen version into its own function
* Implement minor and patch database format versions
* Log and update database format versions when opening database
* Refactor the current list of column families into a constant
* Open all available column families, including from future Zebra versions
* Refactor note commitment tree lookups to go through the height methods
* Make Sapling/Orchard note commitment tree lookup forwards compatible
* Ignore errors reading column family lists from disk
* Update format version comments and TODOs
* Correctly log newly created database formats
* Fix a new cargo lint about resolver versions
* cargo clippy --fix --all-features --all-targets
* cargo fmt --all
* Add missing tokio feature in the state, revealed by the new resolver
* Add missing dev dependencies in zebra-node-services
* Add a missing `tokio` feature from PR #6813
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* rename verifiers
* rename `PreparedBlock` to `SemanticallyVerifiedBlock`
* rename `CommitBlock` to `SemanticallyVerifiedBlock`
* rename `FinalizedBlock` to `CheckpointVerifiedBlock`
* rename `CommitFinalizedBlock` to `CommitCheckpointVerifiedBlock`
* rename `FinalizedWithTrees` to `ContextuallyVerifiedBlockWithTrees`
* rename `ContextuallyValidBlock` to `ContextuallyVerifiedBlock`
* change some `finalized` variables or function arguments to `checkpoint_verified`
* fix docs
* document the difference between `CheckpointVerifiedBlock` and `ContextuallyVerifiedBlock`
* fix doc links
* apply suggestions to request
Co-authored-by: Marek <mail@marek.onl>
* apply suggestions to service
Co-authored-by: Marek <mail@marek.onl>
* apply suggestions to finalized_state.rs and write.rs
Co-authored-by: Marek <mail@marek.onl>
* fmt
* change some more variable names
* change a few missing generics
* fix checkpoint log issue
* rename more `prepared` vars `semantically_verified`
* fix test regex
* fix test regex 2
---------
Co-authored-by: Marek <mail@marek.onl>
* Ignore out of order Address Book changes, and restructure the function
* Handle concurrent changes using the connection state machine order
* Handle out of order changes correctly
* Pass times through the call stack so they are consistent in tests
* Add time arguments to tests
* Fix tests that were broken by the address order checks
* fastmod wall_ local_ zebra*
* cargo fmt --all
* Fix a bug in the concurrent change check
* Test all the new apply and skip checks for address changes
* Document more edge cases and increase the concurrency time slightly
* Simplify enum ordering matches
* Fix comment typos
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* fix panic in height
* Update security comments and turn literals into constants
* Add a test-only assertion that the maximum output index can't ever be reached
* Fix a MISSING_BLOCK_ERROR_CODE that was mistakenly 0
* Add production RPC tests with excessive heights
* Add and update snapshots for production RPCs
* Add excessive height tests for most getblocktemplate RPCs
* Use correct snapshot names by running `cargo insta review`
* Fix some RPC tests so they use a populated state
* Use correct snapshot file names by running `cargo insta review`
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* fix panic in height
* Update security comments and turn literals into constants
* Add a test-only assertion that the maximum output index can't ever be reached
* Fix a MISSING_BLOCK_ERROR_CODE that was mistakenly 0
* Add production RPC tests with excessive heights
* Add and update snapshots for production RPCs
* Add excessive height tests for most getblocktemplate RPCs
* Use correct snapshot names by running `cargo insta review`
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add a PeerSocketAddr type which hides its IP address, but shows the port
* Manually replace SocketAddr with PeerSocketAddr where needed
```sh
fastmod SocketAddr PeerSocketAddr zebra-network
```
* Add missing imports
* Make converting into PeerSocketAddr easier
* Fix some unused imports
* Add a canonical_peer_addr() function
* Fix connection handling for PeerSocketAddr
* Fix serialization for PeerSocketAddr
* Fix tests for PeerSocketAddr
* Remove some unused imports
* Fix address book listener handling
* Remove redundant imports and conversions
* Update outdated IPv4-mapped IPv6 address code
* Make addresses canonical when deserializing
* Stop logging peer addresses in RPC code
* Update zebrad tests with new PeerSocketAddr type
* Update zebra-rpc tests with new PeerSocketAddr type
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* add user agent as argument, use git to auto build zebra user agent
* try to fix test
* fix typo
* change expect text
* remove newline
* fix some docs
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* refuse to run Zebra if it is too old
* update the release checklist to consider the constants
* bring newline back
* apply new end of support code
* attempt to add tests (not working yet)
* move eos to progress task
* move tests
* add acceptance test (not working)
* fix tests
* change to block height checks (ugly code)
* change warn days
* refactor estimated blocks per day, etc
* move end of support code to its own task
* change test
* fix some docs
* move constants
* remove uneeded conversions
* downgrade tracing
* reduce end of support time, fix ci changing debugs to info again
* update instructions
* add failure messages
* cargo lock update
* unify releaase name constant
* change info msg
* clippy fixes
* add a block explorer
* ignore testnet in end of support task
* change panic to 16 weeks
* add some documentation about end of support
* Tweak docs wording
---------
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* check last seen tip hash from mempool in fetch_mempool_transactions()
* Moves last_seen_tip_hash to ActiveState
* fixes tests and tests fixes
* continues to the next iteration of the loop to make fresh state and mempool requests if called with a long poll id
* Update zebra-rpc/src/methods/get_block_template_rpcs.rs
Co-authored-by: teor <teor@riseup.net>
* adds allow[unused_variable) for linter
* expects a chain tip when not(test)
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Addresses comments in code review
* - Removes second call to `last_tip_change()`
- Fixes tests using enabled mempool
* Adds note about chain tip action requirement to test method `enable()`
* updates doc comment
* Update zebrad/src/components/mempool.rs
Co-authored-by: teor <teor@riseup.net>
* fixes test
---------
Co-authored-by: teor <teor@riseup.net>
* adds confirmation field to getrawtransaction
* Updates tests
* Adds comment about correctness
* Apply suggestion revisions from review
* fixes overflow bug and adds test for valid confirmations value
* Update test to check that confirmations isn't too high
* Update transaction request to return confirmations
* Applies suggestions from PR review
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* fixes test
* restore derives that were lost in a bad merge
---------
Co-authored-by: teor <teor@riseup.net>
* Mark Zebra coinbase transactions with extra coinbase data
* Log when we commit a block mined by Zebra to our state
* Reduce logging instrumentation during block writes
* Remove debug types in Zebra block log
* Add network and commit to write task logs
* Apply an allow-list before we log arbitrary user messages from blocks
* Rate-limit Zebra mined block logging to once every 1000 blocks
* Add mining configs for extra coinbase data and imitating zcashd, but don't use them yet
* Check CoinbaseData size limit when building transparent transactions
* Replace LIKE_ZCASHD constants with a config
* Take extra coinbase data from the configured string
* Update the zebrad configs in the tests with new config fields
* Add the response type for `z_validateaddress`
* Add UAs to the address variants
* Remove a redundant TODO
* Impl `z_validateaddress`
* Add basic test vectors
* Add basic snapshots
* Remove a deprecated field from the RPC response
* Add basic snapshot data
* Check the semantic validity of UAs
* Refactor imports
* Refactor snapshot filenames
This PR removes the `.new` filename extensions from snapshots.
* Rename `address` to `unified_address`
* Stabilise the BestChainBlockHash state request
* Always include the block hash in the getblock RPC response
* Make the lightwalletd integration tests compatible with zcash/lightwalletd
* Update getblock RPC snapshots
* Return the correct missing block error code
* Use existing Work type to calculate get_difficulty RPC, round in f64 tests
* Use a valid difficulty in all snapshots
* Explain compatibility TODO
* Fix typo
* Document consensus rules in the difficulty module
* Calculate the difficulty using the high 128 bits of the U256 values
* Require 6 significant figures of accuracy in the difficulty unit tests
* fixup! Calculate the difficulty using the high 128 bits of the U256 values
* Update snapshots
* clippy: remove unnecessary return statement
* Add hash, height, and confirmations fields to getblock RPC
* Remove a test that is already checked by snapshots
* Document the performance requirements of the getblock RPC
* Update snapshots, use new naming scheme
* Fix off-by-one error in confirmations
* Fix spelling mistakes
* change error format in proposals
* use trim_matches
* make regex feature dependency
* remove regex fully
* remove char in comment to leave the file as it was before
* remove assertion lines from snapshots
* Make it clearer that Zebra only supports transparent funding streams
* Initial getblocksubsidy RPC types and method, without ZEC conversion
* Add a ZEC fixed-point format wrapper to Amount
* Format getblocksubsidy fields in Zec
* Add snapshot tests for getblocksubsidy RPC
* Re-order RPC output to match zcashd
* Switch to formatting Zec with f64, because the getblocksubsidy RPC requires JSON numbers
* Sort RPC responses in zcashd funding stream order
* Add getblocksubsidy snapshots
* Fix a doc link
* Move Zec JSON formatter from zebra-chain to zebra-rpc
* Remove Ord impl for Zec, it's just for formatting
* Use fully-qualified path for serde derives
* Fix a clippy warning for the doc link fix
* Fix RPC comments and an error message
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Sorts transactions like zcashd in getrawmempool
* Simplifies sort logic and condenses duplicate code
* adds comment clarifying the intended byte order for transaction hashes
* Multiplies fee by MAX_BLOCK_BYTES/tx-size instead of tx-size
- Removes feature flag on get_raw_mempool
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Always uses `TransactionIds` request in tests
* reverts switch from #[cfg()] to cfg!()
* Adds feature flag to constant
* Updates tests and removes !cfg(not(test))
* Moves up comment
* adds missing transaction_ids
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* adds ValidateBlock request to state
* adds `Request` enum in block verifier
skips solution check for BlockProposal requests
calls CheckBlockValidity instead of Commit block for BlockProposal requests
* uses new Request in references to chain verifier
* adds getblocktemplate proposal mode response type
* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state
* Adds PR review revisions
* adds info log in CheckBlockProposalValidity
* Reverts replacement of match statement
* adds `GetBlockTemplate::capabilities` fn
* conditions calling checkpoint verifier on !request.is_proposal
* updates references to validate_and_commit_non_finalized
* adds snapshot test, updates test vectors
* adds `should_count_metrics` to NonFinalizedState
* Returns an error from chain verifier for block proposal requests below checkpoint height
adds feature flags
* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD
* adds back block::Request to zebra-consensus lib
* updates snapshots
* Removes unnecessary network arg
* skips req in tracing intstrument for read state
* Moves out block proposal validation to its own fn
* corrects `difficulty_threshold_is_valid` docs
adds/fixes some comments, adds TODOs
general cleanup from a self-review.
* Update zebra-state/src/service.rs
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Update zebra-rpc/src/methods/get_block_template_rpcs.rs
Co-authored-by: teor <teor@riseup.net>
* check best chain tip
* Update zebra-state/src/service.rs
Co-authored-by: teor <teor@riseup.net>
* Applies cleanup suggestions from code review
* updates gbt acceptance test to make a block proposal
* fixes json parsing mistake
* adds retries
* returns reject reason if there are no retries left
* moves result deserialization to RPCRequestClient method, adds docs, moves jsonrpc_core to dev-dependencies
* moves sleep(EXPECTED_TX_TIME) out of loop
* updates/adds info logs in retry loop
* Revert "moves sleep(EXPECTED_TX_TIME) out of loop"
This reverts commit f7f0926f4050519687a79afc16656c3f345c004b.
* adds `allow(dead_code)`
* tests with curtime, mintime, & maxtime
* Fixes doc comment
* Logs error responses from chain_verifier CheckProposal requests
* Removes retry loop, adds num_txs log
* removes verbose info log
* sorts mempool_txs before generating merkle root
* Make imports conditional on a feature
* Disable new CI tests until bugs are fixed
* adds support for getpeerinfo RPC
* adds vector test
* Always passes address_book to RpcServer
* Update zebra-network/src/address_book.rs
Co-authored-by: teor <teor@riseup.net>
* Renames PeerObserver to AddressBookPeers
* adds getpeerinfo acceptance test
* Asserts that addresses parse into SocketAddr
* adds launches_lightwalletd field to TestType::LaunchWithEmptyState and uses it from the get_peer_info test
* renames peer_observer mod
* uses SocketAddr as `addr` field type in PeerInfo
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Clean up some getblocktemplate difficulty code and tests
* Fix minimum difficulty adjustment
* Use clamp rather than max/min
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove unused imports
* Document the Zebra-specific standard rule that allows testnet miners time to mine a block
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
* adds ValidateBlock request to state
* adds `Request` enum in block verifier
skips solution check for BlockProposal requests
calls CheckBlockValidity instead of Commit block for BlockProposal requests
* uses new Request in references to chain verifier
* adds getblocktemplate proposal mode response type
* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state
* Adds PR review revisions
* adds info log in CheckBlockProposalValidity
* Reverts replacement of match statement
* adds `GetBlockTemplate::capabilities` fn
* conditions calling checkpoint verifier on !request.is_proposal
* updates references to validate_and_commit_non_finalized
* adds snapshot test, updates test vectors
* adds `should_count_metrics` to NonFinalizedState
* Returns an error from chain verifier for block proposal requests below checkpoint height
adds feature flags
* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD
* adds back block::Request to zebra-consensus lib
* updates snapshots
* Removes unnecessary network arg
* skips req in tracing intstrument for read state
* Moves out block proposal validation to its own fn
* corrects `difficulty_threshold_is_valid` docs
adds/fixes some comments, adds TODOs
general cleanup from a self-review.
* Update zebra-state/src/service.rs
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Update zebra-rpc/src/methods/get_block_template_rpcs.rs
Co-authored-by: teor <teor@riseup.net>
* check best chain tip
* Update zebra-state/src/service.rs
Co-authored-by: teor <teor@riseup.net>
* Applies cleanup suggestions from code review
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Make Zebra's getblocktemplate like zcashd's
* Update snapshots
* Add missing docs
* Fix typo
* Sort coinbase outputs by serialized script for zcashd
* Sort excluding the length byte, make transaction order always stable
* Update snapshots
* Explain that `zcashd` doesn't seem to have a fixed transaction order
* Add constants and fix comments for mempool timer / state watch
* Add a best_tip_changed() method to trait ChainTip
* Check for chain tip changes using a future, and return on error
* Ignore state changes before the most recent state fetch
* Add a submit old field to the getblocktemplate RPC during long polling
* Make the submit_old field optional, rather than `null` in JSON
* Update some TODOs
* Add long polling snapshot tests, use RON for deserialized coinbase fields
* Clarify some comments
* Simplify the BestTipChanged future implementations
* Fix some comment typos
* Add some TODOs
* Move and rename height_from_signed_int()
* Move get_block_template() support functions to a submodule
* Fix incorrect P2SH comments and logs
* Split initial checks into their own functions
* Split state fetch into its own function, do some cleanup
* Move get_block_template_opts to get_block_template::parameters
* Fix and simplify test imports
* Rename block_height to next_block_height
* Rename to chain_tip_and_local_time to make it clear what it contains
* Split fetching mempool transactions out, include them in long poll id
* Refactor coinbase generation
* Split default root calculation into a separate function
* Use DateTime32 for getblocktemplate times
* Use typed difficulty fields rather than strings
* Split out a generate coinbase and roots function
* Move GetBlockTemplate construction into a method
* Document what happens to unusual difficulty values
* Clean up some TODOs
* fastmod check_address check_miner_address
* cargo fmt --all
* Fix an incorrect panic message
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Declare support for long polling in the getblocktemplate RPC
* Add long polling input and ID structs
* Split out an update_checksum() function
* Implement LongPollId conversion to and from a string
* Use the LongPollId type in the RPC
* Add a longpollid field to the getblocktemplate parameters and responses, but don't use it yet
* Use multiple RPC threads with the getblocktemplate feature, to enable efficient long polling
* Update RPC snapshots
* Remove the "longpoll" capability, it's for miners, not nodes
* Use the long poll length constant in tests
* Update snapshots
* Remove the "long polling is not supported" error
* Fix minor compilation issues after the merge/rebase
* Expand long poll id comments
* Rename estimated height to local height, because that's what it actually is
* Add an invalid params test and fix the long poll id test
* Add modified config for config_tests
* Instrument all the config sub-tests
* Show the missing config file when the test fails
* Fix the generated config file
* Allow a clippy lint in tests
* Explain conversion from bytes to u32
* Remove a duplicate test case
* adds type and stub
* adds:
- SolutionRate state request
- getnetworksolps, getnetworkhashps, & getmininginfo RPCs
- vectors tests
* adds snapshot tests
updates ReadRequest::SolutionRate doc link
* removes random slash in doc comment
moves snapshot tests up where it can use the populated state service
* adds snapshots
* updates doc comments
* applies `num_blocks` default in RPC instead of `solution_rate`
* adds # Correctness comment
* Add testnet field to getmininginfo response
* use PartialCumulativeWork instead of u128
* document why `solution_rate` takes an extra block
* add comment explaining why the work for the last block in the iterator is not added to `total_work`
* use `as_u128` method instead of deref for PartialCumulativeWork
* Updates `chain` field of getmininginfo response
* Updates snapshots
Adds "arbitrary_precision" feature to serde_json in zebra-rpc
* populate `blockcommitmenthash` and `defaultroot` missing fields
* remove assertion line manually from snaps
* fix some imports and docs
* fix some docs
* add a consistency check
* Rename a constant to FINALIZED_STATE_QUERY_RETRIES and use it everywhere
* Move tip query inside retry, split tip into tip_height and tip_hash
* Return retry failures rather than panicking
* Query relevant chain inside the retry
* Check the entire state for consistency, not just the finalized tip
Co-authored-by: teor <teor@riseup.net>
* Adds ignored jsonrequestobject argument
* adds docs
returns an error for unsupported options
* returns an error when longpollid is provided
* comments out capabilities enum and uses Vec<String> instead
* fixes spelling mistake
* uncomments GetBlockTemplateCapability enum and uses allow(dead_code) instead
* adds UnknownCapability
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* adds derived traits.
* removes unused [allow(dead_code)]
* test invalid params error for block data
* add capablity to gbt vectors test
* reverts passing in default options to getblocktemplate request in vectors test
* adds a jsonrequestobject with an unknown capability to gbt acceptance test
Co-authored-by: teor <teor@riseup.net>
* Mine both standard and min difficulty blocks on testnet
* Add a POW_ADJUSTMENT_BLOCK_SPAN and fix an incorrect assertion
* Split the testnet adjustment into its own function
* Clarify a panic message
* Fix comments
* populate block height
* populate cur_time
* populate min_time
* populate capabilities
* populate last_block_hash
* create read state request for getblocktemplate
* refactor to get difficulty fields more properly
* populate bits and target fields
* fix tests
* add target and bits documentation
* docs
* fix docs
* docs
* remove metrixs counter calls
* apply some suggestions from code review
* hide some code behind feature
* simplify the service
* fix error handling
* remove comment
* fox doc
* panic if we dont have enough state
* bring tip data from the state
* make proposal empty
* fix time
* fix docs, consensus rules
* remove non used anymore fn
* remove another non used fn
* remove no needed change
* remove more unused changes
* remove unused anymore change
* apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* fix build and snapshots
* apply testnet consensus rule
* fix test
* rustfmt
* remove time as allowed field to be modified by the miner if mining minimum difficulty block
* move all times to before calculating difficulty
* do some cleanup
* Adjust times so the whole time range is a testnet minimum difficulty block
* Return a GetBlockTemplateChainInfo struct from the difficulty calculation
* Add a Zebra-only max_time field to the getblocktemplate RPC
Co-authored-by: teor <teor@riseup.net>
Alfredo Garcia
Arya
Pili Guerra
teor
Marek