* Initial async RFC version
* Add a table of contents
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add a toc anchor
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add some words that need definitions
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Write guide intro based on feedback
* Add a code example for each reference section
* Link to code examples using commit hashes
* Link to PR and commit for each code example
* Fix typos
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Remove redundant version in docs.rs link
* Link the guide to the reference
And expand the guide descriptions
* Mention TurboWish as a future diagnostic tool
* Add an example of a compiler error that prevents deadlock
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Redesign Sapling data model for V5 shared anchor and spends
The shared anchor is only present if there are any spends.
As part of this change, delete the manual PartialEq impl and its tests,
because we can derive PartialEq now.
* Stop creating a temporary Vec for the spend and output iterators
* Rename TransferData variants
Interactive rename using the following commands:
```sh
fastmod Spends SpendsAndMaybeOutputs
fastmod NoSpends JustOutputs
```
* Refactor out common sprout nullifier code
* Implement the AtLeastOne constrained vector type
This vector wrapper ensures that it always contains at least one element.
* Simplify Sapling TransferData using AtLeastOne
Also update the RFC to use AtLeastOne for Orchard.
* add sapling shielded data to transaction V5
* implement nullifiers
* test v5 in shielded_data_roundtrip
* Explicitly design serialization for Transaction V5
Implement serialization for V4 and V5 spends and outputs, to make sure
that the design works.
* Test serialization for v5 spends and outputs
Also add a few missing v4 tests.
* Delete a disabled proptest
* Make v5 transactions a top-level heading
And add a missing serialized type.
* Fix a comment typo
* v5 transaction RFC: split array serialization
Based on #2017
* RFC: explicitly describe serialized field order
And link to the spec
* RFC: add the shared anchor serialization rule test
Co-authored-by: teor <teor@riseup.net>
* create block subsidy rfc
* add reference in miner subsidy
* remove `or 0` from function descriptions.
It is clear from the signature that the function will return `Error` on any failure(including input errors, for example a wrong height). Describing this will force us to write it for all functions that returns an error so better remove it.
* add links to subsidy categories
* fix design in Funding streams parameter constants
* fix order in FUNDING_STREAM_RECEIVER_NUMERATORS
* return `PayToScriptHash` in `funding_stream_address`
* add shielded coinbase
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
* improve transparent value pool section
* add a test plan section
* rename `subsidy_is_correct` to `subsidy_is_valid`
* add funding streams address constants
* fix funding stream addresses
* add errors
* remove ending dot for error descriptions
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* modify founders reward
* change ECC to BP
* add note about state needed for transparent pool
* split constants
* Tweak block subsidy RFC wording
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* Clarify transaction fees definition
* Use consistent fee terminology
* Clarify transparent value pool
* Revise implementation plan based on latest priorities
* Change tests based on new priorities
* Fix markdown
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Fix value pool description
* change block and miner subsidy definitions
* fee pool definitions, fix all links to protocol
* add reference to zip209
* add note for MG stream number of addresses
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* rfc: initial draft of release process proposal
This is a version of the notes I posted in slack for preliminary
discussion, slightly reformatted to fit the Zebra RFC template.
@yaahc suggested I post them as an RFC, to have a concrete proposal for
discussion.
* Add draft note and link to ticket and PR
Co-authored-by: teor <teor@riseup.net>
When we're naming the anchors without the corresponding type or struct,
it's not clear if they are shared or per spend.
Rename the fields as `shared_anchor` or `per_spend_anchor`.
* propose a v5 transaction rfc
* define orchard flags
* Update test sections
* Add security section
* move some code into sapling and orchard crates, do renames
* Move sapling value balance into shielded data
* Add AuthorizedAction type
* Order fields based on last serialized data
* Add the proofsOrchard field
* Delete visibility modifiers for readability
All those `pub`s make the design harder to read.
* Model anchor variants as generic trait types
* Be specific about the network upgrade
* Specify a file for each new type
* Document how the Orchard flags are used
Co-authored-by: teor <teor@riseup.net>
* Difficulty Contextual RFC: Introduction
Add a header, summary, and motivation
* Difficulty RFC: Add draft definitions
And update the state RFC definitions to match.
* Difficulty RFC: Add relevant chain
* Difficulty RFC: draft guide-level explanation
Outline the core calculations and checks.
* Difficulty RFC: Revised based on spec fixes
Update the design based on the spec bugs in #1276, #1277, and
zcash/zips#416.
These changes make the difficulty filter into a context-free check,
so we remove it from this contextual validation RFC.
* Difficulty RFC: Explain how Zebra's calculations can match the spec
* Difficulty RFC: write most of the reference section
Includes most of the implementation, modules for each function, and
draft notes for some of the remaining parts of the RFC.
* Difficulty RFC: Add an AdjustedDifficulty struct
* Difficulty RFC: Summarise module structure in the one place
* Difficulty RFC: Create implementation notes subsections
* Difficulty RFC: add consensus critical order of operations
* Difficulty RFC: Use the ValidateContextError type
* Difficulty RFC: make the median_time arg mut owned
We have to clone the data to pass a fixed-length array to a function,
so we might as well sort that array to find the median, and avoid a
copy.
## Motivation
Prior to this PR we've been using `sled` as our database for storing persistent chain data on the disk between boots. We picked sled over rocksdb to minimize our c++ dependencies despite it being a less mature codebase. The theory was if it worked well enough we'd prefer to have a pure rust codebase, but if we ever ran into problems we knew we could easily swap it out with rocksdb.
Well, we ran into problems. Sled's memory usage was particularly high, and it seemed to be leaking memory. On top of all that, the performance for writes was pretty poor, causing us to become bottle-necked on sled instead of the network.
## Solution
This PR replaces `sled` with `rocksdb`. We've seen a 10x improvement in memory usage out of the box, no more leaking, and much better write performance. With this change writing chain data to disk is no longer a limiting factor in how quickly we can sync the chain.
The code in this pull request has:
- [x] Documentation Comments
- [x] Unit Tests and Property Tests
## Review
@hdevalence
Prior to this PR we realized that the RFC had been drafted with the assumption that chains would be ordered from best to worst in `NonFinalizedState`. This assumption was incorrect, since `BTreeSet` only ever orders values in ascending order. This discrepancy was noticed and fixed in the code, but there were still some inconsistencies that needed to be cleaned up.
This PR updates all the incorrect or confusing comments about chain ordering in the RFC and code.
The difficulty validation RFC will introduce a definition of
per-block difficulty. Make it clear that the state RFC
definition is cumulative difficulty.
## Motivation
The zebra-state service needs to be able to handle duplicate blocks.
## Solution
This implements changes already outlined by [The State
RFC](https://zebra.zfnd.org/dev/rfcs/0005-state-updates.html). We check for
successfully committed blocks first, since interacting with the queued blocks
struct at this point just complicates the implimentation. If the block has not
already been committed we then check if the block has already been queued, if
not we handle the block normally (normally here being the bit we already had
implemented).
## Documentation Changes
- [x] Update the state RFC to match the ways this fix departs from the design
- the main thing is that I switched the order of checking for duplicates
- [x] ~~Add newly added functions to the state rfc~~ Decided not to do this because they're minor getters that don't influence the rest of the design and aren't exposed as part of the API
- [x] Document newly added functions inline
## Testing
## Related Issues
- fixes https://github.com/ZcashFoundation/zebra/issues/1182
- tracking issue https://github.com/ZcashFoundation/zebra/issues/1049
Co-authored-by: teor <teor@riseup.net>
* Reverse displayed endianness of transaction and block hashes
* fix zebra-checkpoints utility for new hash order
* Stop using "zebrad revhex" in zebrad-hash-lookup
* Rebuild checkpoint lists in new hash order
This change also adds additional checkpoints to the end of each list.
* Replace TransactionHash with transaction::Hash
This change should have been made in #905, but we missed Debug impls
and some docs.
Co-authored-by: Ramana Venkata <vramana@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
* implement most of the chain functions
* implement fork
* fix outpoint handling in Chain struct
* update expect for work
* split utxo into two sets
* update the Chain definition
* remove allow attribute in zebra-state/lib.rs
* merge ChainSet type into MemoryState
* Add error messages to asserts
* export proptest impls for use in downstream crates
* add testjob for disabled feature in zebra-chain
* try to fix github actions syntax
* add module doc comment
* update RFC for utxos
* add missing header
* working proptest for Chain
* propagate back results over channel
* Start updating RFC to match changes
* implement queued block pruning
* and now it syncs wooo!
* remove empty modules
* setup config for proptests
* re-enable missing_docs lint
* update RFC to match changes in impl
* add documentation
* use more explicit variable names
teor
Alfredo Garcia
Deirdre Connolly
Henry de Valence
Jane Lusby
Jane Lusby