octo
/
Zebra
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Implement RFC5: State updates `Chain` type (#1069) 

* Begin work on RFC5 implementation

* I think this is necessary

* holy shit supertrait implemented via subtrait

* implement most of the chain functions

* change to slightly better name

* implement fork

* fix outpoint handling in Chain struct

* update expect for work

* resolve review comment

* split utxo into two sets

* update the Chain definition

* just a little more

* update comment

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* apply changes from code review

* remove allow attribute in zebra-state/lib.rs

* Update zebra-state/src/memory_state.rs

Co-authored-by: teor <teor@riseup.net>

* merge ChainSet type into MemoryState

* rename state impl types

* Add error messages to asserts

* add module doc comment

* update RFC for utxos

* add missing header

Co-authored-by: teor <teor@riseup.net>
This commit is contained in:
Jane Lusby 2020-09-24 15:46:04 -07:00 committed by GitHub
parent d5ce5eeee2
commit 352721bd88
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 609 additions and 76 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
book/src/dev/rfcs/0004-asynchronous-script-verification.md
View File

@ -19,15 +19,16 @@ and in parallel on a thread pool.
# Definitions
[definitions]: #definitions
- *UTXO*: unspent transaction output. Transaction outputs are modeled in `zebra-chain` by the [`TransparentOutput`][transout] structure.
- Transaction input: an output of a previous transaction consumed by a later transaction (the one it is an input to). Modeled in `zebra-chain` by the [`TransparentInput`][transin] structure.
- lock script: the script that defines the conditions under which some UTXO can be spent. Stored in the [`TransparentOutput::lock_script`][lock_script] field.
- unlock script: a script satisfying the conditions of the lock script, allowing a UTXO to be spent. Stored in the [`TransparentInput::PrevOut::lock_script`][lock_script] field.
- *UTXO*: unspent transaction output. Transaction outputs are modeled in `zebra-chain` by the [`transparent::Output`][transout] structure.
- Transaction input: an output of a previous transaction consumed by a later transaction (the one it is an input to). Modeled in `zebra-chain` by the [`transparent::Input`][transin] structure.
- lock script: the script that defines the conditions under which some UTXO can be spent. Stored in the [`transparent::Output::lock_script`][lock_script] field.
- unlock script: a script satisfying the conditions of the lock script, allowing a UTXO to be spent. Stored in the [`transparent::Input::PrevOut::lock_script`][lock_script] field.
[transout]: https://doc.zebra.zfnd.org/zebra_chain/transparent/struct.Output.html
[lock_script]: https://doc.zebra.zfnd.org/zebra_chain/transparent/struct.Output.html#structfield.lock_script
[transin]: https://doc.zebra.zfnd.org/zebra_chain/transparent/enum.Input.html
[unlock_script]: https://doc.zebra.zfnd.org/zebra_chain/transparent/enum.Input.html#variant.PrevOut.field.unlock_script
[transout]: https://doc.zebra.zfnd.org/zebra_chain/transaction/struct.TransparentOutput.html
[lock_script]: https://doc.zebra.zfnd.org/zebra_chain/transaction/struct.TransparentOutput.html#structfield.lock_script
[transin]: https://doc.zebra.zfnd.org/zebra_chain/transaction/enum.TransparentInput.html
[unlock_script]: https://doc.zebra.zfnd.org/zebra_chain/transaction/enum.TransparentInput.html#variant.PrevOut.field.unlock_script
# Guide-level explanation
[guide-level-explanation]: #guide-level-explanation
@ -55,8 +56,8 @@ done later, at the point that its containing block is committed to the chain.
At a high level, this adds a new request/response pair to the state service:
- `Request::AwaitUtxo(OutPoint)` requests a `TransparentOutput` specified by `OutPoint` from the state layer;
- `Response::Utxo(TransparentOutput)` supplies requested the `TransparentOutput`.
- `Request::AwaitUtxo(OutPoint)` requests a `transparent::Output` specified by `OutPoint` from the state layer;
- `Response::Utxo(transparent::Output)` supplies requested the `transparent::Output`.
Note that this request is named differently from the other requests,
`AwaitUtxo` rather than `GetUtxo` or similar. This is because the request has
@ -82,7 +83,7 @@ parallelism.
[reference-level-explanation]: #reference-level-explanation
We add a `Request::AwaitUtxo(OutPoint)` and
`Response::Utxo(TransparentOutput)` to the state protocol. As described
`Response::Utxo(transparent::Output)` to the state protocol. As described
above, the request name is intended to indicate the request's behavior: the
request does not resolve until the state layer learns of a UTXO described by
the request.
@ -163,7 +164,7 @@ structure described below.
```rust
// sketch
#[derive(Default, Debug)]
struct PendingUtxos(HashMap<OutPoint, oneshot::Sender<TransparentOutput>>);
struct PendingUtxos(HashMap<OutPoint, oneshot::Sender<transparent::Output>>);
impl PendingUtxos {
// adds the outpoint and returns (wrapped) rx end of oneshot
@ -171,7 +172,7 @@ impl PendingUtxos {
pub fn queue(&mut self, outpoint: OutPoint) -> impl Future<Output=Result<Response, ...>>;
// if outpoint is a hashmap key, remove the entry and send output on the channel
pub fn respond(&mut self, outpoint: OutPoint, output: TransparentOutput);
pub fn respond(&mut self, outpoint: OutPoint, output: transparent::Output);
// scans the hashmap and removes any entries with closed senders

104
book/src/dev/rfcs/0005-state-updates.md
View File

@ -264,16 +264,18 @@ is completely empty.
The `Chain` type is defined by the following struct and API:
```rust
#[derive(Debug, Default, Clone)]
struct Chain {
blocks: BTreeMap<block::Height, Arc<Block>>,
height_by_hash: HashMap<block::Hash, block::Height>,
tx_by_hash: HashMap<transaction::Hash, (block::Height, tx_index)>,
tx_by_hash: HashMap<transaction::Hash, (block::Height, usize)>,
utxos: HashSet<transparent::Output>,
sapling_anchors: HashSet<sapling::tree::Root>,
created_utxos: HashSet<transparent::OutPoint>,
spent_utxos: HashSet<transparent::OutPoint>,
sprout_anchors: HashSet<sprout::tree::Root>,
sapling_nullifiers: HashSet<sapling::Nullifier>,
sapling_anchors: HashSet<sapling::tree::Root>,
sprout_nullifiers: HashSet<sprout::Nullifier>,
sapling_nullifiers: HashSet<sapling::Nullifier>,
partial_cumulative_work: PartialCumulativeWork,
}
```
@ -283,14 +285,16 @@ struct Chain {
Push a block into a chain as the new tip
1. Update cumulative data members
- Add block to end of `self.blocks`
- Add hash to `height_by_hash`
- for each `transaction` in `block`
- add key: `transaction.hash` and value: `(height, tx_index)` to `tx_by_hash`
- Add new utxos and remove consumed utxos from `self.utxos`
- Add anchors to the appropriate `self.<version>_anchors`
- Add nullifiers to the appropriate `self.<version>_nullifiers`
- Add the block's hash to `height_by_hash`
- Add work to `self.partial_cumulative_work`
- For each `transaction` in `block`
- Add key: `transaction.hash` and value: `(height, tx_index)` to `tx_by_hash`
- Add created utxos to `self.created_utxos`
- Add spent utxos to `self.spent_utxos`
- Add anchors to the appropriate `self.<version>_anchors`
- Add nullifiers to the appropriate `self.<version>_nullifiers`
2. Add block to `self.blocks`
#### `pub fn pop_root(&mut self) -> Arc<Block>`
@ -300,11 +304,13 @@ Remove the lowest height block of the non-finalized portion of a chain.
2. Update cumulative data members
- Remove the block's hash from `self.height_by_hash`
- for each `transaction` in `block`
- remove `transaction.hash` from `tx_by_hash`
- Remove new utxos from `self.utxos`
- Remove the anchors from the appropriate `self.<version>_anchors`
- Remove the nullifiers from the appropriate `self.<version>_nullifiers`
- Subtract work from `self.partial_cumulative_work`
- For each `transaction` in `block`
- Remove `transaction.hash` from `tx_by_hash`
- Remove created utxos from `self.created_utxos`
- Remove spent utxos from `self.spent_utxos`
- Remove the anchors from the appropriate `self.<version>_anchors`
- Remove the nullifiers from the appropriate `self.<version>_nullifiers`
3. Return the block
@ -321,7 +327,7 @@ Fork a chain at the block with the given hash, if it is part of this chain.
4. Return `forked`
#### `fn pop_tip(&mut self) -> Arc<Block>`
#### `fn pop_tip(&mut self)`
Remove the highest height block of the non-finalized portion of a chain.
@ -329,14 +335,13 @@ Remove the highest height block of the non-finalized portion of a chain.
2. Update cumulative data members
- Remove the corresponding hash from `self.height_by_hash`
- Subtract work from `self.partial_cumulative_work`
- for each `transaction` in `block`
- remove `transaction.hash` from `tx_by_hash`
- Add consumed utxos and remove new utxos from `self.utxos`
- Remove anchors from the appropriate `self.<version>_anchors`
- Remove the nullifiers from the appropriate `self.<version>_nullifiers`
- Subtract work from `self.partial_cumulative_work`
3. Return the block
- Remove created utxos from `self.created_utxos`
- Remove spent utxos from `self.spent_utxos`
- Remove anchors from the appropriate `self.<version>_anchors`
- Remove the nullifiers from the appropriate `self.<version>_nullifiers`
#### `Ord`
@ -358,7 +363,8 @@ handled by `#[derive(Default)]`.
1. initialise cumulative data members
- Construct an empty `self.blocks`, `height_by_hash`, `tx_by_hash`,
`self.utxos`, `self.<version>_anchors`, `self.<version>_nullifiers`
`self.created_utxos`, `self.spent_utxos`, `self.<version>_anchors`,
`self.<version>_nullifiers`
- Zero `self.partial_cumulative_work`
**Note:** The chain can be empty if:
@ -367,23 +373,35 @@ handled by `#[derive(Default)]`.
all its blocks have been `pop`ped
### `ChainSet` Type
[chainset-type]: #chainset-type
### `NonFinalizedState` Type
[nonfinalizedstate-type]: #nonfinalizedstate-type
The `ChainSet` type represents the set of all non-finalized state. It
consists of a set of non-finalized but verified chains and a set of
The `NonFinalizedState` type represents the set of all non-finalized state.
It consists of a set of non-finalized but verified chains and a set of
unverified blocks which are waiting for the full context needed to verify
them to become available.
`ChainSet` is defined by the following structure and API:
`NonFinalizedState` is defined by the following structure and API:
```rust
struct ChainSet {
chains: BTreeSet<Chain>,
/// The state of the chains in memory, incuding queued blocks.
#[derive(Debug, Default)]
pub struct NonFinalizedState {
/// Verified, non-finalized chains.
chain_set: BTreeSet<Chain>,
/// Blocks awaiting their parent blocks for contextual verification.
contextual_queue: QueuedBlocks,
}
queued_blocks: BTreeMap<block::Hash, QueuedBlock>,
queued_by_parent: BTreeMap<block::Hash, Vec<block::Hash>>,
queued_by_height: BTreeMap<block::Height, Vec<block::Hash>>,
/// A queue of blocks, awaiting the arrival of parent blocks.
#[derive(Debug, Default)]
struct QueuedBlocks {
/// Blocks awaiting their parent blocks for contextual verification.
blocks: HashMap<block::Hash, QueuedBlock>,
/// Hashes from `queued_blocks`, indexed by parent hash.
by_parent: HashMap<block::Hash, Vec<block::Hash>>,
/// Hashes from `queued_blocks`, indexed by block height.
by_height: BTreeMap<block::Height, Vec<block::Hash>>,
}
```
@ -470,10 +488,10 @@ cannot be committed due to missing context.
- `Chain` represents the non-finalized portion of a single chain
- `ChainSet` represents the non-finalized portion of all chains and all
- `NonFinalizedState` represents the non-finalized portion of all chains and all
unverified blocks that are waiting for context to be available.
- `ChainSet::queue` handles queueing and or commiting blocks and
- `NonFinalizedState::queue` handles queueing and or commiting blocks and
reorganizing chains (via `commit_block`) but not finalizing them
- Finalized blocks are returned from `finalize` and must still be committed
@ -759,6 +777,20 @@ if the block is not in any non-finalized chain:
the `block_by_height` tree (to get the block data).
### `Request::AwaitUtxo(OutPoint)`
Returns
- `Response::Utxo(transparent::Output)`
Implemented by querying:
- (non-finalized) if any `Chains` contain an `OutPoint` in their `created_utxos` and not their `spent_utxo` get the `transparent::Output` from `OutPoint`'s transaction
- (finalized) else if `OutPoint` is in `utxos_by_outpoint` return the associated `transparent::Output`.
- else wait for `OutPoint` to be created as described in [RFC0004]
[RFC0004]: https://zebra.zfnd.org/dev/rfcs/0004-asynchronous-script-verification.html
# Drawbacks
[drawbacks]: #drawbacks

4
zebra-chain/src/amount.rs
View File

@ -17,7 +17,7 @@ use byteorder::{ByteOrder, LittleEndian, ReadBytesExt, WriteBytesExt};
type Result<T, E = Error> = std::result::Result<T, E>;
/// A runtime validated type for representing amounts of zatoshis
#[derive(Debug, Eq, PartialEq, Clone, Copy, Serialize, Deserialize)]
#[derive(Debug, Eq, PartialEq, Clone, Copy, Serialize, Deserialize, Hash)]
#[serde(try_from = "i64")]
#[serde(bound = "C: Constraint")]
pub struct Amount<C = NegativeAllowed>(i64, PhantomData<C>);
@ -234,7 +234,7 @@ impl Constraint for NegativeAllowed {
/// 0..=MAX_MONEY,
/// );
/// ```
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
#[derive(Clone, Copy, Debug, Eq, PartialEq, Hash)]
pub struct NonNegative {}
impl Constraint for NonNegative {

2
zebra-chain/src/sapling/note/nullifiers.rs
View File

@ -26,7 +26,7 @@ fn prf_nf(nk: [u8; 32], rho: [u8; 32]) -> [u8; 32] {
}
/// A Nullifier for Sapling transactions
#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize, Deserialize)]
#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize, Deserialize, Hash)]
#[cfg_attr(
any(test, feature = "proptest-impl"),
derive(proptest_derive::Arbitrary)

2
zebra-chain/src/sapling/tree.rs
View File

@ -62,7 +62,7 @@ struct SaplingNoteCommitmentTree;
/// commitment tree corresponding to the final Sapling treestate of
/// this block. A root of a note commitment tree is associated with
/// each treestate.
#[derive(Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize)]
#[derive(Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize, Hash)]
#[cfg_attr(any(test, feature = "proptest-impl"), derive(Arbitrary))]
pub struct Root(pub [u8; 32]);

2
zebra-chain/src/sprout/note/nullifiers.rs
View File

@ -62,7 +62,7 @@ impl From<NullifierSeed> for [u8; 32] {
}
/// A Nullifier for Sprout transactions
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, Hash)]
#[cfg_attr(
any(test, feature = "proptest-impl"),
derive(proptest_derive::Arbitrary)

2
zebra-chain/src/sprout/tree.rs
View File

@ -22,7 +22,7 @@ use proptest_derive::Arbitrary;
/// commitment tree corresponding to the final Sprout treestate of
/// this block. A root of a note commitment tree is associated with
/// each treestate.
#[derive(Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize)]
#[derive(Clone, Copy, Default, Eq, PartialEq, Serialize, Deserialize, Hash)]
#[cfg_attr(any(test, feature = "proptest-impl"), derive(Arbitrary))]
pub struct Root([u8; 32]);

4
zebra-chain/src/transaction.rs
View File

@ -93,10 +93,10 @@ pub enum Transaction {
expiry_height: block::Height,
/// The net value of Sapling spend transfers minus output transfers.
value_balance: Amount,
/// The shielded data for this transaction, if any.
shielded_data: Option<ShieldedData>,
/// The JoinSplit data for this transaction, if any.
joinsplit_data: Option<JoinSplitData<Groth16Proof>>,
/// The shielded data for this transaction, if any.
shielded_data: Option<ShieldedData>,
},
}

4
zebra-chain/src/transparent.rs
View File

@ -40,7 +40,7 @@ impl AsRef<[u8]> for CoinbaseData {
/// OutPoint
///
/// A particular transaction output reference.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Serialize, Deserialize)]
#[derive(Copy, Clone, Debug, Eq, PartialEq, Serialize, Deserialize, Hash)]
#[cfg_attr(any(test, feature = "proptest-impl"), derive(Arbitrary))]
pub struct OutPoint {
/// References the transaction that contains the UTXO being spent.
@ -86,7 +86,7 @@ pub enum Input {
/// I only own one UTXO worth 2 ZEC, I would construct a transaction
/// that spends my UTXO and sends 1 ZEC to you and 1 ZEC back to me
/// (just like receiving change).
#[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)]
#[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize, Hash)]
#[cfg_attr(any(test, feature = "proptest-impl"), derive(Arbitrary))]
pub struct Output {
/// Transaction value.

2
zebra-chain/src/transparent/script.rs
View File

@ -8,7 +8,7 @@ use std::{
};
/// An encoding of a Bitcoin script.
#[derive(Clone, Eq, PartialEq, Serialize, Deserialize)]
#[derive(Clone, Eq, PartialEq, Serialize, Deserialize, Hash)]
#[cfg_attr(
any(test, feature = "proptest-impl"),
derive(proptest_derive::Arbitrary)

41
zebra-chain/src/work/difficulty.rs
View File

@ -320,3 +320,44 @@ impl AddAssign for Work {
*self = *self + rhs;
}
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, PartialOrd, Ord)]
/// Partial work used to track relative work in non-finalized chains
pub struct PartialCumulativeWork(u128);
impl std::ops::Add<Work> for PartialCumulativeWork {
type Output = PartialCumulativeWork;
fn add(self, rhs: Work) -> Self::Output {
let result = self
.0
.checked_add(rhs.0)
.expect("Work values do not overflow");
PartialCumulativeWork(result)
}
}
impl std::ops::AddAssign<Work> for PartialCumulativeWork {
fn add_assign(&mut self, rhs: Work) {
*self = *self + rhs;
}
}
impl std::ops::Sub<Work> for PartialCumulativeWork {
type Output = PartialCumulativeWork;
fn sub(self, rhs: Work) -> Self::Output {
let result = self.0
.checked_sub(rhs.0)
.expect("PartialCumulativeWork values do not underflow: all subtracted Work values must have been previously added to the PartialCumulativeWork");
PartialCumulativeWork(result)
}
}
impl std::ops::SubAssign<Work> for PartialCumulativeWork {
fn sub_assign(&mut self, rhs: Work) {
*self = *self - rhs;
}
}

4
zebra-state/src/lib.rs
View File

@ -19,9 +19,9 @@ mod util;
#[cfg(test)]
mod tests;
use memory_state::MemoryState;
use memory_state::NonFinalizedState;
use service::QueuedBlock;
use sled_state::SledState;
use sled_state::FinalizedState;
pub use config::Config;
pub use request::{HashOrHeight, Request};

462
zebra-state/src/memory_state.rs
View File

@ -1,3 +1,461 @@
pub struct MemoryState {
// TODO
//! Non-finalized chain state management as defined by [RFC0005]
//!
//! [RFC0005]: https://zebra.zfnd.org/dev/rfcs/0005-state-updates.html
#![allow(dead_code)]
use std::{
cmp::Ordering,
collections::BTreeSet,
collections::{BTreeMap, HashMap, HashSet},
ops::Deref,
sync::Arc,
};
use zebra_chain::{
block::{self, Block},
primitives::Groth16Proof,
sapling, sprout, transaction, transparent,
work::difficulty::PartialCumulativeWork,
};
use crate::service::QueuedBlock;
/// The state of the chains in memory, incuding queued blocks.
#[derive(Debug, Default)]
pub struct NonFinalizedState {
/// Verified, non-finalized chains.
chain_set: BTreeSet<Chain>,
/// Blocks awaiting their parent blocks for contextual verification.
contextual_queue: QueuedBlocks,
}
/// A queue of blocks, awaiting the arrival of parent blocks.
#[derive(Debug, Default)]
struct QueuedBlocks {
/// Blocks awaiting their parent blocks for contextual verification.
blocks: HashMap<block::Hash, QueuedBlock>,
/// Hashes from `queued_blocks`, indexed by parent hash.
by_parent: HashMap<block::Hash, Vec<block::Hash>>,
/// Hashes from `queued_blocks`, indexed by block height.
by_height: BTreeMap<block::Height, Vec<block::Hash>>,
}
impl NonFinalizedState {
pub fn finalize(&mut self) -> Arc<Block> {
todo!()
}
pub fn queue(&mut self, _block: QueuedBlock) {
todo!()
}
fn process_queued(&mut self, _new_parent: block::Hash) {
todo!()
}
fn commit_block(&mut self, _block: QueuedBlock) -> Option<block::Hash> {
todo!()
}
}
#[derive(Debug, Default, Clone)]
struct Chain {
blocks: BTreeMap<block::Height, Arc<Block>>,
height_by_hash: HashMap<block::Hash, block::Height>,
tx_by_hash: HashMap<transaction::Hash, (block::Height, usize)>,
created_utxos: HashSet<transparent::OutPoint>,
spent_utxos: HashSet<transparent::OutPoint>,
sprout_anchors: HashSet<sprout::tree::Root>,
sapling_anchors: HashSet<sapling::tree::Root>,
sprout_nullifiers: HashSet<sprout::Nullifier>,
sapling_nullifiers: HashSet<sapling::Nullifier>,
partial_cumulative_work: PartialCumulativeWork,
}
impl Chain {
/// Push a contextually valid non-finalized block into a chain as the new tip.
pub fn push(&mut self, block: Arc<Block>) {
let block_height = block
.coinbase_height()
.expect("valid non-finalized blocks have a coinbase height");
// update cumulative data members
self.update_chain_state_with(&block);
self.blocks.insert(block_height, block);
}
/// Remove the lowest height block of the non-finalized portion of a chain.
pub fn pop_root(&mut self) -> Arc<Block> {
let block_height = self.lowest_height();
// remove the lowest height block from self.blocks
let block = self
.blocks
.remove(&block_height)
.expect("only called while blocks is populated");
// update cumulative data members
self.revert_chain_state_with(&block);
// return the block
block
}
fn lowest_height(&self) -> block::Height {
self.blocks
.keys()
.next()
.cloned()
.expect("only called while blocks is populated")
}
/// Fork a chain at the block with the givin hash, if it is part of this
/// chain.
pub fn fork(&self, fork_tip: block::Hash) -> Option<Self> {
if !self.height_by_hash.contains_key(&fork_tip) {
return None;
}
let mut forked = self.clone();
while forked.non_finalized_tip_hash() != fork_tip {
forked.pop_tip();
}
Some(forked)
}
fn non_finalized_tip_hash(&self) -> block::Hash {
self.blocks
.values()
.next_back()
.expect("only called while blocks is populated")
.hash()
}
/// Remove the highest height block of the non-finalized portion of a chain.
fn pop_tip(&mut self) {
let block_height = self.non_finalized_tip_height();
let block = self
.blocks
.remove(&block_height)
.expect("only called while blocks is populated");
assert!(
!self.blocks.is_empty(),
"Non-finalized chains must have at least one block to be valid"
);
self.revert_chain_state_with(&block);
}
fn non_finalized_tip_height(&self) -> block::Height {
*self
.blocks
.keys()
.next_back()
.expect("only called while blocks is populated")
}
}
/// Helper trait to organize inverse operations done on the `Chain` type. Used to
/// overload the `update_chain_state_with` and `revert_chain_state_with` methods
/// based on the type of the argument.
///
/// This trait was motivated by the length of the `push` and `pop_root` functions
/// and fear that it would be easy to introduce bugs when updating them unless
/// the code was reorganized to keep related operations adjacent to eachother.
trait UpdateWith<T> {
/// Update `Chain` cumulative data members to add data that are derived from
/// `T`
fn update_chain_state_with(&mut self, _: &T);
/// Update `Chain` cumulative data members to remove data that are derived
/// from `T`
fn revert_chain_state_with(&mut self, _: &T);
}
impl UpdateWith<Arc<Block>> for Chain {
fn update_chain_state_with(&mut self, block: &Arc<Block>) {
let block_height = block
.coinbase_height()
.expect("valid non-finalized blocks have a coinbase height");
let block_hash = block.hash();
// add hash to height_by_hash
let prior_height = self.height_by_hash.insert(block_hash, block_height);
assert!(
prior_height.is_none(),
"block heights must be unique within a single chain"
);
// add work to partial cumulative work
let block_work = block
.header
.difficulty_threshold
.to_work()
.expect("work has already been validated");
self.partial_cumulative_work += block_work;
// for each transaction in block
for (transaction_index, transaction) in block.transactions.iter().enumerate() {
let (inputs, outputs, shielded_data, joinsplit_data) = match transaction.deref() {
transaction::Transaction::V4 {
inputs,
outputs,
shielded_data,
joinsplit_data,
..
} => (inputs, outputs, shielded_data, joinsplit_data),
_ => unreachable!(
"older transaction versions only exist in finalized blocks pre sapling",
),
};
// add key `transaction.hash` and value `(height, tx_index)` to `tx_by_hash`
let transaction_hash = transaction.hash();
let prior_pair = self
.tx_by_hash
.insert(transaction_hash, (block_height, transaction_index));
assert!(
prior_pair.is_none(),
"transactions must be unique within a single chain"
);
// add the utxos this produced
self.update_chain_state_with(&(transaction_hash, outputs));
// add the utxos this consumed
self.update_chain_state_with(inputs);
// add sprout anchor and nullifiers
self.update_chain_state_with(joinsplit_data);
// add sapling anchor and nullifier
self.update_chain_state_with(shielded_data);
}
}
fn revert_chain_state_with(&mut self, block: &Arc<Block>) {
let block_hash = block.hash();
// remove the blocks hash from `height_by_hash`
assert!(
self.height_by_hash.remove(&block_hash).is_some(),
"hash must be present if block was"
);
// remove work from partial_cumulative_work
let block_work = block
.header
.difficulty_threshold
.to_work()
.expect("work has already been validated");
self.partial_cumulative_work -= block_work;
// for each transaction in block
for transaction in &block.transactions {
let (inputs, outputs, shielded_data, joinsplit_data) = match transaction.deref() {
transaction::Transaction::V4 {
inputs,
outputs,
shielded_data,
joinsplit_data,
..
} => (inputs, outputs, shielded_data, joinsplit_data),
_ => unreachable!(
"older transaction versions only exist in finalized blocks pre sapling",
),
};
// remove `transaction.hash` from `tx_by_hash`
let transaction_hash = transaction.hash();
assert!(
self.tx_by_hash.remove(&transaction_hash).is_some(),
"transactions must be present if block was"
);
// remove the utxos this produced
self.revert_chain_state_with(&(transaction_hash, outputs));
// remove the utxos this consumed
self.revert_chain_state_with(inputs);
// remove sprout anchor and nullifiers
self.revert_chain_state_with(joinsplit_data);
// remove sapling anchor and nullfier
self.revert_chain_state_with(shielded_data);
}
}
}
impl UpdateWith<(transaction::Hash, &Vec<transparent::Output>)> for Chain {
fn update_chain_state_with(
&mut self,
(transaction_hash, outputs): &(transaction::Hash, &Vec<transparent::Output>),
) {
for (utxo_index, _) in outputs.iter().enumerate() {
self.created_utxos.insert(transparent::OutPoint {
hash: *transaction_hash,
index: utxo_index as u32,
});
}
}
fn revert_chain_state_with(
&mut self,
(transaction_hash, outputs): &(transaction::Hash, &Vec<transparent::Output>),
) {
for (utxo_index, _) in outputs.iter().enumerate() {
assert!(
self.created_utxos.remove(&transparent::OutPoint {
hash: *transaction_hash,
index: utxo_index as u32,
}),
"created_utxos must be present if block was"
);
}
}
}
impl UpdateWith<Vec<transparent::Input>> for Chain {
fn update_chain_state_with(&mut self, inputs: &Vec<transparent::Input>) {
for consumed_utxo in inputs {
match consumed_utxo {
transparent::Input::PrevOut { outpoint, .. } => {
self.spent_utxos.insert(*outpoint);
}
transparent::Input::Coinbase { .. } => {}
}
}
}
fn revert_chain_state_with(&mut self, inputs: &Vec<transparent::Input>) {
for consumed_utxo in inputs {
match consumed_utxo {
transparent::Input::PrevOut { outpoint, .. } => {
assert!(
self.spent_utxos.remove(outpoint),
"spent_utxos must be present if block was"
);
}
transparent::Input::Coinbase { .. } => {}
}
}
}
}
impl UpdateWith<Option<transaction::JoinSplitData<Groth16Proof>>> for Chain {
fn update_chain_state_with(
&mut self,
joinsplit_data: &Option<transaction::JoinSplitData<Groth16Proof>>,
) {
if let Some(joinsplit_data) = joinsplit_data {
for sprout::JoinSplit {
anchor, nullifiers, ..
} in joinsplit_data.joinsplits()
{
self.sprout_anchors.insert(*anchor);
self.sprout_nullifiers.insert(nullifiers[0]);
self.sprout_nullifiers.insert(nullifiers[1]);
}
}
}
fn revert_chain_state_with(
&mut self,
joinsplit_data: &Option<transaction::JoinSplitData<Groth16Proof>>,
) {
if let Some(joinsplit_data) = joinsplit_data {
for sprout::JoinSplit {
anchor, nullifiers, ..
} in joinsplit_data.joinsplits()
{
assert!(
self.sprout_anchors.remove(anchor),
"anchor must be present if block was"
);
assert!(
self.sprout_nullifiers.remove(&nullifiers[0]),
"nullifiers must be present if block was"
);
assert!(
self.sprout_nullifiers.remove(&nullifiers[1]),
"nullifiers must be present if block was"
);
}
}
}
}
impl UpdateWith<Option<transaction::ShieldedData>> for Chain {
fn update_chain_state_with(&mut self, shielded_data: &Option<transaction::ShieldedData>) {
if let Some(shielded_data) = shielded_data {
for sapling::Spend {
anchor, nullifier, ..
} in shielded_data.spends()
{
self.sapling_anchors.insert(*anchor);
self.sapling_nullifiers.insert(*nullifier);
}
}
}
fn revert_chain_state_with(&mut self, shielded_data: &Option<transaction::ShieldedData>) {
if let Some(shielded_data) = shielded_data {
for sapling::Spend {
anchor, nullifier, ..
} in shielded_data.spends()
{
assert!(
self.sapling_anchors.remove(anchor),
"anchor must be present if block was"
);
assert!(
self.sapling_nullifiers.remove(nullifier),
"nullifier must be present if block was"
);
}
}
}
}
impl PartialEq for Chain {
fn eq(&self, other: &Self) -> bool {
self.partial_cmp(other) == Some(Ordering::Equal)
}
}
impl Eq for Chain {}
impl PartialOrd for Chain {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
impl Ord for Chain {
fn cmp(&self, other: &Self) -> Ordering {
if self.partial_cumulative_work != other.partial_cumulative_work {
self.partial_cumulative_work
.cmp(&other.partial_cumulative_work)
} else {
let self_hash = self
.blocks
.values()
.last()
.expect("always at least 1 element")
.hash();
let other_hash = other
.blocks
.values()
.last()
.expect("always at least 1 element")
.hash();
// This comparison is a tie-breaker within the local node, so it does not need to
// be consistent with the ordering on `ExpandedDifficulty` and `block::Hash`.
match self_hash.0.cmp(&other_hash.0) {
Ordering::Equal => unreachable!("Chain tip block hashes are always unique"),
ordering => ordering,
}
}
}
}

11
zebra-state/src/service.rs
View File

@ -13,9 +13,10 @@ use zebra_chain::{
parameters::Network,
};
use crate::{BoxError, Config, MemoryState, Request, Response, SledState};
use crate::{BoxError, Config, FinalizedState, NonFinalizedState, Request, Response};
// todo: put this somewhere
#[derive(Debug)]
pub struct QueuedBlock {
pub block: Arc<Block>,
// TODO: add these parameters when we can compute anchors.
@ -26,15 +27,15 @@ pub struct QueuedBlock {
struct StateService {
/// Holds data relating to finalized chain state.
sled: SledState,
sled: FinalizedState,
/// Holds data relating to non-finalized chain state.
_mem: MemoryState,
_mem: NonFinalizedState,
}
impl StateService {
pub fn new(config: Config, network: Network) -> Self {
let sled = SledState::new(&config, network);
let _mem = MemoryState {};
let sled = FinalizedState::new(&config, network);
let _mem = NonFinalizedState::default();
Self { sled, _mem }
}
}

10
zebra-state/src/sled_state.rs
View File

@ -18,16 +18,16 @@ use crate::{BoxError, Config, HashOrHeight, QueuedBlock};
/// - *asynchronous* methods that perform reads.
///
/// For more on this distinction, see RFC5. The synchronous methods are
/// implemented as ordinary methods on the [`SledState`]. The asynchronous
/// implemented as ordinary methods on the [`FinalizedState`]. The asynchronous
/// methods are not implemented using `async fn`, but using normal methods that
/// return `impl Future<Output = ...>`. This allows them to move data (e.g.,
/// clones of handles for [`sled::Tree`]s) into the futures they return.
///
/// This means that the returned futures have a `'static` lifetime and don't
/// borrow any resources from the [`SledState`], and the actual database work is
/// borrow any resources from the [`FinalizedState`], and the actual database work is
/// performed asynchronously when the returned future is polled, not while it is
/// created. This is analogous to the way [`tower::Service::call`] works.
pub struct SledState {
pub struct FinalizedState {
/// Queued blocks that arrived out of order, indexed by their parent block hash.
queued_by_prev_hash: HashMap<block::Hash, QueuedBlock>,
@ -42,7 +42,7 @@ pub struct SledState {
// sapling_anchors: sled::Tree,
}
impl SledState {
impl FinalizedState {
pub fn new(config: &Config, network: Network) -> Self {
let db = config.sled_config(network).open().unwrap();
@ -86,7 +86,7 @@ impl SledState {
/// It's the caller's responsibility to ensure that blocks are committed in
/// order. This function is called by [`process_queue`], which ensures order.
/// It is intentionally not exposed as part of the public API of the
/// [`SledState`].
/// [`FinalizedState`].
fn commit_finalized(&mut self, queued_block: QueuedBlock) {
let QueuedBlock { block, rsp_tx } = queued_block;