* Add internal iterator API for accessing relevant chain blocks
* get blocks from all chains in non_finalized state
* Impl FusedIterator for service::Iter
* impl ExactSizedIterator for service::Iter
* let size_hint find heights in side chains
Co-authored-by: teor <teor@riseup.net>
* Add transcript test for requests while state is empty
* Add happy path test for each query once the state is populated
* let populate logic handle out of order blocks
* Add a maximum queued height metric to the finalized state
And rename all the finalized state metrics to contain "finalized".
* Use i32 and -1 instead of Option<Height>
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
Prior to this PR we realized that the RFC had been drafted with the assumption that chains would be ordered from best to worst in `NonFinalizedState`. This assumption was incorrect, since `BTreeSet` only ever orders values in ascending order. This discrepancy was noticed and fixed in the code, but there were still some inconsistencies that needed to be cleaned up.
This PR updates all the incorrect or confusing comments about chain ordering in the RFC and code.
Prior to this PR `memory_state` defined and implemented functionality for three different types, `Chain`, `NonFinalizedState`, and `QueuedBlocks`. Each of these components will need a fair number of unit tests, and I realized that as its currently organized it would be difficult to organize the tests or at a glance figure out which tests are testing which components.
This PR changes the organization of `memory_state` such that each component it exports is defined in its own module. In follow up PRs each module will get its own test module, which will focus exclusively on unit tests for the item defined there-in.
- [Tracking Issue](https://github.com/ZcashFoundation/zebra/issues/1250)
* make service use both finalized and non-finalized state
* Document new functions
* add documentation to sled fns
* cleanup tip fn now that errors are gone
* rename height unwrap fn
## Motivation
While working on the block locator fix PR together with Henry we noticed that we'd accidentally serialized entire transactions in `tx_by_hash`, instead of serializing just the height of the block and the index of the transaction within the block, as described by the original RFC.
## Solution
We've fixed it by adding a `TransactionLocation` new type, which handles the sled format traits. We've removed the sled format impls for `Transaction` to prevent inserting the wrong data in the future. Finally we've bumped the database format to reflect the change in the format on the disk and its incompatibility with previous versions.
Closes#1026
Because of the way that sled uses this parameter, the actual in-memory
size may be much larger. Dialing this down should help avoid high
memory usage.
Jane Lusby
teor
dependabot[bot]
Jane Lusby
Alfredo Garcia
Deirdre Connolly
Henry de Valence