Commit Graph

149 Commits

Author SHA1 Message Date
Pili Guerra 13cd8b9c2c
cleanup(deps): Remove unused `fpe` dependency and cleanup references (#5541)
* Clean up references to fpe that were not removed when we removed orchard code

* Update Cargo.lock

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-04 09:04:14 +00:00
teor 26d0455d02
fix(doc): Use a more reliable release template branch name and docker command (#5519)
* Fix the branch name in the release template

* Use a docker command with colour and Ctrl-C support

* Make branch name example more readable

* Fix a link typo

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-03 00:28:46 +00:00
Alfredo Garcia f27fa21099
docs(audit): Add a list of Zebra dependencies for the audit (#5395)
* add a list of dependencies for auditors

* change intro

Co-authored-by: teor <teor@riseup.net>

* add zebra-checkpoints to audit

* replace colors in the legend as they don't render in github

* make crypto deps out of the scope

* add a reason to keep rocksdb out of the scope

* add a not of why chrono is out of the scope

* add note for bitvec dep

* move structopt to out of scope

* add zcash_script as partially in scope

* applies suggestion, updates versions, bolds ed25519-zebra

* adds zcash_proofs audit notes

* Splits list of zebra deps into sections for full audits, partial audits, and out of scope.

* Update book/src/dev/zebra-dependencies-for-audit.md

Co-authored-by: teor <teor@riseup.net>

* Update book/src/dev/zebra-dependencies-for-audit.md

* Add missing newline

* Update book/src/dev/zebra-dependencies-for-audit.md

Co-authored-by: teor <teor@riseup.net>

* updates crate versions

* Update zcash_script to 0.1.8

* Only link to the audit version of each crate or file

* Clarify out of scope versions

* Fix source paths

* Fix zcash_script paths

* Improve formatting

* Update zebra-dependencies-for-audit.md

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: arya2 <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-02 02:42:20 +00:00
teor 17c0bacca3
add(doc): Add draft audit process, and update some support docs (#5433)
* Document parameter download workaround, particularly for macOS

* Replace outdated build docs with a link to the README

* Remove incorrectly rendered heading links

* Add an initial draft audit process

* Add dependency criteria
2022-10-21 01:20:28 +00:00
Gustavo Valverde 65e9822c43
docs(release): add information on how we tag and release (#5392)
* draft

* docs(release): add information on how we tag and release

* docs(release): remove outdated document

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Remove rough draft analysis of alternatives

Co-authored-by: teor <teor@riseup.net>
2022-10-19 19:31:28 +00:00
Arya de3640ddcc
adds zebra book's CI/CD docs to the sidebar (#5355) 2022-10-06 20:15:16 +00:00
teor 55e5a13fc8
List temporary CI errors that should be retried (#5281) 2022-09-28 13:14:39 +00:00
teor 9a2814a1b2
doc(ci): Explain how to fix Mergify and Zcash parameter download failures (#5240)
* Explain how to find Mergify failures

* Explain how to fix cache errors

* Fix instructions - clear all caches

* Fix which errors need which actions

* Add a newline to appease GitHub markdown renderer

Co-authored-by: Arya <aryasolhi@gmail.com>

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-27 01:32:55 +00:00
teor 12d084f6cc
fix(ci): Delete Google Cloud test instances after 3 days (#5192)
* Delete test instances after 3 days

* Use correct delete command, improve shell quoting

* Use sed to provide the correct zone or region

* Fix quoting

* Fix IFS

* Fix IFS for multiple disks

* Document why we can't quote some shell variables

* Document that instances can get deleted

* Fix exact names in deletion docs
2022-09-21 23:16:38 +00:00
Pili Guerra 64d984340d
Update release-process.md (#5181)
Fixing some small typos
2022-09-16 16:25:43 +02:00
Gustavo Valverde 085367fc37
docs(release): add versioning and releases information (#4917)
* ci(docs): test `cargo doc` warnings and errors

* ci(docs): remove minimal profile

* ci(docs): revert minimal and fix component name

* ci(docs): try cargo doc alone

* docs(release): add versioning and releases information

* dosc(release): apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* docs(release): apply multiple fixes from review

* docs(release): allow more flexibility with release cycles

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2022-09-15 02:17:09 +00:00
teor c2514ba64a
fix(doc): Update deny.toml developer docs and file comments (#5151)
* Make deny.toml instructions more specific

* Add documentation and depdendency comments to deny.toml
2022-09-14 19:52:32 +00:00
teor dcbbf444b9
fix(ci): Increase the full sync timeout (#5129)
* Increase the full sync timeout to 28 hours

And put the timeout constant with the rest of them.

* Update developer documentation for the new constant location

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-13 03:52:23 +00:00
teor 4810f44564
doc(ci): Document how to resolve disk full errors (#5105)
* Document how to resolve disk full errors

* Update to the `main` branch first
2022-09-07 11:16:18 +00:00
teor e4707a6cfd
Explain how Mergify works, and when it can be used (#5100) 2022-09-06 21:47:09 +00:00
teor fec012a006
fix(ci): delete GCP resources, but keep some recent cached state images (#5082)
* Fix delete GCP resources commands

* Don't create a GCP credentials file

* Keep the latest 2 images

* Explain time

* Show the names of disks that are being deleted

* Actually run the image delete steps

* Only delete commit-based instance templates

* Document automated deletion
2022-09-06 02:51:46 +00:00
teor 9d85360243
Explain how to find errors in CI (#5080) 2022-09-05 21:43:24 +00:00
teor 21b1a622e6
fix(deny.toml): Change an exception to the earlier pin-project version (#5070)
* Use the earlier pin-project version in deny.toml

* Change deny.toml instructions to check for updated versions

* Remove trailing whitespace
2022-09-05 20:12:52 +00:00
Alfredo Garcia 5dd3a6ca1a
fix doc render (#5038) 2022-09-02 08:55:37 +00:00
teor edd32a7320
doc(ci): Explain how to resolve duplicate dependency warnings (#5017)
* Explain how to resolve duplicate dependency warnings

* Explain why the errors happen and add the CI job name

* Add extra steps

* Add some examples, tools, and extra steps
2022-09-01 19:13:23 +00:00
teor 09420d57be
fix(ci): Update mainnet and testnet checkpoints (#4972)
* Update mainnet and testnet checkpoints

* Add instructions for resolving CI sync timeouts

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-29 02:45:13 +00:00
Gustavo Valverde 8227dabe56
ci(build): deploy long running node instances on release (#4939)
* feat(build): deploy long running instances on release

Previous behavior:
Each time we merged to main new nodes would be deployed, this is an
expected behavior as we need to ensure nodes get deployed and run
without issues, but this could also replace nodes very hastily.

Expected behavior:
We want instances which would run for a longer time, to allow us to
troubleshoot issues or inspect the behavior of this instances for longer
periods of time (2+ weeks)

Applied solution:
Deploy a versioned manage instance group (MiG) using the major version
of the release semver. We just use the first part of the version to
replace old instances, and change it when a major version is released
to keep a segregation between new and old versions.

* ci(build): allow v0 as a major version tag

* fix(build): use rust conventions for versioning

* fix(deploy): improve documentation and trigger on release

* Update .github/workflows/continous-delivery.yml

Co-authored-by: teor <teor@riseup.net>

* fix(versioning): typo

* fix(deploy): use `zebrad-v1` as the instance name, with no SHA

* fix(deploy): create and update MiG must use the same name

* docs(deployments): add Continuous Delivery process

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-28 12:46:21 +00:00
teor 0a39011b88
fix(ci): Write cached state images after update syncs, and use the latest image from any commit (#4949)
* Save cached state on full syncs and updates

* Add an -update suffix to CI images created by updating cached state

* Make disk image names unique by adding a time suffix

* Use the latest image from any branch, but prefer the current commit if available

* Document Zebra's continuous integration tests

* Fix typos in environmental variable names

* Expand documentation

* Fix variable name typo

* Fix shell syntax
2022-08-25 13:09:20 +00:00
Marek 32faa94fb4
fix(state): Update column family names to match Zebra's database design (#4639)
* Rename `block_by_height` to `block_header_by_height` in fin state

* Rename `tx_by_hash` to `tx_loc_by_hash` in both (non & fin) states

* Rename `utxo_by_outpoint` to `utxo_by_out_loc` in finalized state

* Reorder the column families so that they match the docs

* Update `struct Chain` in the RFCs

* Increment `DATABASE_FORMAT_VERSION` to 25

* Remove obsolete docs from `0004-asynchronous-script-verification.md`

* Remove an obsolete `TODO` from `disk_db.rs`

* Delete unused snapshots

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-30 23:00:23 +00:00
Deirdre Connolly 21af125bfa
doc: add tokio-console page in book dev section (#4641)
* doc: add tokio-console page in dev section

* Add screenshot

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-21 03:38:12 +00:00
teor 12426fe078
Fix incorrect AddressBalance name in diagram (#4342)
* Fix incorrect AddressLocation size in diagram

* Actually the name is wrong
2022-05-06 11:10:21 -03:00
Conrado Gouvea ff0122044f
change(doc): add transaction index diagram to RFC-0005 (#4330)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-06 03:32:55 +00:00
teor 05440763ac
change(verifier): Update Zebra's block hash checkpoints (#4183)
* Update zebra-checkpoints docs for new default checkpoint config

Also:
- update commands
- link to the spec definition of settled network upgrades

* Update mainnet checkpoints

* Update testnet checkpoints
2022-04-25 14:43:06 +00:00
teor b895b8973a
change(rfc): Cleanup transparent address index database design (#4019)
* Rename location-based column families

* Use more consistent terminology for database changes

* Remove AddressLocation from utxo_by_out_loc
2022-04-01 20:40:43 +00:00
teor 98ec995d56
fix(design): update state RFC to put ordered list values in RocksDB keys (#3997)
* Update RFC to use BTreeSets for ordered lists

* Update to put value sets in the rocksdb key

* Clarify some workding

* Update type name
2022-03-30 23:42:09 +00:00
teor 7d4c92f7d6
fix(doc): Fix bugs in the lightwalletd database design (#3964)
* Re-order column families in design in dependency order

* Minor RFC design tweaks and fixes

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-27 22:59:43 +00:00
teor 2b51f9098d
change(doc): Simplify the database design using prefix iterators (#3916)
* Simplify the database design using prefix iterators

* Fix typos and missed changes

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Marek <mail@marek.onl>
2022-03-21 22:38:23 +00:00
teor 419770409a
doc(db): update database design for read-only state service (#3843)
* Add a TODO for a history tree concurrent write issue

* Update database design for read-only state service
2022-03-12 00:37:01 +00:00
teor 081cda7990
3. refactor(db): add disk serialization types for transactions (#3741)
* refactor(db): simplify block height serialization

* refactor(db): make height serialization length generic

* refactor(db): create a TransactionIndex type

This changes the names of some snapshot types,
but doesn't change any data.

* refactor(db): create transparent OutputIndex and OutputLocation types

This keeps the same serialization, to avoid changing the database version.

* doc(rfc/db): make transparent database type names consistent

* doc(rfc/db): fix a bug in the Utxo.is_coinbase derivation

* fix(db): use the correct serialized size for OutputLocation
2022-03-09 01:22:00 +00:00
teor ea7253ee5b
fix(rfc): rename some lightwalletd database types (#3567)
* fix(rfc): rename some lightwalletd database types

```
fastmod BlockTransactionCount HeightTransactionCount
fastmod FirstOutLocation TransparentAddrLoc
```

* fix(rfc): make lightwalletd column family names consistent

* feat(rfc): add sub-headings to the RocksDB column family list

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-17 22:41:15 +00:00
teor 9101e9a815
Revert "Updated utxo_by_outpoint column value type in documentation (#3376)" (#3385)
This reverts commit a007a49489.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-24 19:48:04 +00:00
Janito Vaqueiro Ferreira Filho ebd94b2e60
Fix deadlock in chain tip watch channel (#3378)
* Avoid sequential borrows in `LatestChainTip`

Calling `watch::Receiver::borrow` more than once in the same scope can
cause a deadlock. The instrumented methods were calling `borrow` twice
to record instrumented fields.

This refactors things to ensure `borrow` is only called once to record
the fields and perform any actions with the chain tip block.

* Remove `borrow()` calls in `ChainTipChange`

Refactor to use a `LatestChainTip` instance instead, which safely
protects the internal `watch::Receiver` so that it is not borrowed more
than once in the same scope.

* Add a paragraph to the Asynchronous guide

Warn against using two borrow guards in the same scope, and describe why
that can lead to a deadlock.
2022-01-24 00:36:40 +00:00
Conrado Gouvea a007a49489
Updated utxo_by_outpoint column value type in documentation (#3376) 2022-01-20 19:24:57 +00:00
Conrado Gouvea 4aeabd0b52
Fix interstitial sprout anchors check (#3283)
* Fix interstitial Sprout anchors check

* Update state docs; add sprout_trees_by_anchor to comparisons

* Update book/src/dev/rfcs/0005-state-updates.md

Co-authored-by: Marek <mail@marek.onl>

* Rename `interstitial_roots` to `interstitial_trees`

* Document consensus rules

* Refactor the docs

* Improve the docs for consensus rules

* Update reference to cached state

* Update zebra-state/src/service/check/anchors.rs

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Fix formatting

Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-18 20:18:49 +00:00
teor 361125079e
Draft design for lightwalletd database support (#3137)
* Draft design for lightwalletd database support

* Explain how to create blocks from headers and transactions

* Make keys smaller to improve database performance

* Support transparent address balances and UTXOs

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-12-22 06:09:53 +00:00
teor 6814525a7a
Update async correctness docs and the async in Zebra RFC (#3243)
* Justify that the ErrorSlot Mutex is deadlock-safe

* Document cancellation safety in the async RFC

* Document task starvation in the async RFC

Co-authored-by: Marek <mail@marek.onl>
2021-12-21 07:10:15 +00:00
teor 62c78ad939
Update the State RFC to match the current database format (#3139)
* Update the State RFC to match the current database format

* Formatting and name fixes

* Remove redundant generic parameter

* Remove redundant generics

* Fix history tree types

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Fix spacing

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-12-10 13:35:35 +00:00
Dimitris Apostolou afb8b3d477
Fix typos (#3055)
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-12 19:30:22 +00:00
teor 01e63da3fe
Create CompactSize64 for non-message-length fields (#3008)
* Add CompactSize64 and CompactSizeMessage types

But don't remove read_compactsize and write_compactsize yet.

* Fix CompactSize capitalisation

```sh
fastmod compactSize CompactSize zebra* book
fastmod compactsize CompactSize zebra* book
```

* Make CompactSize patterns consistent with integer lengths

* Replace unwrap_err with asserting is_err

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Replace a panic with an assertion

* Make generic serialization use CompactSizeMessage

* Fix type inference and borrow-checker errors

* Doctest fixes

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-04 15:57:30 +00:00
Conrado Gouvea 4367b232fc
Improve documentation for Beta release (#2894)
* Improve documentation

* Incorporate text about Zebra from the last newsletter

* Organize README and user docs

* Add table of contents, organize heading levels

* Fix link

* capitalize list items

* fix table of contents

* format spacing issue

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-10-29 19:25:30 +00:00
Marek 86d05c5e90
Update the documentation for value pools (#2919)
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-10-21 20:30:11 +00:00
Deirdre Connolly 5afefd55bf
Specify Zebra Client will only support Unifed Addresses (#2706) 2021-08-30 22:40:07 +00:00
teor a1cec27871
Update the signs in the value pools design to match the spec (#2576)
* Update the signs in the value pools design to match the spec

Also split the definitions and guide sections into transaction value balances
and chain value pools. And explain the sign differences between them in terms of
inputs and outputs.

* fix typo

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-06 22:00:02 +00:00
teor 7218aec19f
Design: Fix Transparent Value Sign and Definition (#2555)
* Design: Fix Transparent Value Sign and Definition

* make the transparent value pool the sum of unspent output values, like other pools
* swap the sign of the transparent value balance
* make the remaining transaction value balance the sum of unspent input values
* update the consensus rules based on recent spec changes
* explain why the transparent chain value pool balance can't be negative
* clarify the guide and consensus rules sections

* Use consistent notation for sprout value balances

* Fix typos

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-02 23:52:21 +00:00
teor ccf93cf5c6
Update state RFC for incremental trees, value pools, and RocksDB (#2456)
* Update state RFC for double-spends and other recent designs

* Update the value pool column family name

* Mark incremental note commitment trees as tentative

* Change history tree type

* Apply suggestions from code review
2021-07-12 10:33:02 -03:00