* Refact proposal pallet

* Update runtime/rococo/src/propose_parachain.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Fix

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <write@reusable.software>

* AHH

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Bastian Köcher <info@kchr.de>

* update release-bot.yml github action

* fix release-bot.yml

* configure channels

* start graphing message flow between subsystems

This commit includes messages originating from the first three subsystems
(alphabetically).

* use standard arrowhead names

* add messages from bitfield distribution && use circo layout

* finish adding all subsystem message traffic to graph

* reduce line spam

* start work on graphing v1 types

* finish graphing candidate-related structs

* show hash relations, format

* show provenance of ValidityAttestation

* add the rest of the v1 polkadot primitives

* add type diagram from polkadot::parachain::primitives

* start work digramming the path to backing

* diagram message flow from collation generation -> candidate selection

* document flow through CandidateBacking, CandidateValidation

* graph data flow through StatementDistribution

* trace data flow through PoVDistribution to Backing

* finish documenting node side subsystem data flow

* initial impl approval distribution

* initial tests and fixes

* batching seems difficult: different peers have different needs

* bridge: fix test after merge

* some guide updates

* only send assignments to peers who know about the block

* fix a test, add approvals test

* simplify

* do not send assignment to peers for finalized blocks

* guide: protocol input and output

* one more test

* more comments, logs, initial metrics

* fix a typo

* one more thing: early return when reimporting a thing locally

* Add a `convert_payload` function to `Signed`

* Review feedback

* removes old bootnodes from kusama chainspec

* removes old bootnodes from polkadot chainspec

* Put parachain head hash into `CandidateDescriptor`

* Update guide

* Add some checks

Bumps [zstd](https://github.com/gyscos/zstd-rs) from 0.5.3+zstd.1.4.5 to 0.5.4+zstd.1.4.7.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/commits

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* service: update for substrate PR #7339

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Add support for Grandpa warp sync

* Don't pull the warp sync crate if not full-node

* Update pallet_staking.rs

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

* Compress the PoV block before sending it over the network

This pr changes the way we send PoV blocks over the network. We now
compress the PoV block before it is send over the network. This should
reduce the size significant for PoVs which contain the runtime WASM for
example.

* Preallocate 1KB

* Try something..

* Switch to zstd and some renamings

* Make compression/decompression fail in browsers

* Use some sane maximum value

* Update roadmap/implementers-guide/src/types/network.md

Co-authored-by: Andronik Ordian <write@reusable.software>

* Fix and add test

* add

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Companion for sub/7040

* Fix duplicate type.

* Fix syntax

* Sync with latest changes.

* Add weight info.

* Fix build

* Fix some tests

* Fix build again.

* fix migration amounts.

* new migration

* Builds fine with dummy weights.

* fix build

* Add correct weight files.

* Fix to latest version

* remove unneeded upgrade.

* Update to latest again.

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Adapt service creation to new substrate API

* Fix test

* WIP

* Revert "WIP"

This reverts commit 816a3633

.

* WIP

* Adapt to current code

* Fix tests

* Yet another fix

* CLEANUP

* WIP

* WIP

* WIP

* Adapt code to changes on substrate

* Adapt code

* Introduce kick.

* Fixes

* WIP

* WIP

* WIP

* WIP

* Bump

* Update sp-io

* WIP

Co-authored-by: Gav Wood <gavin@parity.io>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.119 to 1.0.120.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.119...v1.0.120

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* glossary: define DMQ

We are using DMQ quite often including in rustdoc, however, it may be
hard to know what is that. Adding it to the glossary.

* Clarify that DMP and HRMP are also module names

* Define PDK

* Define preimage

* Companion PR for adding max notification sizes

* Add TODO

* Missing ;

* =>

* Common limit

* "Update Substrate"

Co-authored-by: parity-processbot <>

* collation-generation: use persisted validation data

* node: remote FullValidationData API

* runtime: remove FullValidationData API

* backing tests: use persisted validation data

* FullCandidateReceipt: use persisted validation data

This is not a big change since this type is not used anywhere

* Remove ValidationData and TransientValidationData

Also update the guide

* skeleton

* skeleton aux-schema module

* start approval types

* start aux schema with aux store

* doc

* finish basic types

* start approval types

* doc

* finish basic types

* write out schema types

* add debug and codec impls to approval types

* add debug and codec impls to approval types

also add some key computation

* add debug and codec impls to approval types

* getters for block and candidate entries

* grumbles

* remove unused AssignmentId

* load_decode utility

* implement DB clearing

* function for adding new block entry to aux store

* start `canonicalize` implementation

* more skeleton

* finish implementing canonicalize

* tag TODO

* implement a test AuxStore

* add allow(unused)

* basic loading and deleting test

* block_entry test function

* add a test for `add_block_entry`

* ensure range is exclusive at end

* test clear()

* test that add_block sets children

* add a test for canonicalize

* Update node/core/approval-voting/src/aux_schema/mod.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Update node/core/approval-voting/src/aux_schema/tests.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Update node/core/approval-voting/src/aux_schema/mod.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Make large collations downloads work

* Actually, bump it more to be sure

* Grmlblbl Rust feature unstable nianiania

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.11 to 0.3.12.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.11...0.3.12

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Improve logging to make debugging parachains easier

This pr should make debugging parachains easier, by printing more
information about the validation process.

* 🤦

* moare

* Convert to debug

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.118 to 1.0.119.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.118...v1.0.119

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix

* Fix

* Bump Substrate

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Switch to Wasmtime for parachain execution

* Fix the signature for halt's validate_block

* wasmtime feature enabled from cli

* Adds missing feature

Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

* Add warning when dropping inclusion inherent

* improve warning when dropping inclusion inherent

- use the actual runtime `warn!` macro
- do not speculate about why the error occurred
- show the actual error returned

* BROKEN: add note if the current block matches the session start block

This doesn't compile for a variety of reasons; this commit demonstrates
what the issues are. I don't think this is an essential feature, so I'm
stopping work on it here instead of pushing through it.

* Revert "BROKEN: add note if the current block matches the session start block"

This reverts commit eeb79ab7.

* ensure a runtime logger exists before attempting to log a warning to it

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.10 to 0.3.11.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.10...0.3.11

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Adds message types

* Add code skeleton

* Adds subsystem code.

* Adds a first test

* Adds interaction result to availability_lru

* Use LruCache instead of a HashMap

* Whitespaces to tabs

* Do not ignore errors

* Change error type

* Add a timeout to chunk requests

* Add custom errors and log them

* Adds replace_availability_recovery method

* recovery_threshold computed by erasure crate

* change core to std

* adds docs to error type

* Adds a test for invalid reconstruction

* refactors interaction run into multiple methods

* Cleanup AwaitedChunks

* Even more fixes

* Test that recovery with wrong root is an error

* Break to launch another requests

* Styling fixes

* Add SessionIndex to API

* Proper relay parents for MakeRequest

* Remove validator_discovery and use message

* Remove a stream on exhaustion

* On cleanup free the request streams

* Fix merge and refactor

* Update for new service config

* Naming

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Companion for Substrate#7692

* "Update Substrate"

Co-authored-by: parity-processbot <>

* session_info: fix authorities docstring

* overseer: more consistent metrics naming

* session_info: mention ordering

* use correct bucket sizes

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* More doc fixes.

* Minor refactorings in the process of #2177

By having everything peer set related depend directly on the enum the
code becomes more clear and it is also straight forward to add more
peersets/protocols as the compiler will complain if you forget to
implement parts of it.

* Add peer set infos on startup properly

For feature real_overseer.

+ Fixes from review. Thanks @coriolinus and @ordian

!

* More structure in network-bridge

Some changes, which would have helped me in groking the code faster.

Entry points/public types more to the top. Factored out implementation
in their own files, to clear up the top-level view.

* Get rid of local ProtocolName type definition.

Does not add much at this level.

* Fix tests + import cleanup.

* Make spaces tabs.

* Clarify what correct parameters to send_message are

* Be more less vague in docs of send_message.

* Apply suggestions from code review

Extend copyright on new files to 2021 as well.

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Andronik Ordian <write@reusable.software>

Bumps [rand](https://github.com/rust-random/rand) from 0.8.1 to 0.8.2.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/0.8.1...0.8.2

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>