* Implement PVF validation host

* WIP: Diener

* Increase the alloted compilation time

* Add more comments

* Minor clean up

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix pruning artifact removal

* Fix formatting and newlines

* Fix the thread pool

* Update node/core/pvf/src/executor_intf.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove redundant test declaration

* Don't convert the path into an intermediate string

* Try to workaround the test failure

* Use the puppet_worker trick again

* Fix a blip

* Move `ensure_wasmtime_version` under the tests mod

* Add a macro for puppet_workers

* fix build for not real-overseer

* Rename the puppet worker for adder collator

* play it safe with the name of adder puppet worker

* Typo: triggered

* Add more comments

* Do not kill exec worker on every error

* Plumb Duration for timeouts

* typo: critical

* Add proofs

* Clean unused imports

* Revert "WIP: Diener"

This reverts commit ff2d3ff2

.

* Sync version of wasmtime

* Update cargo.lock

* Update Substrate

* Merge fixes still

* Update wasmtime version in test

* bastifmt

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Squash spaces

* Trailing new line for testing.rs

* Remove controversial code

* comment about biasing

* Fix suggestion

* Add comments

* make it more clear why unwrap_err

* tmpfile retry

* proper proofs for claim_idle

* Remove mutex from ValidationHost

* Add some more logging

* Extract exec timeout into a constant

* Add some clarifying logging

* Use blake2_256

* Clean up the merge

Specifically the leftovers after removing real-overseer

* Update parachain/test-parachains/adder/collator/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>

* use compressed blob in candidate-validation

* add some tests for compressed code blobs

* remove CompressedPoV and apply compression in collation-generation

* decompress BlockData before executing

* don't produce oversized collations

* add test for PoV decompression failure

* fix tests and clean up

* fix test

* address review and fix CI

* take this )

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Andronik Ordian <write@reusable.software>

* Introduce plurality XCM locations

* Add RelayedFrom

* remove real-overseer

* overseer: only activate leaves which support parachains

* integrate HeadSupportsParachains into service

* remove unneeded line

* Remove unused relaying XCM

* Aggregate HRMP (XCMP/HMP) messages. Payloads for spambot.

* Revert lock

* Fix

* Broken example

* Introduce fee payment mechanics into XCM.

* Weight limitations on XCM execution

* Mock environment for tests and the first test

* Tests for XCM and a few refactors.

* Remove code that's not ready

* Fix for an XCM and an additional test

* Query response system

* XCMP message dispatch system reimagining

- Moved most of the logic into xcm-handler pallet
- Altered the outgoing XCMP API from push to pull
- Changed underlying outgoing queue data structures to avoid multi-page read/writes
- Introduced queuing for incoming messages
- Introduced signal messages as a flow-control sub-stream
- Introduced flow-control with basic threshold back-pressure
- Introduced overall weight limitation on messages executed
- Additonal alterations to XCM APIs for the new system

* Some build fixes

* Remove the Encode bounds sprayed around

* More faff

* Fix bounds amek use latest scale codec.

* remove println

* fixes

* Fix XcmExecutor Tests

* Fix XCM bounds using derivative crate

* Refactor names of XcmGeneric &c into Xcm

* Repot the xcm-executor into xcm-builder

* Docs

* Docs

* Fixes

* Update xcm/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Fixes

* Docs

* Update runtime/parachains/src/ump.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Docs

* Fixes

* Fixes

* Fixes

* Docs

* Fixes

* Fixes

* Introduce transfer_asset specialisation.

* Fixes

* Fixes

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update hrmp.rs

* add root functions

* Relax Origin Checks in Registrar

* fix tests

* Update runtime/common/src/paras_registrar.rs

* introduce para locks

* apply a lock after upgrade

* add test

* add lock when creating crowdloan

* remove WASM_MAGIC

* fix test warnings

* move parachains_db.rs to a module

* parachain versioning

* fixes

* more fixes

* Update node/service/src/parachains_db/upgrade.rs

* fix web-wasm check

* suggested style fixes

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update node/service/src/parachains_db/upgrade.rs

* compilation fixes

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* stmnt-dist: do not verify signature on duplicates

* renames

* more renames

* Properly remove peers from sets

* Actually rename all, I guess

* Merge the two Network traits

* Rename function

* Update node/network/bridge/src/network.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Fix erroneous change

* Update node/network/bridge/src/network.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* tests/av-store: use future::join instead of future::select

* tests/backing: use future::join instead of future::select

* tests/provisioner: use future::join instead of future::select

* tests/av-dist: use future::join instead of future::select

* tests/av-recovery: use future::join instead of future::select

* tests/bridge: use future::join instead of future::select

* tests/collator-protocol: use future::join instead of future::select

* tests/stmt-dist: use future::join instead of future::select

* fix tests

* add metrics (unused) to network bridge

* fix test compilation

* trigger metrics messages

* add some more metrics

* track sent and received notifications

* restore metrics import

* integrate into service

* Update node/network/bridge/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update node/network/bridge/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Andronik Ordian <write@reusable.software>

* bitfield-dist: fix state update on gossip

* fixes

* doc fixes

* oops

* 2 lines of code change

* fix tests

* add test for rejecting declares on collators

* fix bad test

* feat: Add poke to Crowdloan

* fixes

* less hard-coded benchmark

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* implement auto refund

* add back retiring @jacogr



* bump substrate dep

* trying to get to work

* fix tests

* fix benchmarks

* Update runtime/common/src/crowdloan.rs

* fix docs

* remove unused traits

* remove unused

* unused

* Update runtime/common/src/crowdloan.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update runtime/common/src/crowdloan.rs

* Update crowdloan.rs

* Update runtime/common/src/crowdloan.rs

* Update runtime/common/src/crowdloan.rs

Co-authored-by: Gavin Wood <gavin@parity.io>

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Gavin Wood <gavin@parity.io>

* Macro for generating `SlotRange`

* remove test code

* refactor slots into crate

* add no_std flag

* finish macro

* make compile

* fix copyright date

* don't assume lease periods per slot

* Update runtime/common/Cargo.toml

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* fixes

* Update lib.rs

* tests

* Move consts into struct

* docs

* fix compile

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update Cargo.lock

* Update runtime/common/src/slot_range.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* fixes

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* guide: declare one para as a collator

* add ParaId to Declare messages and clean up

* fix build

* fix the testerinos

* begin adding keystore to collator-protocol

* remove request_x_ctx

* add core_for_group

* add bump_rotation

* add some more helpers to subsystem-util

* change signing_key API to take ref

* determine current and next para assignments

* disconnect collators who are not on current or next para

* add collator peer count metric

* notes for later

* some fixes

* add data & keystore to test state

* add a test utility for answering runtime API requests

* fix existing collator tests

* add new tests

* remove sc_keystore

* update cargo lock

Co-authored-by: Andronik Ordian <write@reusable.software>

* approval-distribution: split peer knowledge into sent and received

* guide updates

* fixes

* revert doc changes

* update using diener

* fix zstd upgrade

* Revert "update using diener"

This reverts commit 701fdfe3.

* update zstd to 0.6

* update using diener

* Revert "update using diener"

This reverts commit fc81df63.

* update Substrate

Co-authored-by: parity-processbot <>

* add disputes members to `HostConfiguration`

* revert bad rename

* code stored in para + modify CandidateDescriptor.

* WIP: digest + some more impl

* validation_code_hash in payload + check in inclusion

* check in client + refator

* tests

* fix encoding indices

* remove old todos

* fix test

* fix test

* add test

* fetch validation code inside collation-generation from the relay-chain

* HashMismatch -> PoVHashMismatch + miscompilation

* refactor, store hash when needed

* storage rename: more specific but slightly too verbose

* do not hash on candidate validation, fetch hash instead

* better test

* fix test

* guide updates

* don't panic in runtime

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* extract database from av-store itself

* generalize approval-voting over database type

* modes (without handling) and pruning old wakeups

* rework approval importing

* add our_approval_sig to ApprovalEntry

* import assignment

* guide updates for check-full-approval changes

* some aux functions

* send messages when becoming active.

* guide: network bridge sends view updates only when done syncing

* network bridge: send view updates only when done syncing

* tests for new network-bridge behavior

* add a test for updating approval entry with sig

* fix some warnings

* test load-all-blocks

* instantiate new parachains DB

* fix network-bridge empty view updates

* tweak

* fix wasm build, i think

* Update node/core/approval-voting/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* add some versioning to parachains_db

* warnings

* fix merge changes

* remove versioning again

Co-authored-by: Andronik Ordian <write@reusable.software>