* polkadot-service: Make native runtime configurable

This pull requests adds support for configuring the native runtimes used
by polkadot-service. While this whole pr doesn't change that much for
polkadot, besides not having the light-node enabled for the default
polkadot binary. However, downstream projects (parachains) will have a
much better compile time. In cumulus for example the `cargo test --all
--release` is about 4m faster to compile.

* Fixes

* Fix

* Enable rococo-native

* Fix light client

* 🤦

* Fixes

* Create validator_side module

* Subsume Candidate Selection

* Add test to ensure candidate backing logic is correct

* Ensure secondings are adequately cleaned up and address test flakyness

* Address Feedback

* node: fix grandpa voting rule

* node: cleanup find_target

* node: fix babe worker params

* update Substrate

Co-authored-by: parity-processbot <>

* put a cap on finality lag

* Update node/service/src/grandpa_support.rs

Co-authored-by: André Silva <[email protected]>

* apply lag to best, not to current vote

Co-authored-by: André Silva <[email protected]>

* Update for the new substrate client API

* update Substrate

Co-authored-by: parity-processbot <>

* Stuff to help inspect the DMP activity

* Fix teleport accounting

* Fixes

* Fixes

* Fixes

* Fixes

* fix approval-checking GRANDPA voting rule

a `None` return value implies to vote on the best, not to vote on the base.

this explicitly changes the logic to vote on the base

* refactor logic out and test

* Use proper host configuration everywhere

* Fix dumb errors

* enable Config for Paras

* fix

* Replace NetworkStatusSinks with Arc<NetworkService>

* update Substrate

Co-authored-by: parity-processbot <>

* grandpa: enable observer for light client

* update Substrate

* service: fix missing import

* runtime: bump spec versions

Co-authored-by: parity-processbot <>

* Add `UsageProvider` trait to `AbstractClient`

* Fix

* Add bridges code to workspace

* Add Rococo and Wococo bridge instances to Rococo runtime

* Add Rococo and Wococo runtime APIs

* Add GenesisConfig parameters for bridge pallet

* Update Rococo spec_version used by bridge relayer

* Add scripts for running Rococo and Wococo dev nodes

* Add scripts for running Rococo<>Wococo header sync

* Apply patch for build artifact location

* Remove bridges crates from workspace

* Downgrade async related dependencies

* Change bridge pallet owner to be `root_key`

* Bump number of `MaxRequests` allowed

* Revert changes in `bridges` subtree folder

* Use correct account for Sudo

* Add comment explaining duplicate bridge pallets

* Remove WeightInfo comment.

Co-authored-by: Tomasz Drwięga <[email protected]>

* Initial integration of Gilts pallet (Kusama)

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_gilt --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* Use real weights

* Update lock

* Merge

* Fixes

* Add working.

* Add proper curve arithmetic

* Fixes

* Fixes

* Make build

* Fixes

* Fixes

* Fix build

* remove dep.

* undo dep.

* upadte substrate

* Fix

* Bump Substrate

* Fixes

* Fixes

* Fix test

* Remove cap and some tests

* Fixes

* Fixes

* Update runtime/kusama/src/lib.rs

Co-authored-by: Kian Paimani <[email protected]>

* bump the lock file

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

Co-authored-by: Parity Benchmarking Bot <[email protected]>
Co-authored-by: kianenigma <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Add --disable-beefy CLI flag

* Update cli/src/command.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Update node/test/service/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update parachain/test-parachains/adder/collator/src/main.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update cli/src/command.rs

Co-authored-by: André Silva <[email protected]>

* Update cli/src/cli.rs

Co-authored-by: André Silva <[email protected]>

* Update cli/src/cli.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update cli/src/cli.rs

Co-authored-by: André Silva <[email protected]>

Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: André Silva <[email protected]>

* runtime: remove beefy and mmr from westend (again)

* node: add borked westend runtime upgrade as bad block

* Update branch

* Make it compile

* Compile

* gate approval-checking logic (#2470)

* Fix build

* Updates

* Fix merge

* Adds missing crate

* Companion for Substrate#8386

https://github.com/paritytech/substrate/pull/8386



* Fix fix fix

* Fix

* Fix compilation

* Rewrite to `ParachainsInherentDataProvider`

* Make it compile

* Renamings

* Revert stuff

* Remove stale file

* Guide updates

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Apply suggestions from code review

* Reset accidental changes

* More

* Remove stale file

* update Substrate

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: parity-processbot <>

* add `force_set_active_config`

* add parachains modules to Westend

* add parachains modules to Kusama runtime

* use real runtime API impl

* add module indices and remove auctions, crowdloan

* add benchmarks

* remove previous migrations and add host configuration set migration

* make compile

* Add Call Filter for Registrar and Slots except Root

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix build

* update `add_benchmark`

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_common::paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_common_paras_registrar.rs

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_common::slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_common_slots.rs

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_common::paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_common_paras_registrar.rs

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_common::slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_common_slots.rs

* fix weights

* tweak more constants

* Fix up the kusama runtime

* Westend runtime fixups

* Fix MMR & Beefy for westend

* Fixes

* fix tests

* Update runtime/polkadot/src/constants.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Update runtime/westend/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>
Co-authored-by: Gav Wood <[email protected]>

* Factor out runtime module into utils.

* First fatal error design.

* Better error handling infra.

* Error handling cleanup.

* Send to peers of our group first.

* Finish backing group prioritization.

* Little cleanup.

* More cleanup.

* Forgot to checkin error.rs.

* Notes.

* Runtime -> RuntimeInfo

* qed in debug assert.

* PolkaErr -> Fault.

* Add `rococo-dev` chain option

* Add `wococo-dev` CLI option

* Sort Chain ID match arms

* runtime: remove mmr and beefy from westend runtime

* runtime: westend: remove pallet_beefy config

* node: only start beefy gadget on rococo

* node: remove beefy keys from westend chain spec

* node: add wococo testnet chain spec

* node: add comments about beefy gadget task

* runtime: update wococo chainspec

* Remove stale comment

Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: adoerr <[email protected]>

* runtime: remove mmr and beefy from westend runtime

* runtime: westend: remove pallet_beefy config

* node: only start beefy gadget on rococo

* node: remove beefy keys from westend chain spec

* Fix bench bot

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_membership --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* Fix weights files

* Fix'

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_membership --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* bump substrate

* bump beefy

* bump beefy again

* bump beefy

* Companion for substrate#8643

* Update node/service/src/lib.rs

* update Substrate

Co-authored-by: parity-processbot <>

* Add it in rococo runtime

* Add to rococo chain spec

* bump spec version

* Update runtime/rococo/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Robert Klotzner <[email protected]>

* Wip

* Increase proposer timeout.

* WIP.

* Better timeout values now that we are going to be connected to all nodes. (#2778)

* Better timeout values.

* Fix typo.

* Fix validator bandwidth.

* Fix compilation.

* Better and more consistent sizes.

Most importantly code size is now 5 Meg, which is the limit we currently
want to support in statement distribution.

* Introduce statement fetching request.

* WIP

* Statement cache retrieval logic.

* Review remarks by @rphmeier

* Fixes.

* Better requester logic.

* WIP: Handle requester messages.

* Missing dep.

* Fix request launching logic.

* Finish fetching logic.

* Sending logic.

* Redo code size calculations.

Now that max code size is compressed size.

* Update Cargo.lock (new dep)

* Get request receiver to statement distribution.

* Expose new functionality for responding to requests.

* Cleanup.

* Responder logic.

* Fixes + Cleanup.

* Cargo.lock

* Whitespace.

* Add lost copyright.

* Launch responder task.

* Typo.

* info -> warn

* Typo.

* Fix.

* Fix.

* Update comment.

* Doc fix.

* Better large statement heuristics.

* Fix tests.

* Fix network bridge tests.

* Add test for size estimate.

* Very simple tests that checks we get LargeStatement.

* Basic check, that fetching of large candidates is performed.

* More tests.

* Basic metrics for responder.

* More metrics.

* Use Encode::encoded_size().

* Some useful spans.

* Get rid of redundant metrics.

* Don't add peer on duplicate.

* Properly check hash

instead of relying on signatures alone.

* Preserve ordering + better flood protection.

* Get rid of redundant clone.

* Don't shutdown responder on failed query.

And add test for this.

* Smaller fixes.

* Quotes.

* Better queue size calculation.

* A bit saner response sizes.

* Fixes.

* Introduce a collective into Rococo runtime

* Intregrate Rococo Collective into XCM

* Fixes

* Update runtime/rococo/src/lib.rs

Co-authored-by: Robert Habermeier <[email protected]>

* Implement PVF validation host

* WIP: Diener

* Increase the alloted compilation time

* Add more comments

* Minor clean up

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Fix pruning artifact removal

* Fix formatting and newlines

* Fix the thread pool

* Update node/core/pvf/src/executor_intf.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Remove redundant test declaration

* Don't convert the path into an intermediate string

* Try to workaround the test failure

* Use the puppet_worker trick again

* Fix a blip

* Move `ensure_wasmtime_version` under the tests mod

* Add a macro for puppet_workers

* fix build for not real-overseer

* Rename the puppet worker for adder collator

* play it safe with the name of adder puppet worker

* Typo: triggered

* Add more comments

* Do not kill exec worker on every error

* Plumb Duration for timeouts

* typo: critical

* Add proofs

* Clean unused imports

* Revert "WIP: Diener"

This reverts commit b9f54e513366c7a6dfdd117ac19fbdc46b900b4d.

* Sync version of wasmtime

* Update cargo.lock

* Update Substrate

* Merge fixes still

* Update wasmtime version in test

* bastifmt

Co-authored-by: Bastian Köcher <[email protected]>

* Squash spaces

* Trailing new line for testing.rs

* Remove controversial code

* comment about biasing

* Fix suggestion

* Add comments

* make it more clear why unwrap_err

* tmpfile retry

* proper proofs for claim_idle

* Remove mutex from ValidationHost

* Add some more logging

* Extract exec timeout into a constant

* Add some clarifying logging

* Use blake2_256

* Clean up the merge

Specifically the leftovers after removing real-overseer

* Update parachain/test-parachains/adder/collator/Cargo.toml

Co-authored-by: Andronik Ordian <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Andronik Ordian <[email protected]>

* remove real-overseer

* overseer: only activate leaves which support parachains

* integrate HeadSupportsParachains into service

* remove unneeded line

* move parachains_db.rs to a module

* parachain versioning

* fixes

* more fixes

* Update node/service/src/parachains_db/upgrade.rs

* fix web-wasm check

* suggested style fixes

Co-authored-by: Bastian Köcher <[email protected]>

* Update node/service/src/parachains_db/upgrade.rs

* compilation fixes

Co-authored-by: Bastian Köcher <[email protected]>