* introduce the OverseerConnector, use it

* introduce is_relay_chain to RelayChainSelection

* Update node/service/src/lib.rs

Co-authored-by: Andronik Ordian <[email protected]>

* avoid the deferred setting of `is_relay_chain` in `RelayChainSelection`

* positive assertion is not mandated, only the negative one, to avoid a stall

* cleanup: overseer residue

* spellcheck

* fixin

* groundwork to obsolete Overseer::new and AllSubsystemsGen proc-macro

* Now all malus & tests can be ported to the builder pattern.

Obsoletes `Overseer::new`, `AllSubsystemsGen` derive macro, `AllSubsystems`.

* spellcheck

* adjust tests, minor fixes

* remove derive macro AllSubsystemsGen

* add forgotten file dummy.rs

* remove residue

* good news everyone!

* spellcheck

* address review comments

* fixup imports

* make it conditional

* fixup docs

* reduce import

* chore: fmt

* chore: fmt

* chore: spellcheck / nlprules

* fixup malus variant-a

* fmt

* fix

* fixins

* pfmt

* fixins

* chore: fmt

* remove expanded overseer generation

* tracing version

* Update node/network/statement-distribution/src/lib.rs

Co-authored-by: Robert Habermeier <[email protected]>

* use future::ready instead

* silence warning

* chore: fmt

Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: Robert Habermeier <[email protected]>

* Companion for Substrate#9867

* Fix finality_target_with_fallback()

* update substrate

Co-authored-by: André Silva <[email protected]>

* overseer: remove mut in connector

* rename SelectRelayChainWFallback -> SelectRelayChain

* split Basics

* introduce the OverseerConnector, use it

* introduce is_relay_chain to RelayChainSelection

* chore: rename var

* avoid dummy import in subsystem

* actually remove Disconnecte/Connected enum

* extract DummySubsystem into mod dummy.

* Handle::Connected -> Handle::new

* chore: fmt

* fix test

* select relay chain takes no arg, simplification

* fmt

* Update node/service/src/lib.rs

Co-authored-by: Andronik Ordian <[email protected]>

* chore: improve malus tests

* avoid the deferred setting of `is_relay_chain` in `RelayChainSelection`

* positive assertion is not mandated, only the negative one, to avoid a stall

* chore: fmt

* assure the `RelayChainSelection` is not used before the overseer is up and running

Co-authored-by: Andronik Ordian <[email protected]>

* Attempt to add log stats to gossip-support.

* WIP: Keep track of connected validators.

* Clarify metric.

* WIP: Make gossip support report connectivity.

* WIP: Fixing tests.

* Fix network bridge + integrate in overseer.

* Consistent naming.

* Fix logic error

* cargo fmt

* Pretty logs.

* cargo fmt

* Use `Delay` to trigger periodic checks.

* fmt

* Fix warning for authority set size of 1.

* More correct ratio report if there are no resolved validators.

* Prettier rendering of empty set.

* Fix typo.

* Another typo.

* Don't check on every leaf update.

* Make compatible with older rustc.

* Fix tests.

* Demote warning.

* add parachains pallets to Polkadot runtime

* some updates

* temp update deposits, fix compile

* lower para byte deposit

* fix genesis

* fmt

* fix spell

* updates to consts

* update benchmark list

* fix merge master

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_common::crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_common_crowdloan.rs

* skip large meta

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_common::crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_common_crowdloan.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_common::paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_common_paras_registrar.rs

* fmt

* add and update proxies

* add weight info

* weights

* lower minimum contribution to 5 DOT

* only overwrite when default

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Parity Bot <[email protected]>

This reverts commit 5f637c51.

* remove connected disconnected state from overseer

* foo

* split new partial

* fix

* refactor init code to not require a `OverseerHandle` when we don't have an overseer

* intermediate

* fixins

* X

* fixup

* foo

* fixup

* docs

* conditional

* Update node/service/src/lib.rs

* review by ladi

* rename: MsgFilter -> MessageInterceptor

* feat: add dispute metrics

* fixup

* test fixins

* fix metrics

* dummysubsystem export and trait fn fix

* chore: fmt

* undo unwanted changes

* foo

* pfmt

* fixup

* fixup

* revert

* some more

* Update node/malus/Cargo.toml

Co-authored-by: Andronik Ordian <[email protected]>

* Update node/core/dispute-coordinator/src/metrics.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Update node/core/dispute-coordinator/src/metrics.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Update node/core/dispute-coordinator/src/metrics.rs

Co-authored-by: Andronik Ordian <[email protected]>

* add license header

* fix lockfile

* new with opts

* fmt

* Update node/core/dispute-coordinator/src/metrics.rs

* feature gate

Co-authored-by: Andronik Ordian <[email protected]>

* Introduce new config: ump_max_individual_weight

* Implement overweight msg stashing

* Test

* Add migration module.

Also introduces a test for migration

* Integrate ExecuteOverweightOrigin to runtimes

* Fix more stuff

* Add `yeet` into dictionary

* Use suggested `Error` variant names

* typo

* Use 20ms as the maximum individual message weight

* Update the test value

* rustfmt

* Clean up

* Remove deprecated field from host config

* Remove missed _hrmp_open_request_ttl

* Apply typo fix suggestion

Co-authored-by: Alexander Popiak <[email protected]>

* Rename `migration::migrate_to_latest`

* Restore `_hrmp_open_request_ttl` in `v0::HostConfiguration`

* Apply suggestion for a rustdoc

* Apply the suggestion

* Test v0 config with the raw production data fetched from Kusama

* Update runtime/parachains/src/ump.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Expose migration functions

* Fix spellcheck

Co-authored-by: Alexander Popiak <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>

* make polkadot-runtime optional feature

* sprinkle some cfg statements

* ok

* ok, ok

* add CI check

* set -e

* chmod +x

* fixes

* fmt

* nicer compile errors

* Update outdated comments

* Do not expire HRMP open channel requests

* Fix the build and update the docs

* Implement canceling requests and do not remove them automatically

* Fix a borked merge

* Fix fmt

* Please spellchecker

* Apply suggestions from code review

Co-authored-by: Amar Singh <[email protected]>

* Use `mutate_exists` for maintaining request counts

* Apply `rustfmt`

* Move newly introduced entrypoint to end to preserve ordering

Co-authored-by: Amar Singh <[email protected]>

* allow config NextFreeParaId in genesis

* update westend chain spec

* comment out bridges runtime modules (broken)

* finish commenting out bridges stuff

* more tweaks

* update rococo.json

* More debugging output.

* Fix chain selection in case of disputes.

* Fix flaky test.

* Fix off-by-one bug in approval voting and adjust tests

* Address feedback

* Remove default disputes

* Remove spaces

* cargo +nightly fmt

Co-authored-by: Lldenaurois <[email protected]>
Co-authored-by: Andronik Ordian <[email protected]>

* WiP.

* Things compile.

* cargo fmt

* Passing tests + fix warnings.

* Metrics for availability recovery.

* Basic test.

* Fix typos and actually check for overflow.

* cargo fmt

* Register metrics.

* More tests.

* Fix warning.

* cargo +nightly fmt

* Fix metrics

* Get rid of unsafe.

* tabify

* spellcheck

Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* remove dead_code from chain selection test

Ref #3674

* remove convenience consts

* disable the test suite

It is crafted for usage _with_ disputes.

* fix lockfile

* cleanup of rob's patch

* depend on feature disputes, both lag and query

* typo

* make it work

TODO move the fallback into selection

* option

* Update node/service/src/relay_chain_selection.rs

* remove trait implementation and fix errors

* hotfix: tests

* fmt

* remove unused trait

* make pub(crate) for test

* disable tests with disputed blocks

* Fix warnings

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Lldenaurois <[email protected]>

Companion PR for 'Make choosing an executor an explicit part of service construction' (#9525) (#3615)

* Companion PR

* Update a few files

* Run cargo fmt

* Do better at renaming things

* More renamings

* More fixes

* oops

* Fix simnet problems

* fix compilation

* Update substrate

Co-authored-by: thiolliere <[email protected]>

* WIP: Get rid of request multiplexer.

* WIP

* Receiver for handling of incoming requests.

* Get rid of useless `Fault` abstraction.

The things the type system let us do are not worth getting abstracted in
its own type. Instead error handling is going to be merely a pattern.

* Make most things compile again.

* Port availability distribution away from request multiplexer.

* Formatting.

* Port dispute distribution over.

* Fixup statement distribution.

* Handle request directly in collator protocol.

+ Only allow fatal errors at top level.

* Use direct request channel for availability recovery.

* Finally get rid of request multiplexer

Fixes #2842 and paves the way for more back pressure possibilities.

* Fix overseer and statement distribution tests.

* Fix collator protocol and network bridge tests.

* Fix tests in availability recovery.

* Fix availability distribution tests.

* Fix dispute distribution tests.

* Add missing dependency

* Typos.

* Review remarks.

* More remarks.

* companion pr for substrate:auto_db

* update Substrate

Co-authored-by: parity-processbot <>

* Substrate companion #9491

https://github.com/paritytech/substrate/pull/9491

* update Substrate

* Fix build

Co-authored-by: parity-processbot <>

* cargo +nightly fmt

* add cargo-fmt check to ci

* update ci

* fmt

* fmt

* skip macro

* ignore bridges

* Update for the new GrandpaAPI

* Update substrate

* Update substrate

* move client consensus code out of primitives

* merging crates

* import tweak

* Fixing build: ServiceFactory is a word...

* updating lock file

* Fixed typo in error message

(to bump build)

* update Substrate

Co-authored-by: parity-processbot <>

* migration

* genesis build correctly

* migrate runtimes

* add genesis builds

* expose GenesisBuild

* chainspec migration

* fmt

* check before insert

* Reduce visibility of storage items to crate

Co-authored-by: Keith Yeung <[email protected]>

* finality_target adjustments

* fn finality_target

* partially address review comments

* fixins

* more rustic if condition

* fix tests

* fixins

* Update node/core/approval-voting/src/lib.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Update node/core/approval-voting/src/lib.rs

Co-authored-by: Robert Habermeier <[email protected]>

* review comments part one

* rename candidates -> block_descriptions

* testing outline (incomplete, WIP)

* test foo

* split RelayChainSelection into RelayChainSelection{,WithFallback}, introduce HeaderProvider{,Provider}

* make some stuff public (revert this soon™)

* some test improvements

* slips of pens

* test fixins

* add another trait abstraction

* pending edge case tests + warnings fixes

* more test cases

* fin

* chore fmt

* fix cargo.lock

* Undo obsolete changes

* // comments

* make mod pub(crate)

* fix

* minimize static bounds

* resolve number() as before

* fmt

* post merge fix

* address some nits

Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: Robert Habermeier <[email protected]>

* initial integration and migration code

* fix tests

* fix counting test

* assume the current version on missing file

* use SelectRelayChain

* remove duplicate metric

* Update node/service/src/lib.rs

Co-authored-by: Robert Habermeier <[email protected]>

* remove ApprovalCheckingVotingRule

* address my concern

* never mode for StagnantCheckInterval

* REVERTME: some logs

* w00t

* it's ugly but it works

* Revert "REVERTME: some logs"

This reverts commit e210505a2e83e31c381394924500b69277bb042e.

* it's handle, not handler

* fix a few typos

Co-authored-by: Robert Habermeier <[email protected]>

* CI: add spellcheck

* revert me

* CI: explicit command for spellchecker

* spellcheck: edit misspells

* CI: run spellcheck on diff

* spellcheck: edits

* spellcheck: edit misspells

* spellcheck: add rules

* spellcheck: mv configs

* spellcheck: more edits

* spellcheck: chore

* spellcheck: one more thing

* spellcheck: and another one

* spellcheck: seems like it doesn't get to an end

* spellcheck: new words after rebase

* spellcheck: new words appearing out of nowhere

* chore

* review edits

* more review edits

* more edits

* wonky behavior

* wonky behavior 2

* wonky behavior 3

* change git behavior

* spellcheck: another bunch of new edits

* spellcheck: new words are koming out of nowhere

* CI: finding the master

* CI: fetching master implicitly

* CI: undebug

* new errors

* a bunch of new edits

* and some more

* Update node/core/approval-voting/src/approval_db/v1/mod.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Update xcm/xcm-executor/src/assets.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <[email protected]>

* Suggestions from the code review

* CI: scan only changed files

Co-authored-by: Andronik Ordian <[email protected]>

* Dispute protocol.

* Dispute distribution protocol.

* Get network requests routed.

* WIP: Basic dispute sender logic.

* Basic validator determination logic.

* WIP: Getting things to typecheck.

* Slightly larger timeout.

* More typechecking stuff.

* Cleanup.

* Finished most of the sending logic.

* Handle active leaves updates

- Cleanup dead disputes
- Update sends for new sessions
- Retry on errors

* Pass sessions in already.

* Startup dispute sending.

* Provide incoming decoding facilities

and use them in statement-distribution.

* Relaxed runtime util requirements.

We only need a `SubsystemSender` not a full `SubsystemContext`.

* Better usability of incoming requests.

Make it possible to consume stuff without clones.

* Add basic receiver functionality.

* Cleanup + fixes for sender.

* One more sender fix.

* Start receiver.

* Make sure to send responses back.

* WIP: Exposed authority discovery

* Make tests pass.

* Fully featured receiver.

* Decrease cost of `NotAValidator`.

* Make `RuntimeInfo` LRU cache size configurable.

* Cache more sessions.

* Fix collator protocol.

* Disable metrics for now.

* Make dispute-distribution a proper subsystem.

* Fix naming.

* Code style fixes.

* Factored out 4x copied mock function.

* WIP: Tests.

* Whitespace cleanup.

* Accessor functions.

* More testing.

* More Debug instances.

* Fix busy loop.

* Working tests.

* More tests.

* Cleanup.

* Fix build.

* Basic receiving test.

* Non validator message gets dropped.

* More receiving tests.

* Test nested and subsequent imports.

* Fix spaces.

* Better formatted imports.

* Import cleanup.

* Metrics.

* Message -> MuxedMessage

* Message -> MuxedMessage

* More review remarks.

* Add missing metrics.rs.

* Fix flaky test.

* Dispute coordinator - deliver confirmations.

* Send out `DisputeMessage` on issue local statement.

* Unwire dispute distribution.

* Review remarks.

* Review remarks.

* Better docs.

* bridges in W<>R

* fix node compilation

* Update runtime/rococo/src/bridge_messages.rs

Co-authored-by: Hernando Castano <[email protected]>

* Update runtime/rococo/src/bridge_messages.rs

Co-authored-by: Hernando Castano <[email protected]>

* Update runtime/rococo/src/bridge_messages.rs

Co-authored-by: Hernando Castano <[email protected]>

* Update runtime/rococo/src/bridge_messages.rs

Co-authored-by: Hernando Castano <[email protected]>

* long line fix

* comment/remove -> comment/#[ignore]

* explicit instances

Co-authored-by: Hernando Castano <[email protected]>

* to v2 macro

* line width

* fix mock

* actually compile moxk

* compile tests

* add hooks

* origin back in mocks

* assimilate_storage

* add generic

* maybe mock compiles now

* origin between parachain and system causing problem

* change origin

* type alias as origin

* keep origin

* add aliases to tests

* remove unnecessary imports

* Parachain -> Paras

* paras test

* nvm

* use genesis build in mock

* make it easier to dbg

* revert channel sizes

* BAnon