* Add xcm-simulator and xcm-simulator-example.

* Abstract xcmp and dmp handling.

* Use mock message queue.

* Xcm simulator example unit tests.

* Use relay chain block number on sending msg.

* Fix typo.

* fmt

* more fmt

* Fix deps.

* adds test-runner boilerplate

* revert to master

* Update node/test/runtime/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* fix warning

* use polkadot_development_config

* remove vestigial code

* ...

* remove unused dependencies

* adds simnet binary

* adds simnet binary

* merged with remote

* dummy to check pipeline

* add 2 docker files and a build cmd

* adds logging

* atempt to use binary from build host

* fix simnet-binary

* fix docker commands

* switch branches

* ...

* update docker file

* update the dockerfile 2

* add some message in the cheatsheet

* add repo to chaches stage also

* update paths

* do only 1 stage build

* add time when build cmd started

* remove debugg commands

* polkadot-simnet-substrate-working-version-v1

* reduce size of polkadot-simnet image

* update test runner api

* update test-runner

* ...

* revert to master

* Merge branch 'master' of github.com:paritytech/polkadot into substrate-test-runner

* bump impl version

* remove unused imports, fix test

* was_binary.to_vec()

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <write@reusable.software>

* ...

* remove unused import

* remove unused import

* adds post upgrade test

* dry code

* revert spec_version

* update Cargo.lock

* tested and it works

* compare runtime spec version

* fix spaces, remove docker files

* replace spaces with tabs

* Update runtime/polkadot/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* ...

* revert Cargo.lock

* bump cargo.lock

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
Co-authored-by: CI system <>

* Dispute protocol.

* Dispute distribution protocol.

* Get network requests routed.

* WIP: Basic dispute sender logic.

* Basic validator determination logic.

* WIP: Getting things to typecheck.

* Slightly larger timeout.

* More typechecking stuff.

* Cleanup.

* Finished most of the sending logic.

* Handle active leaves updates

- Cleanup dead disputes
- Update sends for new sessions
- Retry on errors

* Pass sessions in already.

* Startup dispute sending.

* Provide incoming decoding facilities

and use them in statement-distribution.

* Relaxed runtime util requirements.

We only need a `SubsystemSender` not a full `SubsystemContext`.

* Better usability of incoming requests.

Make it possible to consume stuff without clones.

* Add basic receiver functionality.

* Cleanup + fixes for sender.

* One more sender fix.

* Start receiver.

* Make sure to send responses back.

* WIP: Exposed authority discovery

* Make tests pass.

* Fully featured receiver.

* Decrease cost of `NotAValidator`.

* Make `RuntimeInfo` LRU cache size configurable.

* Cache more sessions.

* Fix collator protocol.

* Disable metrics for now.

* Make dispute-distribution a proper subsystem.

* Fix naming.

* Code style fixes.

* Factored out 4x copied mock function.

* WIP: Tests.

* Whitespace cleanup.

* Accessor functions.

* More testing.

* More Debug instances.

* Fix busy loop.

* Working tests.

* More tests.

* Cleanup.

* Fix build.

* Basic receiving test.

* Non validator message gets dropped.

* More receiving tests.

* Test nested and subsequent imports.

* Fix spaces.

* Better formatted imports.

* Import cleanup.

* Metrics.

* Message -> MuxedMessage

* Message -> MuxedMessage

* More review remarks.

* Add missing metrics.rs.

* Fix flaky test.

* Dispute coordinator - deliver confirmations.

* Send out `DisputeMessage` on issue local statement.

* Unwire dispute distribution.

* Review remarks.

* Review remarks.

* Better docs.

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <peter.r.goodspeedniklaus@gmail.com>

* bump version

* bump substrate

* bump spec_versions

* bump versions & spec_versions

* bump Cargo.lock

* crate skeleton and type definitions

* add ChainSelectionMessage

* add error type

* run loop

* fix overseer

* simplify determine_new_blocks API

* write an overlay struct and fetch new blocks

* add new function to overlay

* more flow

* add leaves to overlay and add a strong type around leaves-set

* add is_parent_viable

* implement block import, ignoring reversions

* add stagnant-at to overlay

* add stagnant

* add revert consensus log

* flow for reversions

* extract and import block reversions

* recursively update viability

* remove redundant parameter from WriteBlockEntry

* do some removal of viable leaves

* address grumbles

* refactor

* address grumbles

* add comment about non-monotonicity

* extract backend to submodule

* begin the hunt for viable leaves

* viability pivots for updating the active leaves

* remove LeafSearchFrontier

* partially -> explicitly viable and untwist some booleans

* extract tree to submodule

* implement block finality update

* Implement block approval routine

* implement stagnant detection

* ensure blocks pruned on finality are removed from the active leaves set

* write down some planned test cases

* floww

* leaf loading

* implement best_leaf_containing

* write down a few more tests to do

* remove dependence of tree on header

* guide: ChainApiMessage::BlockWeight

* node: BlockWeight ChainAPI

* fix compile issue

* note a few TODOs for the future

* fetch block weight using new BlockWeight ChainAPI

* implement unimplemented

* sort leaves by block number after weight

* remove warnings and add more TODOs

* create test module

* storage for test backend

* wrap inner in mutex

* add write waker query to test backend

* Add OverseerSignal -> FromOverseer conversion

* add test harnes

* add no-op test

* add some more test helpers

* the first test

* more progress on tests

* test two subtrees

* determine-new-blocks: cleaner genesis avoidance and tighter ancestry requests

* don't make ancestry requests when asking for one block

* add a couple more tests

* add to AllMessages in guide

* remove bad spaces from bridge

* compact iterator

* test import with gaps

* more reversion tests

* test finalization pruning subtrees

* fixups

* test clobbering and fix bug in overlay

* exhaustive backend state after finalizaiton tested

* more finality tests

* leaf tests

* test approval

* test ChainSelectionMessage::Leaves thoroughly

* remove TODO

* avoid Ordering::is_ne so CI can build

* comment algorithmic complexity

* Update node/core/chain-selection/src/lib.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* bump spec_version and version

* bump substrate

* remove migrations

* Revert "remove migrations"

This reverts commit ceeab924.

* remove stale migration for kusama + westend

* implement dispute participation subsystem

* guide: minor fix in dispute participation

* Update node/core/dispute-participation/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* dispute: add comments to participation logic

* dispute-coordinator: fix test compilation

* implementers-guide: update dispute participation

* dispute-participation: add error for missing validation code

* dispute-participation: add tests

* Update node/core/dispute-participation/src/lib.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* guide: update overseer protocol dispute participation message

* dispute-participation: remove duplication in tests

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* skeleton for dispute-coordinator

* add coordinator and participation message types

* begin dispute-coordinator DB

* functions for loading

* implement strongly-typed DB transaction

* add some tests for DB transaction

* core logic for pruning

* guide: update candidate-votes key for coordinator

* update candidate-votes key

* use big-endian encoding for session, and implement upper bound generator

* finish implementing pruning

* add a test for note_current_session

* define state of the subsystem itself

* barebones subsystem definition

* control flow

* more control flow

* implement session-updating logic

* trace

* control flow for message handling

* Update node/core/dispute-coordinator/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update node/subsystem/src/messages.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* some more control flow

* guide: remove overlay

* more control flow

* implement some DB getters

* make progress on importing statements

* add SignedDisputeStatement struct

* move ApprovalVote to shared primitives

* add a signing-payload API to explicit dispute statements

* add signing-payload to CompactStatement

* add relay-parent hash to seconded/valid dispute variatns

* correct import

* type-safe wrapper around dispute statements

* use checked dispute statement in message type

* extract rolling session window cache to subsystem-util

* extract session window tests

* approval-voting: use rolling session info cache

* reduce dispute window to match runtime in practice

* add byzantine_threshold and supermajority_threshold utilities to primitives

* integrate rolling session window

* Add PartialOrd to CandidateHash

* add Ord to CandidateHash

* implement active dispute update

* add dispute messages to AllMessages

* add dispute stubs to overseer

* inform dispute participation to participate

* implement issue_local_statement

* implement `determine_undisputed_chain`

* fix warnings

* test harness for dispute coordinator tests

* add more helpers to test harness

* add some more helpers

* some tests for dispute coordinator

* ignore wrong validator indices

* test finality voting rule constraint

* add more tests

* add variants to network bridge

* fix test compilation

* remove most dispute coordinator functionality

as of #3222 we can do most of the work within the approval voting subsystem

* Revert "remove most dispute coordinator functionality"

This reverts commit 9cd615e8

.

* Use thiserror

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Update node/core/dispute-coordinator/src/lib.rs

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* extract tests to separate module

* address nit

* adjust run_iteration API

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Disable color-eyre default features

* update Substrate

Co-authored-by: parity-processbot <>

* polkadot-service: Make native runtime configurable

This pull requests adds support for configuring the native runtimes used
by polkadot-service. While this whole pr doesn't change that much for
polkadot, besides not having the light-node enabled for the default
polkadot binary. However, downstream projects (parachains) will have a
much better compile time. In cumulus for example the `cargo test --all
--release` is about 4m faster to compile.

* Fixes

* Fix

* Enable rococo-native

* Fix light client

* 🤦

* Fixes

* Create validator_side module

* Subsume Candidate Selection

* Add test to ensure candidate backing logic is correct

* Ensure secondings are adequately cleaned up and address test flakyness

* Address Feedback

* bump version, substrate, beefy

* bump substrate

* Bump Rococo/Wococo `spec_version` used by the bridge relayer

* Prevent bridge relayer from being part of workspace

This allows us to build the relayer without pulling in all its
dependencies into the Polkadot repo by indicating its manifest path when
building.

* Revert "Bump Rococo/Wococo `spec_version` used by the bridge relayer"

This reverts commit 90fdeecb.

* bump version & spec version

* update Cargo.lock

* bump substrate

* Add bridges code to workspace

* Add Rococo and Wococo bridge instances to Rococo runtime

* Add Rococo and Wococo runtime APIs

* Add GenesisConfig parameters for bridge pallet

* Update Rococo spec_version used by bridge relayer

* Add scripts for running Rococo and Wococo dev nodes

* Add scripts for running Rococo<>Wococo header sync

* Apply patch for build artifact location

* Remove bridges crates from workspace

* Downgrade async related dependencies

* Change bridge pallet owner to be `root_key`

* Bump number of `MaxRequests` allowed

* Revert changes in `bridges` subtree folder

* Use correct account for Sudo

* Add comment explaining duplicate bridge pallets

* Remove WeightInfo comment.

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>

* bump version and spec_version

* fix generate_release_text.rb

* Revert "fix generate_release_text.rb"

This reverts commit 9e01e0ae.

* fix publish_draft_release

* fix publish_draft_release

* update target cache

* Update branch

* Make it compile

* Compile

* gate approval-checking logic (#2470)

* Fix build

* Updates

* Fix merge

* Adds missing crate

* Companion for Substrate#8386

https://github.com/paritytech/substrate/pull/8386



* Fix fix fix

* Fix

* Fix compilation

* Rewrite to `ParachainsInherentDataProvider`

* Make it compile

* Renamings

* Revert stuff

* Remove stale file

* Guide updates

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Apply suggestions from code review

* Reset accidental changes

* More

* Remove stale file

* update Substrate

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: parity-processbot <>

* bump version and unify spec version

* bump substrate

* spec_version to 900

* bump version

* bump cargo

* update benchmarks

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_identity --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

git-subtree-dir: bridges
git-subtree-split: 89a76998f93c8219e9b1f785dcce73d4891e7068

* Introduce plurality XCM locations

* Add RelayedFrom

* DMP dispatch weight handling.

* Add pallet for XCM sending, add routing logic.

* Update error types & doc

* Fix warnings.

* Fixes

* Fixes

* Fixes

* Bump Substrate

* Fixes

* Docs

* Docs

* Docs

* Fixes

* Fixes

* Fixes

* Update xcm/pallet-xcm/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Docs

* Fixes

* Update lib.rs

* Fixes

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Implement PVF validation host

* WIP: Diener

* Increase the alloted compilation time

* Add more comments

* Minor clean up

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix pruning artifact removal

* Fix formatting and newlines

* Fix the thread pool

* Update node/core/pvf/src/executor_intf.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove redundant test declaration

* Don't convert the path into an intermediate string

* Try to workaround the test failure

* Use the puppet_worker trick again

* Fix a blip

* Move `ensure_wasmtime_version` under the tests mod

* Add a macro for puppet_workers

* fix build for not real-overseer

* Rename the puppet worker for adder collator

* play it safe with the name of adder puppet worker

* Typo: triggered

* Add more comments

* Do not kill exec worker on every error

* Plumb Duration for timeouts

* typo: critical

* Add proofs

* Clean unused imports

* Revert "WIP: Diener"

This reverts commit ff2d3ff2

.

* Sync version of wasmtime

* Update cargo.lock

* Update Substrate

* Merge fixes still

* Update wasmtime version in test

* bastifmt

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Squash spaces

* Trailing new line for testing.rs

* Remove controversial code

* comment about biasing

* Fix suggestion

* Add comments

* make it more clear why unwrap_err

* tmpfile retry

* proper proofs for claim_idle

* Remove mutex from ValidationHost

* Add some more logging

* Extract exec timeout into a constant

* Add some clarifying logging

* Use blake2_256

* Clean up the merge

Specifically the leftovers after removing real-overseer

* Update parachain/test-parachains/adder/collator/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>

* remove real-overseer

* overseer: only activate leaves which support parachains

* integrate HeadSupportsParachains into service

* remove unneeded line

* Macro for generating `SlotRange`

* remove test code

* refactor slots into crate

* add no_std flag

* finish macro

* make compile

* fix copyright date

* don't assume lease periods per slot

* Update runtime/common/Cargo.toml

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* fixes

* Update lib.rs

* tests

* Move consts into struct

* docs

* fix compile

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update Cargo.lock

* Update runtime/common/src/slot_range.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* fixes

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Indentation fix.

* Prepare request-response for PoV fetching.

* Drop old PoV distribution.

* WIP: Fetch PoV directly from backing.

* Backing compiles.

* Runtime access and connection management for PoV distribution.

* Get rid of seemingly dead code.

* Implement PoV fetching.

Backing does not yet use it.

* Don't send `ConnectToValidators` for empty list.

* Even better - no need to check over and over again.

* PoV fetching implemented.

+ Typechecks
+ Should work

Missing:

- Guide
- Tests
- Do fallback fetching in case fetching from seconding validator fails.

* Check PoV hash upon reception.

* Implement retry of PoV fetching in backing.

* Avoid pointless validation spawning.

* Add jaeger span to pov requesting.

* Add back tracing.

* Review remarks.

* Whitespace.

* Whitespace again.

* Cleanup + fix tests.

* Log to log target in overseer.

* Fix more tests.

* Don't fail if group cannot be found.

* Simple test for PoV fetcher.

* Handle missing group membership better.

* Add test for retry functionality.

* Fix flaky test.

* Spaces again.

* Guide updates.

* Spaces.

* bump version and substrate

* bump old forgotten versions

* update weights

* bump substrate

* Revert "bump substrate"

This reverts commit 8b5004b6

.

Co-authored-by: André Silva <andrerfosilva@gmail.com>

* initial subsystem for issuing connection requests

* finish the initial impl

* integrate with the overseer

* rename to gossip-support

* fix renamings leftover

* remove run_inner

* fix compilation

* random subset of sqrt

* Add instructions.

* Squashed 'bridges/' content from commit 345e84a21

git-subtree-dir: bridges
git-subtree-split: 345e84a2146b56628e9888c9f5e129cb40e868a9

* Remove bridges workspace file to avoid confusing Cargo.

* Add some bridges primitives to Polkadot workspace.

* Improve docs.

* test: add unit test to catch missing distribution to subsystems faster

* add a simple count

* introduce proc macro to generate dispatch type

* refactor

* refactor

* chore: add license

* fixup unit test

* fixup merge

* better errors

* better fmt

* fix error spans

* better docs

* better error messages

* ui test foo

* Update node/subsystem/dispatch-gen/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update node/network/bridge/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update node/subsystem/Cargo.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update node/subsystem/dispatch-gen/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update node/subsystem/dispatch-gen/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update node/network/bridge/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* fix compilation

* use find_map

* drop the silly 2, use _inner instead

* Update node/network/bridge/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update node/subsystem/dispatch-gen/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* nail deps down

* more into()

* flatten

* missing use statement

* fix messages order

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Andronik Ordian <write@reusable.software>

* bump version and substrate

* update weights

* bump substrate

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=frame_system --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

* Template

* Midway..

* Make all 3 runtimes work

* Update node/service/Cargo.toml

* undo brach updates

* fix with latest version

* Update runtime/kusama/src/lib.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update runtime/rococo/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Final nits

* Update runtime/kusama/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>