* node/approval-voting: Continue to migrate tests to subsystem tests

* node/approval-voting: Continue to implement subsystem tests.

* Add more tests

* node/approval-voting: Add more tests

* node/approval-voting: Difficuly determining the proper test scenario for threshold tests

* node/approval-voting: Introduce should_trigger_assignment tests

* node/approval-voting: Finalize threshold tests

* node/approval-voting: Address Feedback and add comments

* node/approval-voting: Tidy up approval tests

* Tidy up tests

* Fix rustfmt

* Better size limits.

* Remove reference to private repo.

* Update node/network/protocol/src/request_response/mod.rs

Fix statement response size.

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Andronik Ordian <write@reusable.software>

Fixes #3557

* Add `max_memory_pages`

* update Substrate

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: parity-processbot <>

* Change pallet naming in test-runtime

This is required to make the `well_know_keys::ACTIVE_CONFIG` match the
`ActiveConfig` key.

* Use correct name

* 🤦

* cargo +nightly fmt

* add cargo-fmt check to ci

* update ci

* fmt

* fmt

* skip macro

* ignore bridges

* Update for the new GrandpaAPI

* Update substrate

* Update substrate

* move client consensus code out of primitives

* merging crates

* import tweak

* Fixing build: ServiceFactory is a word...

* updating lock file

* Fixed typo in error message

(to bump build)

* update Substrate

Co-authored-by: parity-processbot <>

* migration

* genesis build correctly

* migrate runtimes

* add genesis builds

* expose GenesisBuild

* chainspec migration

* fmt

* check before insert

* Reduce visibility of storage items to crate

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* change ActiveLeaves to contain at most one activated

* fix test

* finality_target adjustments

* fn finality_target

* partially address review comments

* fixins

* more rustic if condition

* fix tests

* fixins

* Update node/core/approval-voting/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update node/core/approval-voting/src/lib.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* review comments part one

* rename candidates -> block_descriptions

* testing outline (incomplete, WIP)

* test foo

* split RelayChainSelection into RelayChainSelection{,WithFallback}, introduce HeaderProvider{,Provider}

* make some stuff public (revert this soon™)

* some test improvements

* slips of pens

* test fixins

* add another trait abstraction

* pending edge case tests + warnings fixes

* more test cases

* fin

* chore fmt

* fix cargo.lock

* Undo obsolete changes

* // comments

* make mod pub(crate)

* fix

* minimize static bounds

* resolve number() as before

* fmt

* post merge fix

* address some nits

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* initial integration and migration code

* fix tests

* fix counting test

* assume the current version on missing file

* use SelectRelayChain

* remove duplicate metric

* Update node/service/src/lib.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* remove ApprovalCheckingVotingRule

* address my concern

* never mode for StagnantCheckInterval

* REVERTME: some logs

* w00t

* it's ugly but it works

* Revert "REVERTME: some logs"

This reverts commit e210505a

.

* it's handle, not handler

* fix a few typos

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* node/dispute-coordinator: Introduce resume capability

This commit introduces a resume capability for the
dispute coordinator subsystem. Specifically, this will allow
to recover data for disputes for which we have no local statements.

* node/dispute-coordinator: Add resume function to TestState and modify Harness

This commit modifies the TestHarness to return a TestState. We subsequently
define a resume function on TestState that allows to interrupt the test and
test specifically for behavior on startup of the subsystem.

* node/dispute-coordinator: Implement resume functionality

This commit implements the resume functionality for the subsystem.
In addition, we will forward any DisputeParticipation::Participate
message in order to ensure that disputes for which we do not have
local statements may be recovered in due time.

* Address Feedback

* Modify to run handle_leaf on first import

* Modify missing_local_statement logic

* node/dispute-coordinator: Add simple test to ensure we adequately
handle local_statements that are not missing.

* Add missing keystore tests

Bumps [indexmap](https://github.com/bluss/indexmap) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.6.1...1.7.0

)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix

* update

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.91 to 0.2.98.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.91...0.2.98

)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [async-process](https://github.com/smol-rs/async-process) from 1.0.1 to 1.1.0.
- [Release notes](https://github.com/smol-rs/async-process/releases)
- [Changelog](https://github.com/smol-rs/async-process/blob/master/CHANGELOG.md)
- [Commits](https://github.com/smol-rs/async-process/compare/v1.0.1...v1.1.0

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: CI system <>

* CI: add spellcheck

* revert me

* CI: explicit command for spellchecker

* spellcheck: edit misspells

* CI: run spellcheck on diff

* spellcheck: edits

* spellcheck: edit misspells

* spellcheck: add rules

* spellcheck: mv configs

* spellcheck: more edits

* spellcheck: chore

* spellcheck: one more thing

* spellcheck: and another one

* spellcheck: seems like it doesn't get to an end

* spellcheck: new words after rebase

* spellcheck: new words appearing out of nowhere

* chore

* review edits

* more review edits

* more edits

* wonky behavior

* wonky behavior 2

* wonky behavior 3

* change git behavior

* spellcheck: another bunch of new edits

* spellcheck: new words are koming out of nowhere

* CI: finding the master

* CI: fetching master implicitly

* CI: undebug

* new errors

* a bunch of new edits

* and some more

* Update node/core/approval-voting/src/approval_db/v1/mod.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Update xcm/xcm-executor/src/assets.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <write@reusable.software>

* Suggestions from the code review

* CI: scan only changed files

Co-authored-by: Andronik Ordian <write@reusable.software>

* adds test-runner boilerplate

* revert to master

* Update node/test/runtime/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

* fix warning

* use polkadot_development_config

* remove vestigial code

* ...

* remove unused dependencies

* adds simnet binary

* adds simnet binary

* merged with remote

* dummy to check pipeline

* add 2 docker files and a build cmd

* adds logging

* atempt to use binary from build host

* fix simnet-binary

* fix docker commands

* switch branches

* ...

* update docker file

* update the dockerfile 2

* add some message in the cheatsheet

* add repo to chaches stage also

* update paths

* do only 1 stage build

* add time when build cmd started

* remove debugg commands

* polkadot-simnet-substrate-working-version-v1

* reduce size of polkadot-simnet image

* update test runner api

* update test-runner

* ...

* revert to master

* Merge branch 'master' of github.com:paritytech/polkadot into substrate-test-runner

* bump impl version

* remove unused imports, fix test

* was_binary.to_vec()

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <write@reusable.software>

* ...

* remove unused import

* remove unused import

* adds post upgrade test

* dry code

* revert spec_version

* update Cargo.lock

* tested and it works

* compare runtime spec version

* fix spaces, remove docker files

* replace spaces with tabs

* Update runtime/polkadot/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* ...

* revert Cargo.lock

* bump cargo.lock

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: radupopa2010 <radupopa2010@yahoo.com>
Co-authored-by: CI system <>

* node/dispute-coordinator: Modify db to return SubsystemResult.

In preparation of moving to the overlayed backend pattern, this commit
moves the db to return SubsystemResult values.

* node/dispute-coordinator: Add the Backend and OverlayedBackend.

This commit adds the backend and overlayed backend structs to the
dispute-coordinator subsystem.

* node/dispute-coordinator: Implement backend and overlayed-backend.

This commit finalizes the move from the previous transactional model
to the common overlay pattern in subsystem persistency. This can be
observed in the ApprovalVoting and ChainSelection subsystems.

* Add module docs + license

* Touchup merge

* disputes: Allow batch queries in dispute-coordinator

This commit moves to batch queries when responding to QueryCandidateVotes
messages. This simplifies the code in the provisioner and dispute-coordinator
by no longer requiring to make use of a FuturesOrdered when awaiting multiple
quries. Instead, the provisioner need only request the batch itself.

* node/approval-voting: Address Feedback to fail on query element missing.

* Address feedback

* Fix implementer's guide

* finer grained runtime-api caching

* fix av-store

* simplify a request

* remove unused imports

* fix tests

* Move on to next validator after timeout.

* Better naming.

* Wrong implementation of validator fetch timeouts.

* Validator side: Move on to next collator

if download takes too long.

* Drop multiple requests from same validator.

* Add test that next response is sent after timeout.

* Multiple requests by same validator should get dropped.

* Test that another collator is tried

after exclusive download time.

* Add dep.

* Cleanup.

* Merge fix.

* Review remarks.

* Fixes.

* Add log targets to trace logs

Co-authored-by: Andronik Ordian <write@reusable.software>

* add a from_backing_statement to SignedDisputeStatement

* inform dispute coordinator of all backing statements

* add dispute coordinator message to backing tests

* send positive dispute statement with every approval

* issue disputes when encountering invalid candidates.

* try to fix flaky test for CI (passed locally)

* guide: keep track of concluded-positive disputes until pruned

* guide: block implications

* guide: new dispute inherent flow

* mostly implement recency changes for dispute coordinator

* add a clock to dispute coordinator

* adjust DB tests

* fix and add new dispute coordinator tests

* provisioner: select disputes

* import all validators' approvals

* address nit: refactor backing statement submission

* gracefully handle disconnected dispute coordinator

* remove `review` comment

* fix up old_tests

* fix approval-voting compilation

* fix backing compilation

* use known-leaves in WaitForActivation

* follow-up test fixing

* add back allow(dead_code)

* Fix weights

* try something to unbreak companion

* update Substrate

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: parity-processbot <>

* remove duplicate thiserror annotations

* chore: cargo spellcheck

* Dispute protocol.

* Dispute distribution protocol.

* Get network requests routed.

* WIP: Basic dispute sender logic.

* Basic validator determination logic.

* WIP: Getting things to typecheck.

* Slightly larger timeout.

* More typechecking stuff.

* Cleanup.

* Finished most of the sending logic.

* Handle active leaves updates

- Cleanup dead disputes
- Update sends for new sessions
- Retry on errors

* Pass sessions in already.

* Startup dispute sending.

* Provide incoming decoding facilities

and use them in statement-distribution.

* Relaxed runtime util requirements.

We only need a `SubsystemSender` not a full `SubsystemContext`.

* Better usability of incoming requests.

Make it possible to consume stuff without clones.

* Add basic receiver functionality.

* Cleanup + fixes for sender.

* One more sender fix.

* Start receiver.

* Make sure to send responses back.

* WIP: Exposed authority discovery

* Make tests pass.

* Fully featured receiver.

* Decrease cost of `NotAValidator`.

* Make `RuntimeInfo` LRU cache size configurable.

* Cache more sessions.

* Fix collator protocol.

* Disable metrics for now.

* Make dispute-distribution a proper subsystem.

* Fix naming.

* Code style fixes.

* Factored out 4x copied mock function.

* WIP: Tests.

* Whitespace cleanup.

* Accessor functions.

* More testing.

* More Debug instances.

* Fix busy loop.

* Working tests.

* More tests.

* Cleanup.

* Fix build.

* Basic receiving test.

* Non validator message gets dropped.

* More receiving tests.

* Test nested and subsequent imports.

* Fix spaces.

* Better formatted imports.

* Import cleanup.

* Metrics.

* Message -> MuxedMessage

* Message -> MuxedMessage

* More review remarks.

* Add missing metrics.rs.

* Fix flaky test.

* Dispute coordinator - deliver confirmations.

* Send out `DisputeMessage` on issue local statement.

* Unwire dispute distribution.

* Review remarks.

* Review remarks.

* Better docs.

* node/approval-voting: Introduce Backend trait and Overlaybackend

This commit introduces a Backend trait and attempts to move away
from the Action model via an OverlayBackend as in the ChainSelection
subsystem.

* node/approval-voting: Add WriteOps for StoredBlockRange and BlocksAtHeight

* node/approval-voting: Add load_all_blocks to overlay

* node/approval-voting: Get all module tests to pass.

This commit modifies all tests to ensure tests are passing.

* node/approval-voting: Address oversights in the previous commit

This commit addresses some oversights in the prior commit.

1. Inner errors in backend.write were swallowed
2. One-off write functions removed to avoid useless abstraction
3. Touch-ups in general

* node/approval-voting: Move from TestDB to dyn KeyValueDB

This commit removes the TestDB from tests.rs and replaces it with
an in-memory kvdb.

* node/approval-voting: Address feedback

* node/approval-voting: Add license to ops.rs

* node/approval-voting: Address second-pass feedback

* Add TODO

* node/approval-voting: Bump spec_version

* node/approval-voting: Address final comments.

* node/approval-voting: Introduce framework for full subsystem tests

* node/approval-voting: Introduce basic tests to attempt to provide
coverage via full subsystem tests

* node/approval-voting: Introduce Chainbuilder

* node/approval-voting: Introduce Backend trait and Overlaybackend

This commit introduces a Backend trait and attempts to move away
from the Action model via an OverlayBackend as in the ChainSelection
subsystem.

* node/approval-voting: Add WriteOps for StoredBlockRange and BlocksAtHeight

* node/approval-voting: Add load_all_blocks to overlay

* node/approval-voting: Get all module tests to pass.

This commit modifies all tests to ensure tests are passing.

* node/approval-voting: Address oversights in the previous commit

This commit addresses some oversights in the prior commit.

1. Inner errors in backend.write were swallowed
2. One-off write functions removed to avoid useless abstraction
3. Touch-ups in general

* node/approval-voting: Move from TestDB to dyn KeyValueDB

This commit removes the TestDB from tests.rs and replaces it with
an in-memory kvdb.

* node/approval-voting: Address feedback

* node/approval-voting: Add license to ops.rs

* node/approval-voting: Address second-pass feedback

* Add TODO

* node/approval-voting: Bump spec_version

* node/approval-voting: Address final comments.

* update benchmark

* add host function

* update Substrate

* fix unused

Co-authored-by: parity-processbot <>

* Add `canonicalize_nans` and enable `stack_depth_metering`

* Update to the latest change in the substrate PR

* Explain why the numbers are what they are.

* Update Substrate to the latest master

Co-authored-by: parity-processbot <>

* PVF: unresponsive worker doesn't mean the candidate is bad

* s/if let Some/.is_some

* patch branch

* patch bis

* Add missing client method.

* unpatch

* update Substrate

Co-authored-by: parity-processbot <>

* DB skeleton

* key formats

* lexicographic test

* custom types for DB

* implement backend for db-v1

* remove VoidBackend and integrate with real DbBackend

* detect stagnant blocks on in interval

* fix tests

* add tests for stagnant

* send ChainSelectionMessage::Approved

* tests for DB backend

* unused import

* upgrade kvdb-memorydb

Co-authored-by: Andronik Ordian <write@reusable.software>

* bridges in W<>R

* fix node compilation

* Update runtime/rococo/src/bridge_messages.rs

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Update runtime/rococo/src/bridge_messages.rs

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Update runtime/rococo/src/bridge_messages.rs

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* Update runtime/rococo/src/bridge_messages.rs

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>

* long line fix

* comment/remove -> comment/#[ignore]

* explicit instances

Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>