* Requeue UMP queue items if weight exhausted

* Reduce complexity and remove Deque

* Formatting

* Formatting

* Avoid needless storage writes

* Test

* Formatting

* Docs and cleanup

* fmt

* Remove now irrelevant comment.

* Simplify `take_processed` by using `mem::take`

* Clean up & fmt: use `upward_message` directly.

* Grumbles

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Sergei Shulepov <[email protected]>

* Remove Filter and use Contains instead

* Fixes

* Remove patch

* Formatting

* update Substrate

Co-authored-by: parity-processbot <>

* last parachains migration!

* migrate runtimes

* disable frame supertrait

* add pallet::inherent

* mock

* cargo +nightly fmt

* cargo +nightly fmt

* add cargo-fmt check to ci

* update ci

* fmt

* fmt

* skip macro

* ignore bridges

* migration

* migrate runtimes

* storage visability

* Remove unused Call part

* Remove Call part import from parachain mock runtime

* Remove Call part import from test-runtime

Co-authored-by: Keith Yeung <[email protected]>

* migration

* storage visability

* migrate runtimes

* runtimes

* Update runtime/parachains/src/session_info.rs

* Remove Call part import from all runtimes for ParaSessionInfo

* Remove Call part import from runtime/test-runtime/src/lib.rs

* Remove Call part import from parachain mocks

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>

* gotta migrate them all

* migrate rococo construct_runtime

* trigger ci

* fix warnings

* get mocks to work

* add pallet to test runtime

* comments

* calm down mr tabrizi lol

* migrate

* migrate runtimes

* runtimes compile

* ci

* trigger ci

* frame v2

* migrate runtimes

* migrate more

* comments

* docs

* cleanup

* Adjust visibility on storage items

Co-authored-by: Keith Yeung <[email protected]>

* disputes module skeleton and storage

* implement dispute module initialization logic

* implement disputes session change logic

* provide dispute skeletons

* deduplication & ancient check

* fix a couple of warnings

* begin provide_dispute_data impl

* flesh out statement set import somewhat

* move ApprovalVote to shared primitives

* add a signing-payload API to explicit dispute statements

* implement statement signature checking

* some bitflags glue for observing changes in disputes

* implement dispute vote import logic

* flesh out everything except slashing

* guide: tweaks

* declare and use punishment trait

* punish validators for inconclusive disputes

* guide: tiny fix

* guide: update docs

* add disputes getter fn

* guide: small change to spam slots handling

* improve spam slots handling and fix some bugs

* finish API of disputes runtime

* define and deposit `RevertTo` log

* begin integrating disputes into para_inherent

* use precomputed slash_for/against

* return candidate hash from process_bitfields

* implement inclusion::collect_disputed

* finish integration into rest of runtime

* add Disputes to initializer

* address suggestions

* use pallet macro

* fix typo

* Update runtime/parachains/src/disputes.rs

* add test: fix pruning

* document specific behavior

* deposit events on dispute changes

* add an allow(unused) on fn disputes

* add a dummy PunishValidators implementation

* add disputes module to Rococo

* add disputes module to westend runtime

* add disputes module to test runtime

* add disputes module to kusama runtime

* guide: prepare for runtime API for checking frozenness

* remove revert digests in favor of state variable

* merge reversions

* Update runtime/parachains/src/disputes.rs

Co-authored-by: André Silva <[email protected]>

* Update runtime/parachains/src/disputes.rs

Co-authored-by: André Silva <[email protected]>

* Update runtime/parachains/src/disputes.rs

Co-authored-by: André Silva <[email protected]>

* add byzantine_threshold and supermajority_threshold utilities to primitives

* use primitive helpers

* deposit revert event when freezing chain

* deposit revert log when freezing chain

* test revert event and log are generated when freezing

* add trait to decouple disputes handling from paras inherent handling

* runtime: fix compilation and setup dispute handler

* disputes: add hook for filtering out dispute statements

* disputes: add initializer hooks to DisputesHandler

* runtime: remove disputes pallet from all runtimes

* tag TODOs

* don't import any dispute statements just yet...

* address grumbles

* fix spellcheck, hopefully

* maybe now?

* last spellcheck round

* fix runtime tests

* fix test-runtime

Co-authored-by: thiolliere <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Switch from () to AllowAllFilter

* Rename AllowAllFilter -> AllowAll

* update Substrate

Co-authored-by: parity-processbot <>

* to v2 macro

* line width

* fix mock

* actually compile moxk

* compile tests

* add hooks

* origin back in mocks

* assimilate_storage

* add generic

* maybe mock compiles now

* origin between parachain and system causing problem

* change origin

* type alias as origin

* keep origin

* add aliases to tests

* remove unnecessary imports

* Parachain -> Paras

* paras test

* nvm

* use genesis build in mock

* fix build

* fix tests

* update

* fix

* update

* Bump

Co-authored-by: Bryan Chen <[email protected]>

* Stuff to help inspect the DMP activity

* Fix teleport accounting

* Fixes

* Fixes

* Fixes

* Fixes

* Introduce plurality XCM locations

* Add RelayedFrom

* DMP dispatch weight handling.

* Add pallet for XCM sending, add routing logic.

* Update error types & doc

* Fix warnings.

* Fixes

* Fixes

* Fixes

* Bump Substrate

* Fixes

* Docs

* Docs

* Docs

* Fixes

* Fixes

* Fixes

* Update xcm/pallet-xcm/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Docs

* Fixes

* Update lib.rs

* Fixes

Co-authored-by: Shawn Tabrizi <[email protected]>

* dispute types

* add Debug to dispute primitives in std and InherentData

* use ParachainsInherentData on node-side

* change inclusion_inherent to paras_inherent

* RuntimeDebug

* add type parameter to PersistedValidationData users

* fix test client

* spaces

* fix collation-generation test

* fix provisioner tests

* remove references to inclusion inherent

* Make build with OnSetCode

* Fixes

* update Substrate

Co-authored-by: parity-processbot <>

* Add Root functions to Paras

* note past code

* fix build

* fix test build

* compile fix and force_queue_action

* add consensus log type

* origin and issue force_approve

* add origin in runtimes

* ref API

* scrape force_approve digest from header

* add parent_hash to BlockEntry

* add block_number to block entry and force_approve skeleton

* implement and plug in force-approve

* test force_approve

* test force_approve extraction

* westend runtime

* Update node/core/approval-voting/src/approval_db/v1/mod.rs

Co-authored-by: Bastian Köcher <[email protected]>

* rename

* Update runtime/parachains/src/initializer.rs

Co-authored-by: André Silva <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Replace 'Module' with 'Pallet'.

* "Update Substrate"

* fix babe usage

* fix benchmark

Co-authored-by: parity-processbot <>
Co-authored-by: thiolliere <[email protected]>

* runtime: use babe as randomness source

* runtime: fix randomness api

* bridges: fix random_seed

* parachains: use mock TestRandomness from frame_support_test

* parachains: use mock TestRandomness from frame_support_test

* runtime: update randomness source in test-runtime

* runtime: remove unused import

* parachains: add todo to audit usage of randomness api

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Add events for hrmp pallet.

* Apply review suggestions.

* initial implementation of lifecycles and upgrades

* clean up a bit

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* do not include transitioning paras in identifier

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* actions queue

* Revert "actions queue"

This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.

* collapse onboarding state

* starting actions queue

* consolidate actions queue

* schedule para initialize result

* more actions queue for upgrade/downgrade

* clean up with fully implemented actions queue

* fix tests

* fix scheduler tests

* fix hrmp tests

* fix test

* doc fixes

* fix hrmp test w/ valid para

* Update paras.md

* fix paras registrar

* Update propose_parachain.rs

* fix merge

* Introduce "shared" module

* fix rococo build

* fix up and use shared

* guide updates

* add shared config to common tests

* add shared to test-runtime

* remove println

* fix note

Co-authored-by: Gavin Wood <[email protected]>

* Claims

* Crowdloan

* Runtime common

* Parachains registrar

* Impls

* Purchase

* Slots

* parachains runtime mock

* Use MockGenesisConfig in tests

* Update runtime/common/src/claims.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Drive by fixes

The visibility modifiers are remnants of the previous structure where
HRMP wasn't a standalone module, by rather a submodule of the router
module.

* Add Currency assoc type to Config

This would allow us to reserve balance for deposits. This commit also
integrates the HRMP module in rococo, test-runtime and mocks to use the
balances pallet.

* Fix a bug that doesn't increment the age

In case the request is not confirmed, the age would be incremented but
not persisted.

* Fix cleaning the indexes

Before that change, the cleaning of the channel indexes was wrong, because it
naively removed entire rows that was pertaining to the para we delete.
This approach is flawed because it doesn't account for the rows that are
pertaining to other paras that contain the outgoing one.

This clearly violates the invariant imposed on the indexes, that all
the index rows must contain alive paras, but apart from that it also
lead to the situation where ingress index would contain the a different
set of channels that an egress have.

* Reserve currency for opening the channels

Note the ugly `unique_saturated_into` calls. The reason for them is the
currency trait accepts and defines the `Balance` associated type and the
deposit values are coming from the `HostConfiguration` where they are
defined using the `Balance`.

I figured that parameterising `HostConfiguration` would be annoying. On
the other hand, I don't expect these `unique_saturated_into` calls to
give us problems since it seems to be a reasonable assumption that this
module will be instantiated within a runtime where the Currency provided
will have a Balance that matches the one used in the configuration.

* Tests: Adapt `run_to_block` so that it submits a proper config

* Tests: exercise the deposit logic

* don't modify inherent data on heavy block

* write up current thinking on block weight detection

* extract inherent inclusion check into its own function

* put heavy block check into runtime

* the `inclusion` inherent call is Operational, not Mandatory

This resolves a lot of the trickiness about this issue, because
we no longer need to override or supplant any existing proposer
logic; the existing logic should exhibit these behaviors:

- the `inclusion` inherent is prioritized over standard transactions
- but if it's too heavy, i.e. in case of runtime upgrade, it'll be
  dropped in favor of that.

It is my belief that allowing the proposer to just not include
this data won't have any adverse effects: it's equivalent to replacing
them with empty versions of themselves, which the `ProvideInherent`
impl already does.

* Revert "the `inclusion` inherent call is Operational, not Mandatory"

This reverts commit e58858d109b18b84e7af3ac47981c6900b2d9a3e.

* Revert "write up current thinking on block weight detection"

This reverts commit fd587b80c46761b2a2b62448193348237863f99f.

* Revert "don't modify inherent data on heavy block"

This reverts commit 38299d3c23e9efb5a354d8cfa658e62a5c8c7ddf.

* add backed candidate block weight assumption to configuration

* Limit backed candidates according to a candidate weight heuristic.

This approach replaces making the inclusion inherent non-mandatory.
It's still not ideal in that we have to configure a heuristic for
how much each backed candidate 'weighs', instead of directly
measuring it somehow.

This approach also never truncates the signed bitfields. The
rationale for that depends on some assumptions:

- processing the signed bitfields is cheap compared to the
  backed candidates
- it is beneficial to the progress of the relay chain
  to update the signed bitfields even if not all backed candidates
  are updated

* simplify limit_backed_candidates and weight assumption

* don't trust the provisioner to fairly distribute candidates

* use saturating subtraction

* empty commit to restart ci

* use new mechanism for getting max block weight

* apply weight refunds to the inclusion inherent

This makes some assumptions about fundamental weights, which are
encapsulated as constants. From there, it lets Substrate know
what the actual computed weight of the inherent is.

* use a correct fixed weight for the inclusion inherent

Co-authored-by: Guillaume Thiolliere <[email protected]>

* use dynamic inclusion weight so we reduce calculated weight when excluding candidates

* don't double-count this intrinsic's weight in the block weight

* add unit tests of fn limit_backed_candidates

* add tests that the inclusion inherent's weight correctly updates

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Companion for #7810

* Added missing trait items for tests

* Add another missing trait item

* fixup

* "Update Substrate"

Co-authored-by: parity-processbot <>

* plumbing for rewarding backers

* give validators reward points for participating

* fix tests

* add bitfield rewarding

* add mocks for backing rewards

* add testing for backing & availability rewards

* implement RewardValidators on top of staking

* add to test-runtime and rococo

* add to test-runtime & rococo

* point to source on rewards values

* fix common tests

* do not reward availability anymore

* Change branch.

* Update runtime.

* Revert "Change branch."

This reverts commit 841c59f3398136c27cc235a29d7d459e8a4c8ce0.

* Update substrate.

* Fix tests.

* Fix compilation.

* Fix frame system imports.

* Fix usages of system

* Fix stuff.

* Fix compilation.

* Fixes.

* Fix block_weight usage.

* Bump substrate.

* rename Trait -> Config

* revert diener changes

* rename HostConfig to ActiveConfig as more meaningful

* fix merge

* "Update Substrate"

* cargo update -p sp-io

Co-authored-by: parity-processbot <>

* guide: fix formatting for SessionInfo module

* primitives: SessionInfo type

* punt on approval keys

* ah, revert the type alias

* session info runtime module skeleton

* update the guide

* runtime/configuration: sync with the guide

* runtime/configuration: setters for newly added fields

* runtime/configuration: set codec indexes

* runtime/configuration: update test

* primitives: fix SessionInfo definition

* runtime/session_info: initial impl

* runtime/session_info: use initializer for session handling (wip)

* runtime/session_info: mock authority discovery trait

* guide: update the initializer's order

* runtime/session_info: tests skeleton

* runtime/session_info: store n_delay_tranches in Configuration

* runtime/session_info: punt on approval keys

* runtime/session_info: add some basic tests

* Update primitives/src/v1.rs

* small fixes

* remove codec index annotation on structs

* fix off-by-one error

* validator_discovery: accept a session index

* runtime: replace validator_discovery api with session_info

* Update runtime/parachains/src/session_info.rs

Co-authored-by: Sergei Shulepov <[email protected]>

* runtime/session_info: add a comment about missing entries

* runtime/session_info: define the keys

* util: expose connect_to_past_session_validators

* util: allow session_info requests for jobs

* runtime-api: add mock test for session_info

* collator-protocol: add session_index to test state

* util: fix error message for runtime error

* fix compilation

* fix tests after merge with master

Co-authored-by: Sergei Shulepov <[email protected]>

* Guide: Split router module in guide.

Now we have: DMP, UMP and Router module.

* Add a glossary entry for what used to be called Router

* Extract DMP

* Extract UMP

* Extract HRMP

* Switch over to new modules

* Router: goodbye sweet prince

* Link to messaging overview for details.

* Update missed rococo and test runtimes.

* Commit destroyed by rebase changes

* Don't deprecate Router but rather make it a meta-project

Co-authored-by: Bernhard Schuster <[email protected]>

* Fix typos suggestion

Co-authored-by: Bernhard Schuster <[email protected]>

* Fix repetition in the impl guide

* Clarify that processed_downward_messages has the u32 type

* Remove the router subdir.

* Deabbreviate DMP,UMP,HRMP

Co-authored-by: Bernhard Schuster <[email protected]>

* HRMP: Update the impl guide

* HRMP: Incorporate the channel notifications into the guide

* HRMP: Renaming in the impl guide

* HRMP: Constrain the maximum number of HRMP messages per candidate

This commit addresses the HRMP part of https://github.com/paritytech/polkadot/issues/1869

* XCM: Introduce HRMP related message types

* HRMP: Data structures and plumbing

* HRMP: Configuration

* HRMP: Data layout

* HRMP: Acceptance & Enactment

* HRMP: Test base logic

* Update adder collator

* HRMP: Runtime API for accessing inbound messages

Also, removing some redundant fully-qualified names.

* HRMP: Add diagnostic logging in acceptance criteria

* HRMP: Additional tests

* Self-review fixes

* save test refactorings for the next time

* Missed a return statement.

* a formatting blip

* Add missing logic for appending HRMP digests

* Remove the channel contents vectors which became empty

* Tighten HRMP channel digests invariants.

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Remove a note about sorting for channel id

* Add missing rustdocs to the configuration

* Clarify and update the invariant for HrmpChannelDigests

* Make the onboarding invariant less sloppy

Namely, introduce `Paras::is_valid_para` (in fact, it already is present
in the implementation) and hook up the invariant to that.

Note that this says "within a session" because I don't want to make it
super strict on the session boundary. The logic on the session boundary
should be extremely careful.

* Make `CandidateCheckContext` use T::BlockNumber for hrmp_watermark

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* UMP: Update the impl guide

* UMP: Incorporate XCM related changes into the guide

* UMP: Data structures and configuration

* UMP: Initial plumbing

* UMP: Data layout

* UMP: Acceptance criteria & enactment

* UMP: Fix dispatcher bug and add the test for it

* UMP: Constrain the maximum size of an UMP message

This commit addresses the UMP part of https://github.com/paritytech/polkadot/issues/1869

* Fix failing test due to misconfiguration

* Make the type of RelayDispatchQueueSize be more apparent in the guide

* Revert renaming `max_upward_queue_capacity` to `max_upward_queue_count`

* convert spaces to tabs

Co-authored-by: Bernhard Schuster <[email protected]>

* Update runtime/parachains/src/router/ump.rs

Co-authored-by: Bernhard Schuster <[email protected]>

Co-authored-by: Bernhard Schuster <[email protected]>

* DMP: data structures and plumbing

* DMP: Implement DMP logic in the router module

DMP: Integrate DMP parts into the inclusion module

* DMP: Introduce the max size limit for the size of a downward message

* DMP: Runtime API for accessing inbound messages

* OCD

Small clean ups

* DMP: fix the naming of the error

* DMP: add caution about a non-existent recipient

* parachains: use the root origin for mock

* registrar: notify the router module about the cleanup

* add missing call to schedule_para_cleanup

* Initial commit.

* Fix build

* Add comments, remove Event

* Dont expose calls

* Remove TODO and origins

* Fix merge

* Enable or disable parathread registration

* Companion for `ModuleToIndex` to `PalletInfo` rename

* "Update Substrate"

Co-authored-by: parity-processbot <>
Co-authored-by: Benjamin Kampmann <[email protected]>

The purpose of this commit is to establish a foundation of the router module, it paves the way for the implementation of other features.