* replace HistoricalValidationCode usages with ValidationCodeByHash

* runtime-api: tabify tests file

* update implementers guide

* remove some 'allow(unused)'

* fix historical validation code runtime API

* Relax Origin Checks in Registrar

* fix tests

* Update runtime/common/src/paras_registrar.rs

* introduce para locks

* apply a lock after upgrade

* add test

* add lock when creating crowdloan

* add disputes members to `HostConfiguration`

* revert bad rename

* code stored in para + modify CandidateDescriptor.

* WIP: digest + some more impl

* validation_code_hash in payload + check in inclusion

* check in client + refator

* tests

* fix encoding indices

* remove old todos

* fix test

* fix test

* add test

* fetch validation code inside collation-generation from the relay-chain

* HashMismatch -> PoVHashMismatch + miscompilation

* refactor, store hash when needed

* storage rename: more specific but slightly too verbose

* do not hash on candidate validation, fetch hash instead

* better test

* fix test

* guide updates

* don't panic in runtime

Co-authored-by: Robert Habermeier <[email protected]>

* Add Root functions to Paras

* note past code

* fix build

* fix test build

* compile fix and force_queue_action

* Replace 'Module' with 'Pallet'.

* "Update Substrate"

* fix babe usage

* fix benchmark

Co-authored-by: parity-processbot <>
Co-authored-by: thiolliere <[email protected]>

* TODOs

* Add auctions.rs, comment on changes needed.

* Remove cruft from slots

* Remove more from auctions.rs

* More logic drafting in slots.

* More logic in slots.rs

* patch some errors

* more fixes

* last nit

* Cleanups in slots.rs

* Cleanups in slots.rs

* patches

* make build

* crowdloan to new api

* auction compile

* Use ParaId instead of FundIndex in Crowdloan (#2303)

* use paraid instead of fundindex

* Update crowdloan.rs

* check caller is manager

* Auction tests and fix build warnings.

* Configurable origin for initiating auctions

* Remove on_finalize

* #2303 (manual merge)

* Tests for Slots

* some registrar tests

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Slots uses Registrar for CurrentChains

* swap works test

* on swap impl

* traitify parachain cleanup

* explicit lifecycle tracking for paras

* initial implementation of lifecycles and upgrades

* clean up a bit

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* Revert "explicit lifecycle tracking for paras"

This reverts commit 4177af7ba473bbd9c26bccd861793f25265b6657.

* remove lifecycle tracking from registrar

* do not include transitioning paras in identifier

* Update paras_registrar.rs

* final patches to registrar

* Fix test

* use noop in test

* clean up pending swap on deregistration

* finish registrar tests

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* Remove direct need for Slots and Registrar from Crowdloan

* Rejig things slightly

* actions queue

* Revert "actions queue"

This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.

* Traitify Auction interface.

* Mockups and initial code for Crowdloan testing

* One test...

* collapse onboarding state

* fix some crowdloan tests

* one more

* start benchmarks for auctions

* benchmark bid

* fix more crowdloan tests

* onboard and begin retirement no longer exist

* Revert "onboard and begin retirement no longer exist"

This reverts commit 2e100fd94e3540bff5f172328b5d917896f1c6fc.

* Simplify crowdloan and make it work.

* Fixes

* fix some

* finish merge fixes

* fix refund bug in auctions

* Add traits to Registrar for tests and benchmarks

* fix more auction benchmarks

* Fix TestAuctioneer

* finish crowdloan benchmarks

* start setting up full integration tests

* expand integration tests

* finish basic integration test

* add more integration tests

* begin slots benchmarks

* start paras registrar benchmarks

* fix merge

* fix tests

* clean up paras registrar

* remove println

* remove outdated cleanup config

* update benchmarks

* Add WeightInfo

* enable runtime-benchmarks feature flag

* complete swap benchmark

* add parachains and onboarding into westend

* add benchmarks and genesis

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix benchmark execution

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Use `new_raise_len` in crowdloan on_initialize

* Update paras_registrar.rs

* fix westend merge

* impl on_swap for crowdloan

* Check fund exists before create

* update for crowdloan sig

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* slots on_initialize

* use integration tests environment for benchmarks

* fix hrmp event

* auction on_initialize

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix storage name in auctions

* add auction_index to winning data

* winning data takes into account current auction index

* remove println

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Revert "add auction_index to winning data"

* PastRandomness.

* Fixes

* Use new randomness

* fix use of randomness in auctions and runtime config

* expose consts

* fix auction test

* add deposit per byte for para registration

* basic swap integration test

* make swap test more comprehensive

* Add WinningVec for easier retrieval in the front-end.

* clean up `WinningVec` at the end

* Add event for when a new best bid comes in

* Fix propagation of winners in ending period

* fix benchmarks, refund weight in dissolve

* fix unused

* remove some TODOs

* setup opaque keys for paras in westend

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* remove unused

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* back to regular runtime config

* use saturating math where user input can be

* better first slot check

* Update runtime/common/src/claims.rs

* update westend onswap impl

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Guillaume Thiolliere <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>

* guide: max_validators

* guide: new approach

* restrict validators based on configurable max

* add some tests

* fix wasm build (Vec)

* clean up warnings

* add logging

* set validator indices in tests as well

* fix common tests

* Update runtime/parachains/src/util.rs

Co-authored-by: Andronik Ordian <[email protected]>

Co-authored-by: Andronik Ordian <[email protected]>

* initial implementation of lifecycles and upgrades

* clean up a bit

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* do not include transitioning paras in identifier

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* actions queue

* Revert "actions queue"

This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.

* collapse onboarding state

* starting actions queue

* consolidate actions queue

* schedule para initialize result

* more actions queue for upgrade/downgrade

* clean up with fully implemented actions queue

* fix tests

* fix scheduler tests

* fix hrmp tests

* fix test

* doc fixes

* fix hrmp test w/ valid para

* Update paras.md

* fix paras registrar

* Update propose_parachain.rs

* fix merge

* Introduce "shared" module

* fix rococo build

* fix up and use shared

* guide updates

* add shared config to common tests

* add shared to test-runtime

* remove println

* fix note

Co-authored-by: Gavin Wood <[email protected]>

* initial hack in

* finish up

* use notification to pass outgoing paras

* move outgoing paras from notifications

* missing comma

* update guides

* clean up

* Claims

* Crowdloan

* Runtime common

* Parachains registrar

* Impls

* Purchase

* Slots

* parachains runtime mock

* Use MockGenesisConfig in tests

* Update runtime/common/src/claims.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

Co-authored-by: Guillaume Thiolliere <[email protected]>

* initial implementation of lifecycles and upgrades

* clean up a bit

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* do not include transitioning paras in identifier

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* actions queue

* Revert "actions queue"

This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.

* collapse onboarding state

Co-authored-by: Gavin Wood <[email protected]>

* Adds propose parachain pallet

* Update runtime/rococo/src/propose_parachain.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Fix runtime benchmarks

* Get rid of staking

* Fix benchmarking feature..

* Remove accidentally added crate

* Bump Rococo spec_version

Co-authored-by: Shawn Tabrizi <[email protected]>

* rename Trait -> Config

* revert diener changes

* rename HostConfig to ActiveConfig as more meaningful

* fix merge

* "Update Substrate"

* cargo update -p sp-io

Co-authored-by: parity-processbot <>

* update tiny-keccak to 0.2

* update deps except bitvec and shared_memory

* fix some warning after futures upgrade

* remove useless package rename caused by bug in cargo-upgrade

* revert parity-util-mem *

* remove unused import

* cargo update

* remove all renames on parity-scale-codec

* remove the leftovers

* remove unused dep

* Add a sudo wrapper for Dmp::queue_downward_message

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Stylistic changes

* Remove trailing whitespaces

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Guide: Split router module in guide.

Now we have: DMP, UMP and Router module.

* Add a glossary entry for what used to be called Router

* Extract DMP

* Extract UMP

* Extract HRMP

* Switch over to new modules

* Router: goodbye sweet prince

* Link to messaging overview for details.

* Update missed rococo and test runtimes.

* Commit destroyed by rebase changes

* Don't deprecate Router but rather make it a meta-project

Co-authored-by: Bernhard Schuster <[email protected]>

* Fix typos suggestion

Co-authored-by: Bernhard Schuster <[email protected]>

* Fix repetition in the impl guide

* Clarify that processed_downward_messages has the u32 type

* Remove the router subdir.

* Deabbreviate DMP,UMP,HRMP

Co-authored-by: Bernhard Schuster <[email protected]>

* HRMP: Update the impl guide

* HRMP: Incorporate the channel notifications into the guide

* HRMP: Renaming in the impl guide

* HRMP: Constrain the maximum number of HRMP messages per candidate

This commit addresses the HRMP part of https://github.com/paritytech/polkadot/issues/1869

* XCM: Introduce HRMP related message types

* HRMP: Data structures and plumbing

* HRMP: Configuration

* HRMP: Data layout

* HRMP: Acceptance & Enactment

* HRMP: Test base logic

* Update adder collator

* HRMP: Runtime API for accessing inbound messages

Also, removing some redundant fully-qualified names.

* HRMP: Add diagnostic logging in acceptance criteria

* HRMP: Additional tests

* Self-review fixes

* save test refactorings for the next time

* Missed a return statement.

* a formatting blip

* Add missing logic for appending HRMP digests

* Remove the channel contents vectors which became empty

* Tighten HRMP channel digests invariants.

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Remove a note about sorting for channel id

* Add missing rustdocs to the configuration

* Clarify and update the invariant for HrmpChannelDigests

* Make the onboarding invariant less sloppy

Namely, introduce `Paras::is_valid_para` (in fact, it already is present
in the implementation) and hook up the invariant to that.

Note that this says "within a session" because I don't want to make it
super strict on the session boundary. The logic on the session boundary
should be extremely careful.

* Make `CandidateCheckContext` use T::BlockNumber for hrmp_watermark

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Make register/deregister parachain dispatchables

This makes `register_parachain` and `deregister_parachain` of
`Registrar` dispatchables.

Besides that it brings back the functionality of the test node to
register a parachain.

* Fix tests

* PR review comments

* Remove old service, 3rd try

i.e.
Revert "Revert "Remove Old Service, 2nd try (#1732)" (#1758)"

This reverts commit 9a0f08bf

.

Closes #1757.

We now have some evidence that the polkadot validator was producing
blocks after all; the reason the blocks_constructed metric was 0 was
that as a new metric it hadn't yet been incorporated into that
branch's codebase. See
https://github.com/paritytech/polkadot/issues/1757#issuecomment-700977602

As this PR is based on a newer `master` branch than the previous one,
that should hopefully no longer be an issue.

* paras trait now has an Origin type

* initial work running a two node local net

* use the right incantations so the nodes produce blocks together

* improve internal documentation

Co-authored-by: Bastian Köcher <[email protected]>

* parachains: use the root origin for mock

* registrar: notify the router module about the cleanup

* add missing call to schedule_para_cleanup

* Initial commit.

* Fix build

* Add comments, remove Event

* Dont expose calls

* Remove TODO and origins

* Fix merge

* Enable or disable parathread registration

* update primitives

* correct parent_head field

* make hrmp field pub

* refactor validation data: runtime

* refactor validation data: messages

* add arguments to full_validation_data runtime API

* port runtime API

* mostly port over candidate validation

* remove some parameters from ValidationParams

* guide: update candidate validation

* update candidate outputs

* update ValidationOutputs in primitives

* port over candidate validation

* add a new test for no-transient behavior

* update util runtime API wrappers

* candidate backing

* fix missing imports

* change some fields of validation data around

* runtime API impl

* update candidate validation

* fix backing tests

* grumbles from review

* fix av-store tests

* fix some more crates

* fix provisioner tests

* fix availability distribution tests

* port collation-generation to new validation data

* fix overseer tests

* Update roadmap/implementers-guide/src/node/utility/candidate-validation.md

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* fix compile warning

* remove tests for weights

* add custom weights

* "Update Substrate"

Co-authored-by: parity-processbot <>

* cargo.toml updates

* session and system

* more

* more

* more

* more

* more

* fix

* compiles

* fix tests

* fix more tests

* fix mock

* fix deleted space

* Update validation/Cargo.toml

Co-authored-by: Bastian Köcher <[email protected]>

* Update Cargo.lock

* update rococo

* remove unused warning

* update add benchmarks

* rename weight file

* forgot a file

* Update chain_spec.rs

* Revert "remove unused warning"

This reverts commit 4227cd0d1525286fb466dccb817564c9b37f8645.

* fix merge

Co-authored-by: Bastian Köcher <[email protected]>

* Cleanup upcoming paras

* Also remove pending genesis

* Not there but here

* Sudo wrapper for paras

* Move to separate module

* Add some docs

* Sort cleanups

* Bump versions

* rename GlobalValidationSchedule to GlobalValidationData

* guide: update candidate descriptor to contain validation data hash

* guide: add note in inclusion module about checking validation data hash

* primitives: update CandidateDescriptor to contain new hash

* fix payload computation

* add helpers for computing validation data to runtime modules

* guide: note routines

* inclusion: check validation data hash and fix local_validation_data bug

* add a case to candidate_checks and improve that test substantially

* bump versions

* address review comments

* add a test for including code upgrade

* bump kusama version

* bump westend & polkadot versions

* create a v1 primitives module

* Improve guide on availability types

* punctuate

* new parachains runtime uses new primitives

* tests of new runtime now use new primitives

* add ErasureChunk to guide

* export erasure chunk from v1 primitives

* subsystem crate uses v1 primitives

* node-primitives uses new v1 primitives

* port overseer to new primitives

* new-proposer uses v1 primitives (no ParachainHost anymore)

* fix no-std compilation for primitives

* service-new uses v1 primitives

* network-bridge uses new primitives

* statement distribution uses v1 primitives

* PoV distribution uses v1 primitives; add PoV::hash fn

* move parachain to v0

* remove inclusion_inherent module and place into v1

* remove everything from primitives crate root

* remove some unused old types from v0 primitives

* point everything else at primitives::v0

* squanch some warns up

* add RuntimeDebug import to no-std as well

* port over statement-table and validation

* fix final errors in validation and node-primitives

* add dummy Ord impl to committed candidate receipt

* guide: update CandidateValidationMessage

* add primitive for validationoutputs

* expand CandidateValidationMessage further

* bikeshed

* add some impls to omitted-validation-data and available-data

* expand CandidateValidationMessage

* make erasure-coding generic over v1/v0

* update usages of erasure-coding

* implement commitments.hash()

* use Arc<Pov> for CandidateValidation

* improve new erasure-coding method names

* fix up candidate backing

* update docs a bit

* fix most tests and add short-circuiting to make_pov_available

* fix remainder of candidate backing tests

* squanching warns

* squanch it up

* some fallout

* overseer fallout

* free from polkadot-test-service hell

* Update decl_storage usage with frame_system as default ident.

* More decl_module usage update.

* 'Update substrate'

Co-authored-by: parity-processbot <>

* add availability bitfield types to primitives

* begin inclusion module

* use GitHub issue link for limitation

* fix some compiler errors

* integrate validators into initializer

* add generic signing context

* make signing-context more generic

* fix issues with inclusion module

* add TODO

* guide: add validators and session index to inclusion

* guide: add session index to change notification

* implement session change logic

* add BackedCandidate type

* guide: refine inclusion pipeline

* guide: rename group_on to group_validators

* guide: add check about collator for parathread

* guide: add last_code_upgrade to paras and use in inclusion

* implement Paras::last_code_upgrade

* implement most checks in process_candidates

* make candidate receipt structs more generic

* make BackedCandidate struct more generic

* use hash param, not block number

* check that candidate is in context of the parent block

* include inclusion module in initializer

* implement enact-candidate

* check that only occupied cores have bits set

* finish implementing bitfield processing

* restructure consistency checks on candidates

* make some more primitives generic

* signature checking logic for backed candidates

* finish implementing process_candidates

* implement collect_pending

* add some trait implementations to primitives

* implement InclusionInherent and squash warnings

* test bitfield signing checks

* rename parachain head to para_head

* fix note_new_head bug in paras

* test bitfield enactment in inclusion

* helpers for candidate checks

* add test for most candidate checks

* add test for backing setting storage

* test session change logic

* remove extraneous type parameter

* remove some allow(unused)s

* extract threshold computation to const fn

* remove some more allow(unused)s

* improve doc

* add debug assertion

* fix primitive test compilation

* tag unanimous variant as unused

* scheduler module skeleton

* update scheduler skeleton to match latest version of guide

* better session change notification

* add mock randomness and fix test compilation

* shuffle validators into groups

* finish implementing session change logic for scheduler

* tweak core assignment type to track retries of parathread

* reframe queued parathread core as offset

* implement initialzation and finalization routines

* implement parathread claim queuing

* implement core_para

* implement the group_validators routine and fix errors

* add a reason for freeing cores

* implement `schedule` function

* add some docs to the scheduled function

* implement `occupied` helper

* implement availability predicate

* fix some warnings

* integrate scheduler into initializer

* integrate scheduler into mock module

* avoid conflict with Substrate's scheduler storage

* add parathreads index to paras module

* implement parathreads map in paras module

* add is_parathread to paras

* test adding parathread claim

* test that you cannot add claims when no parathread cores exist

* check session change parathread queue pruning

* test validator shuffling

* add allow_unused to scheduler items

* add test for scheduling

* add some more tests for scheduling logic

* test core rotation

* check parathread claim pruning after retries

* add bound notes

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>
Co-authored-by: Bernhard Schuster <[email protected]>

* more suggestions from review

* test availability predicate, add box to please compiler

* add changes to guide

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>
Co-authored-by: Bernhard Schuster <[email protected]>

* file structure and initializer skeleton

* ensure session changes happen before initialization

* add a couple tests for initializer flow

* integrate with session handling

* configuration update logic

* configuration methods

* move test mock to its own module

* integrate configuration into initializer

* add note about initialization order

* integrate configuration module into mock

* add some tests for config module

* paras module storage

* implement paras session change operation

* amend past code pruning to fully cover acceptance period

* update guide again

* do pruning of historical validation code

* add weight to initialization

* integrate into mock & leave notes for next session

* clean up un-ended sentence

* alter test to account for double index in past code meta

* port over code-at logic test

* clarify checking for conflicting code upgrades

* add genesis for paras, include in mock, ensure incoming paras are processed

* note on return value of `validation_code_at`

* implement paras routines from implementor's guide

* bring over some existing tests and begin porting

* port over code upgrade tests

* test parachain registration

* test code_at with intermediate block

* fix warnings

* clean up docs and extract to separate struct

* adjust implementor's guide to include replacementtimes

* kill stray println

* rename expected_at to applied_after

* rewrite ParaPastCodeMeta to avoid reversal

* clarify and test interface of validation_code_at

* make FutureCode optional

* rename do_old_code_pruning

* add comment on Option<()> to answer FAQ

* address some more grumbles