* Companion for #7997

https://github.com/paritytech/substrate/pull/7997

* rename slot_number to slot

* rename SlotNumber type in overseer docs

* "Update Substrate"

Co-authored-by: André Silva <[email protected]>
Co-authored-by: parity-processbot <>

* bump tx version for westend and test-runtime

& bump substrate

* Update runtime/test-runtime/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Use extensible multiaddress

* copy pasta

* Update pallet_staking.rs

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Adapt service creation to new substrate API

* Fix test

* WIP

* Revert "WIP"

This reverts commit 816a3633e91abc943b12b2bfa77ce98b959e78b2.

* WIP

* Adapt to current code

* Fix tests

* Yet another fix

* CLEANUP

* WIP

* WIP

* WIP

* Adapt code to changes on substrate

* Adapt code

* Introduce kick.

* Fixes

* WIP

* WIP

* WIP

* WIP

* Bump

* Update sp-io

* WIP

Co-authored-by: Gav Wood <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>

* collation-generation: use persisted validation data

* node: remote FullValidationData API

* runtime: remove FullValidationData API

* backing tests: use persisted validation data

* FullCandidateReceipt: use persisted validation data

This is not a big change since this type is not used anywhere

* Remove ValidationData and TransientValidationData

Also update the guide

* Companion for Substrate#7692

* "Update Substrate"

Co-authored-by: parity-processbot <>

* runtime: add Babe::next_epoch runtime api method

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Companion for #7810

* Added missing trait items for tests

* Add another missing trait item

* fixup

* "Update Substrate"

Co-authored-by: parity-processbot <>

* point to branch

* update babe API implementations

* build

* Revert "point to branch"

This reverts commit 669fde1e58cac1eef1847d5e6b39520fa4953f33.

* update Cargo.lock

* bump

* guide: add candidate information to OccupiedCore

* add descriptor and hash to occupied core type

* guide: add candidate hash to inclusion

* runtime: return candidate info in core state

* bitfield signing: stop querying runtime as much

* minimize going to runtime in availability distribution

* fix availability distribution tests

* guide: remove para ID from Occupied core

* get all crates compiling

* guide: merge backing and approval keys

* bump substrate master & update primitives

* use new SessionInfo struct in session_info

* session keys upgrade for Polkadot

* kusama & westend runtimes

* bump westend, kusama, and polkadot versions

* add session key to rococo & test-runtime

* update prepare-test-net to latest subkey

* update chain specs to support new para_assignment session key

* get cargo.lock from master

* formatting

* update kill_storage based on substrate master

* fix test-service

* assgn -> asgn

* use session info module for assignment session key

* Update weights for polkadot

* add Kusama weights

* add westend weights

* remove claims weight files

claims still uses in-line weights which is fine cause it is a polkadot specific pallet

Co-authored-by: Parity Benchmarking Bot <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Change branch.

* Update runtime.

* Revert "Change branch."

This reverts commit 841c59f3398136c27cc235a29d7d459e8a4c8ce0.

* Update substrate.

* Fix tests.

* Fix compilation.

* Fix frame system imports.

* Fix usages of system

* Fix stuff.

* Fix compilation.

* Fixes.

* Fix block_weight usage.

* Bump substrate.

* rename Trait -> Config

* revert diener changes

* rename HostConfig to ActiveConfig as more meaningful

* fix merge

* "Update Substrate"

* cargo update -p sp-io

Co-authored-by: parity-processbot <>

* guide: move erasure-root to candidate descriptor

* primitives: move erasure root to descriptor

* guide: unify candidate commitments and validation outputs

* primitives: unify validation outputs and candidate commitments

* parachains-runtime: fix fallout

* runtimes: fix fallout

* collation generation: fix fallout

* fix stray reference in primitives

* fix fallout in node-primitives

* fix remaining fallout in collation generation

* fix fallout in candidate validation

* fix fallout in runtime API subsystem

* fix fallout in subsystem messages

* fix fallout in candidate backing

* fix fallout in availability distribution

* don't clone

* clone

Co-authored-by: Sergei Shulepov <[email protected]>

* guide: fix formatting for SessionInfo module

* primitives: SessionInfo type

* punt on approval keys

* ah, revert the type alias

* session info runtime module skeleton

* update the guide

* runtime/configuration: sync with the guide

* runtime/configuration: setters for newly added fields

* runtime/configuration: set codec indexes

* runtime/configuration: update test

* primitives: fix SessionInfo definition

* runtime/session_info: initial impl

* runtime/session_info: use initializer for session handling (wip)

* runtime/session_info: mock authority discovery trait

* guide: update the initializer's order

* runtime/session_info: tests skeleton

* runtime/session_info: store n_delay_tranches in Configuration

* runtime/session_info: punt on approval keys

* runtime/session_info: add some basic tests

* Update primitives/src/v1.rs

* small fixes

* remove codec index annotation on structs

* fix off-by-one error

* validator_discovery: accept a session index

* runtime: replace validator_discovery api with session_info

* Update runtime/parachains/src/session_info.rs

Co-authored-by: Sergei Shulepov <[email protected]>

* runtime/session_info: add a comment about missing entries

* runtime/session_info: define the keys

* util: expose connect_to_past_session_validators

* util: allow session_info requests for jobs

* runtime-api: add mock test for session_info

* collator-protocol: add session_index to test state

* util: fix error message for runtime error

* fix compilation

* fix tests after merge with master

Co-authored-by: Sergei Shulepov <[email protected]>

* update tiny-keccak to 0.2

* update deps except bitvec and shared_memory

* fix some warning after futures upgrade

* remove useless package rename caused by bug in cargo-upgrade

* revert parity-util-mem *

* remove unused import

* cargo update

* remove all renames on parity-scale-codec

* remove the leftovers

* remove unused dep

* HRMP: Update the impl guide

* HRMP: Incorporate the channel notifications into the guide

* HRMP: Renaming in the impl guide

* HRMP: Constrain the maximum number of HRMP messages per candidate

This commit addresses the HRMP part of https://github.com/paritytech/polkadot/issues/1869

* XCM: Introduce HRMP related message types

* HRMP: Data structures and plumbing

* HRMP: Configuration

* HRMP: Data layout

* HRMP: Acceptance & Enactment

* HRMP: Test base logic

* Update adder collator

* HRMP: Runtime API for accessing inbound messages

Also, removing some redundant fully-qualified names.

* HRMP: Add diagnostic logging in acceptance criteria

* HRMP: Additional tests

* Self-review fixes

* save test refactorings for the next time

* Missed a return statement.

* a formatting blip

* Add missing logic for appending HRMP digests

* Remove the channel contents vectors which became empty

* Tighten HRMP channel digests invariants.

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Remove a note about sorting for channel id

* Add missing rustdocs to the configuration

* Clarify and update the invariant for HrmpChannelDigests

* Make the onboarding invariant less sloppy

Namely, introduce `Paras::is_valid_para` (in fact, it already is present
in the implementation) and hook up the invariant to that.

Note that this says "within a session" because I don't want to make it
super strict on the session boundary. The logic on the session boundary
should be extremely careful.

* Make `CandidateCheckContext` use T::BlockNumber for hrmp_watermark

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Update proxy filters

* add session

* new polkadot weights

* kusama weights

* update for westend

* cargo update -p sp-io

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_session --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_session --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_session --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* fix: ensure candidate validation gets code based on occupied core assumption

* guide: runtime API for historical validation code

* add historical runtime API

* integrate into runtime API subsystem

* remove blocked TODO

* fix service build: enable notifications protocol only under real overseer

* Update node/subsystem/src/messages.rs

Co-authored-by: Sergei Shulepov <[email protected]>

* fix compilation

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Sergei Shulepov <[email protected]>

* adjustments for substrate PR

https://github.com/paritytech/substrate/pull/6912

* Update runtime/kusama/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <[email protected]>

* update substrate

Co-authored-by: Guillaume Thiolliere <[email protected]>

* DMP: data structures and plumbing

* DMP: Implement DMP logic in the router module

DMP: Integrate DMP parts into the inclusion module

* DMP: Introduce the max size limit for the size of a downward message

* DMP: Runtime API for accessing inbound messages

* OCD

Small clean ups

* DMP: fix the naming of the error

* DMP: add caution about a non-existent recipient

Companion for #7188: Implement `batch_all` and update Utillity Pallet for Weight/Fee Refunds (#1775)

* add missing method

* "Update Substrate"

* bump spec

* cargo update -p sp-io

Co-authored-by: parity-processbot <>
Co-authored-by: Shawn Tabrizi <[email protected]>

* annoying whitespaces

* update guide

Add `CheckValidationOutputs` runtime api and also change the
candidate-validation stuff

* promote ValidationOutputs to global primitives

i.e. move it from node specific primitives to global v1 primitives. This
will be needed when we share it later in the runtime inclusion module

* refactor acceptance checks in the inclusion module

factor out the common code to share it during the block inclusion and
for the forthcoming CheckValidationOutputs runtime api.

Also note that the acceptance criteria was updated to incorporate checks
that exist now in candidate-validation

* plumb the runtime api outside

* extract validation_data from ValidationOutputs

* use runtime-api to check validation outputs

apart from that refactor, update docs and tidy a bit

* Update the maxium code size

This is to fix a test that performs an upgrade.

* Update constant weight signatures

* Keep timestamp constant time

* constant as_derivative

* cargo update -p sp-io

Co-authored-by: parity-processbot <>

* runtime: remove finality-tracker

* runtime: bump spec_version

* update substrate

* make polkadot work again,

* Fix build

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Remove old service, 3rd try

i.e.
Revert "Revert "Remove Old Service, 2nd try (#1732)" (#1758)"

This reverts commit 9a0f08bf

.

Closes #1757.

We now have some evidence that the polkadot validator was producing
blocks after all; the reason the blocks_constructed metric was 0 was
that as a new metric it hadn't yet been incorporated into that
branch's codebase. See
https://github.com/paritytech/polkadot/issues/1757#issuecomment-700977602

As this PR is based on a newer `master` branch than the previous one,
that should hopefully no longer be an issue.

* paras trait now has an Origin type

* initial work running a two node local net

* use the right incantations so the nodes produce blocks together

* improve internal documentation

Co-authored-by: Bastian Köcher <[email protected]>

* stupid, but it compiles

* redo

* cleanup

* add ValidatorDiscovery to msgs

* sketch network bridge code

* ConnectToAuthorities instead of validators

* more stuff

* cleanup

* more stuff

* complete ConnectToAuthoritiesState

* Update node/network/bridge/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Collator protocol subsystem (#1659)

* WIP

* The initial implementation of the collator side.

* Improve comments

* Multiple collation requests

* Add more tests and comments to validator side

* Add comments, remove dead code

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Fix build after suggested changes

* Also connect to the next validator group

* Remove a Future impl and move TimeoutExt to util

* Minor nits

* Fix build

* Change FetchCollations back to FetchCollation

* Try this

* Final fixes

* Fix build

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* handle multiple in-flight connection requests

* handle cancelled requests

* Update node/core/runtime-api/src/lib.rs

Co-authored-by: Bernhard Schuster <[email protected]>

* redo it again

* more stuff

* redo it again

* update comments

* workaround Future is not Send

* fix trailing spaces

* clarify comments

* bridge: fix compilation in tests

* update more comments

* small fixes

* port collator protocol to new validator discovery api

* collator tests compile

* collator tests pass

* do not revoke a request when the stream receiver is closed

* make revoking opt-in

* fix is_fulfilled

* handle request revokation in collator

* tests

* wait for validator connections asyncronously

* fix compilation

* relabel my todos

* apply Fedor's patch

* resolve reconnection TODO

* resolve revoking TODO

* resolve channel capacity TODO

* resolve peer cloning TODO

* resolve peer disconnected TODO

* resolve PeerSet TODO

* wip tests

* more tests

* resolve Arc TODO

* rename pending to non_revoked

* one more test

* extract utility function into util crate

* fix compilation in tests

* Apply suggestions from code review

Co-authored-by: Fedor Sakharov <[email protected]>

* revert pin_project removal

* fix while let loop

* Revert "revert pin_project removal"

This reverts commit ae7f529d8de982ef66c3007dd1ff74c6ddce80d2.

* fix compilation

* Update node/subsystem/src/messages.rs

* docs on pub items

* guide updates

* remove a TODO

* small guide update

* fix a typo

* link to the issue

* validator discovery: on_request docs

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>
Co-authored-by: Fedor Sakharov <[email protected]>
Co-authored-by: Bernhard Schuster <[email protected]>

* Companion for substrate/pull/7215

* More fixes

* Fix build

* fix

* Fix again with normal

* Fix build

* Latest changes

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Remove stale upgrades

* add back origin migration

* remove unused

* Update benchmark list

* copy weights from substrate

* Run benchmarks on Polkadot

* Update kusama benchmark list

* update westend benchmark list

* Run benchmarks on Kusama

* update payout staking weights

* cargo update -p sp-io

* remove babe and grandpa from benchmark list

* remove babe and grandpa weights

* re-run staking weights

* update staking weights

* copy weights from Polkadot into Westend

* Update Cargo.lock

This is required to ensure that on chain isn't seen as compatible to
current master.

This reverts commit 96f26155.

* Restore "Remove service, migrate all to service-new (#1630)"

i.e.
Revert "Revert "Remove service, migrate all to service-new (#1630)" (#1731)"

This reverts commit b4457f55

.

This allows us to get the changeset from #1630 into a new branch
which can be merged sometime in the future after appropriate burnin
tests have completed.

* remove ',)' from codebase outside of macros

* restore bdfl-preferred formatting

* attempt to improve destructuring formatting

* rename polkadot-service-new -> polkadot-service

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* remove unused import

* Update runtime/rococo-v1/README.md

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* use construct_runtime index

* fix

* "Update Substrate"

Co-authored-by: parity-processbot <>