* migrate subsystem expansion PR from w3f/research

As the file at this new location included changes not present in
https://github.com/w3f/research/pull/87, this is effectively a
rebase, applied manually. I believe that I have successfully retained
all of, and only, the intended changes.

* add section on collators

* note why the overseer is the terminal message point for the validation subsystem

* add detail about how the validitiy system gets PoV blocks

* rename to Validity Subsystems Overview

* get rid of changes to the implementor's guide

I think it makes most sense to track and review the subsystems
overview in a different PR than the one which adds content to
the implementor's guide.

* punt misbehavior arbitration details to the future

* empty commit to rerun CI

* Add block event forwarding into the Overseer

* Add a doc comment

* Fix a doc comment

* Adds start and stop work heartbeats.

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Fix code after suggested changes

* Finalizing stops work on earlier lower blocks.

* Fix func parameter and flaky test

Co-authored-by: Bastian Köcher <[email protected]>

* note pruning of orphan blocks in the overseer description

* remove note about finalization events being maybe not needed

* Make it work

* Upsub

* file structure and initializer skeleton

* ensure session changes happen before initialization

* add a couple tests for initializer flow

* integrate with session handling

* configuration update logic

* configuration methods

* move test mock to its own module

* integrate configuration into initializer

* add note about initialization order

* integrate configuration module into mock

* add some tests for config module

* paras module storage

* implement paras session change operation

* amend past code pruning to fully cover acceptance period

* update guide again

* do pruning of historical validation code

* add weight to initialization

* integrate into mock & leave notes for next session

* clean up un-ended sentence

* alter test to account for double index in past code meta

* port over code-at logic test

* clarify checking for conflicting code upgrades

* add genesis for paras, include in mock, ensure incoming paras are processed

* note on return value of `validation_code_at`

* implement paras routines from implementor's guide

* bring over some existing tests and begin porting

* port over code upgrade tests

* test parachain registration

* test code_at with intermediate block

* fix warnings

* clean up docs and extract to separate struct

* adjust implementor's guide to include replacementtimes

* kill stray println

* rename expected_at to applied_after

* rewrite ParaPastCodeMeta to avoid reversal

* clarify and test interface of validation_code_at

* make FutureCode optional

* rename do_old_code_pruning

* add comment on Option<()> to answer FAQ

* address some more grumbles

* Initial commit

* Licenses, spaces, docs

* Add a spawner

* Watch spawned subsystems with a FuturesUnordered

* Move the types around a bit

* Suggested fixes by Max

* Add a handler to talk to the Overseer

* FromOverseer and ToOverseer msgs and stopping

* Docs and return errors

* Dont broadcast, have add a from field to messages

* Allow communication between subsystems and outside world

* A message with a oneshot to send result example

* Remove leftover can_recv_msg

* Remove from field from messages

* Dont be generic over stuff

* Gather messages with StreamUnordered

* Fix comments and formatting

* More docs fixes and an example

* Apply suggestions from code review

Co-authored-by: Robert Habermeier <[email protected]>

* Fixes from review

Move function from impl block.
Do not panic but resolve with errors if spawner fails or subsystem
resolves.

* Dropping a handler results in a flaky test

Co-authored-by: Robert Habermeier <[email protected]>

* remove old runtime requirements section

* reorganize guide and improve ToC

* Identity

* Bump spec

* Update runtime/polkadot/src/lib.rs

Co-authored-by: kaichao <[email protected]>

* Fix runtime

* Bump runtimes

Co-authored-by: kaichao <[email protected]>

This pr ensures that the table router stays alive for the lifetime of
the validation instance. This is required to ensure that the node
responds onto gossip messages for the particular relay chain round.
Before, the table router was only kept alive for relay chain nodes that
were assigned to a Parachain, however the lifetime was also relative
short. This lead to bugs where a relay chain node did not include
PoVBlock's, because it did not receive them (rejected them on receive,
because it was not listening on the particular round).

* Revert async-std to 1.5.0

* Funny, some part of the Cargo.lock didn't get commited

* Use Substrate block builder.

* Clean up metrics.

* Lock.

* Lock.

* Switch to newest basic authorship interface

* Update Substrate reference and polkadot spec_version

* Let's improve

Co-authored-by: Bastian Köcher <[email protected]>

* remove redundant resize instruction from guide

* amend past code pruning to fully cover acceptance period

* update guide again

* note on return value of `validation_code_at`

* rename execution cores to availability cores

* clarify terminology

* add ReplacementTimes to the guide

* fully decouple inclusion and scheduler

* clarify parathread scheduling substantially

* clarify validator group selection

This lets offline signers know that the transaction format has changed,
due to the addition of `transaction_version` to the signed payload.

* Bump

* Bump

* Bump

* Make sure that vested claims are not transferable

We need to deposit the claimed balance to the destination account before
setting the locks through the vesting module. Otherwise we loose the
locks and the vested claim is directly transferable.

* Add comment

* Enable missing feature for webbrowser check

* Remove unneeded error variant

* Increment `spec_version`'s

* Fix tests

* companion for https://github.com/paritytech/substrate/pull/6132

* Fix more tests

* upsub

* Bump

* Update runtime/common/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* remove cleanup

* Update runtime/westend/src/lib.rs

* Apply suggestions from code review

* Fix build

Co-authored-by: Bastian Köcher <[email protected]>

* Add tests for Balance module weights

* Add tests for some of the Staking functions

* Add tests for system Pallet weights

* Add weight tests for Session pallet

* Add tests for Democracy Pallet weights

* Add tests for Phragment weights

* Add tests for Treasury weights

* Use consistent naming for tests

* Split tests into separate weight and fee suites

* Move weight constant checks into one test

* Use formulas instead of hardcoded weights

* Slim down number of tests

Mainly removing ones that are either root calls that aren't "important",
or others which will likely change if another test (which we kept) also
changes.

* Update timestamp set() weight

* Update Staking Pallet weights

* Update Democracy Pallet weights

* Update copyright year

Co-authored-by: Kian Paimani <[email protected]>

Co-authored-by: Kian Paimani <[email protected]>

* Test CC1 spec

* Adaptive default chainspec

* Strip -pre

* Update to next CC1 iteration

* Raw chainspec

* Test CC1 spec

* Adaptive default chainspec

* Include polkadot spec in release notes, retitle releases

* add CC1 to release title

* Update claim statements

* Update URLs

* Minor tweak to make enum agree with URL

* final html multihashes

* New hashes

* Version

* updated SAFT statement hash

* Update runtimes; this relies on substrate #6131

* Bump

* Share transaction filterer and fix Kusama

* Warning

* Gah!

* Tidy

* Westend fixes

* Westend fix

* Tweak constants

* Implement TCF stuff

* Fix

* Warning

* 15 minute lookahead

* 15 minute lookahead in Polkadot

* update max transactions

* Enable utility, disable vested_transfer

* Update runtime/common/src/lib.rs

Co-authored-by: Tomasz Drwięga <[email protected]>

* Filter calls for validate unsigned also

* Deduplicate

* Fix

* Introduce Polkadot (mainnet) chainspec

* Fix naming.

* Enable indices and fix comment.

* Fix compilation

* Enable indices and fix comment.

* polkadot: babe: enable secondary VRF slots

* Test JSON

* Allow set_heads

* Fix

* Raw chain spec added

Co-authored-by: keorn <[email protected]>
Co-authored-by: NikVolf <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: André Silva <[email protected]>

* include implementor's guide in this repo

* most updated guide

* update readme for current networks

* remove networks docs (they were copy/paste from readme)

* remove trailing whitespace

* add polkadot wiki

* remove hello world name

* remove authors/revision

* More benchmarks

* Apply suggestions from code review

* update cargo

Co-authored-by: Gavin Wood <[email protected]>

* Fixed point methods renames.

* Bump

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Gav Wood <[email protected]>

* Update impls.rs

* companion for substrate#6069

* update to latest changes

* Update Cargo.lock

* Bump

Co-authored-by: André Silva <[email protected]>
Co-authored-by: Gav Wood <[email protected]>

Co-authored-by: dud1337 <[email protected]>

* add system to benchmarks

* Update cargo files

* Add new trait to offences

* companion for substrate#6069

* update to latest changes

* Update Cargo.lock

Co-authored-by: André Silva <[email protected]>
Co-authored-by: Gav Wood <[email protected]>

* make council members able to note preimage using operation extirnsic

* Bump

Co-authored-by: Gav Wood <[email protected]>