* rename GlobalValidationSchedule to GlobalValidationData

* guide: update candidate descriptor to contain validation data hash

* guide: add note in inclusion module about checking validation data hash

* primitives: update CandidateDescriptor to contain new hash

* fix payload computation

* add helpers for computing validation data to runtime modules

* guide: note routines

* inclusion: check validation data hash and fix local_validation_data bug

* add a case to candidate_checks and improve that test substantially

* bump versions

* address review comments

* add a test for including code upgrade

* bump kusama version

* bump westend & polkadot versions

* Bump Substrate

* Bump

* Enable transfers

Also quash any conviction from Referendum Zero; Sudo was always
going to have been removed so lock-voting doesn't make sense in
this case.

* Add test for migration; remove superfluous comment.

* Fixes

* Bump

* Weekly elections

* Remove Sudo

NOTE: To ensure minimal index changes to pre-existing pallet deployments,
this is done with a "swap_remove" style; the previous last pallet
(Purchase), which is hitherto unused, has been shifted into the old index
of Sudo.

* Remove CC1 designation.

* Fixes

* Bump

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* create a README on Runtime APIs

* add ParaId type

* write up runtime APIs

* more preamble

* rename

* rejig runtime APIs

* add occupied_since to `BlockNumber`

* skeleton crate for runtime API subsystem

* improve group_for_core

* improve docs on availability cores runtime API

* guide: freed -> free

* add primitives for runtime APIs

* create a v1 ParachainHost API trait

* guide: make validation code return `Option`al.

* skeleton runtime API helpers

* make parachain-host runtime-generic

* skeleton for most runtime API implementation functions

* guide: add runtime API helper methods

* implement new helpers of the inclusion module

* guide: remove retries check, as it is unneeded

* implement helpers for scheduler module for Runtime APIs

* clean up `validator_groups` implementation

* implement next_rotation_at and last_rotation_at

* guide: more helpers on GroupRotationInfo

* almost finish implementing runtime APIs

* add explicit block parameter to runtime API fns

* guide: generalize number parameter

* guide: add group_responsible to occupied-core

* update primitives due to guide changes

* finishing touches on runtime API implementation; squash warnings

* break out runtime API impl to separate file

* add tests for next_up logic

* test group rotation info

* point to filed TODO

* remove unused TODO [now]

* indentation

* guide: para -> para_id

* rename para field to para_id for core meta

* remove reference to outdated AvailabilityCores type

* add an event in `inclusion` for candidates being included or timing out

* guide: candidate events

* guide: adjust language

* Candidate events type from guide and adjust inclusion event

* implement `candidate_events` runtime API

* fix runtime test compilation

* max -> min

* fix typos

* guide: add `RuntimeAPIRequest::CandidateEvents`

* create a v1 primitives module

* Improve guide on availability types

* punctuate

* new parachains runtime uses new primitives

* tests of new runtime now use new primitives

* add ErasureChunk to guide

* export erasure chunk from v1 primitives

* subsystem crate uses v1 primitives

* node-primitives uses new v1 primitives

* port overseer to new primitives

* new-proposer uses v1 primitives (no ParachainHost anymore)

* fix no-std compilation for primitives

* service-new uses v1 primitives

* network-bridge uses new primitives

* statement distribution uses v1 primitives

* PoV distribution uses v1 primitives; add PoV::hash fn

* move parachain to v0

* remove inclusion_inherent module and place into v1

* remove everything from primitives crate root

* remove some unused old types from v0 primitives

* point everything else at primitives::v0

* squanch some warns up

* add RuntimeDebug import to no-std as well

* port over statement-table and validation

* fix final errors in validation and node-primitives

* add dummy Ord impl to committed candidate receipt

* guide: update CandidateValidationMessage

* add primitive for validationoutputs

* expand CandidateValidationMessage further

* bikeshed

* add some impls to omitted-validation-data and available-data

* expand CandidateValidationMessage

* make erasure-coding generic over v1/v0

* update usages of erasure-coding

* implement commitments.hash()

* use Arc<Pov> for CandidateValidation

* improve new erasure-coding method names

* fix up candidate backing

* update docs a bit

* fix most tests and add short-circuiting to make_pov_available

* fix remainder of candidate backing tests

* squanching warns

* squanch it up

* some fallout

* overseer fallout

* free from polkadot-test-service hell

* Updates guide for CandidateBacking

* Move assignment types to primitives

* Initial implementation.

* More functionality

* use assert_matches

* Changes to report misbehaviors

* Some fixes after a review

* Remove a blank line

* Update guide and some types

* Adds run_job function

* Some comments and refactorings

* Fix review

* Remove warnings

* Use summary in kicking off validation

* Parallelize requests

* Validation provides local and global validation params

* Test issued validity tracking

* Nits from review

* introduce candidatedescriptor type

* add PoVDistribution message type

* loosen bound on PoV Distribution to account for equivocations

* re-export some types from the messages module

* begin PoV Distribution subsystem

* remove redundant index from PoV distribution

* define state machine for pov distribution

* handle overseer signals

* set up control flow

* remove `ValidatorStatement` section

* implement PoV fetching

* implement distribution logic

* add missing `

* implement some network bridge event handlers

* stub for message processing, handle our view change

* control flow for handling messages

* handle `awaiting` message

* handle any incoming PoVs and redistribute

* actually provide a subsystem implementation

* remove set-builder notation

* begin testing PoV distribution

* test that we send awaiting messages only to peers with same view

* ensure we distribute awaited PoVs to peers on view changes

* test that peers can complete fetch and are rewarded

* test some reporting logic

* ensure peer is reported for flooding

* test punishing peers diverging from awaited protocol

* test that we eagerly complete peers' awaited PoVs based on what we receive

* test that we prune the awaited set after receiving

* expand pov-distribution in guide to match a change I made

* remove unneeded import

Closes #1339.

* set up data types and control flow for statement distribution

* add some set-like methods to View

* implement sending to peers

* start fixing equivocation handling

* Add a section to the statement distribution subsystem on equivocations and flood protection

* fix typo and amend wording

* implement flood protection

* have peer knowledge tracker follow when peer first learns about a candidate

* send dependents after circulating

* add another TODO

* trigger send in one more place

* refactors from review

* send new statements to candidate backing

* instantiate active head data with runtime API values

* track our view changes and peer view changes

* apply a benefit to peers who send us statements we want

* remove unneeded TODO

* add some comments and improve Hash implementation

* start tests and fix `note_statement`

* test active_head seconding logic

* test that the per-peer tracking logic works

* test per-peer knowledge tracker

* test that peer view updates lead to messages being sent

* test statement circulation

* address review comments

* have view set methods return references

* network bridge skeleton

* move some primitives around and add debug impls

* protocol registration glue & abstract network interface

* add send_msgs to subsystemctx

* select logic

* transform different events into actions and handle

* implement remaining network bridge state machine

* start test skeleton

* make network methods asynchronous

* extract subsystem out to subsystem crate

* port over overseer to subsystem context trait

* fix minimal example

* fix overseer doc test

* update network-bridge crate

* write a subsystem test-helpers crate

* write a network test helper for network-bridge

* set up (broken) view test

* Revamp network to be more async-friendly and not require Sync

* fix spacing

* fix test compilation

* insert side-channel for actions

* Add some more message types to AllMessages

* introduce a test harness

* impl ProvideInherent for InclusionInherent

* reduce import churn; correct expect message

* move inclusion inherent identifier into primitives

It's not clear precisely why this is desired, but it's a pattern
I've seen in several places, so I'm going this to be on the
safe side. Worst case, we can revert this commit pretty easily.

* bump kusama spec_version to placate CI

* copy sc_basic_authorship::{ProposerFactory, Proposer}

We have from the problem description:

> This Proposer will require an OverseerHandle to make requests via.

That's next on the plate.

* use polkadot custom proposer instead of basic-authorship one

* add some tests

* ensure service compiles and passes tests

* fix typo

* fix service-new compilation

* Subsystem test helpers send messages synchronously

* remove smelly action inspector

* remove superfluous let binding

* fix warnings

* add license header

* empty commit; maybe github will notice the one with changes

* Update node/network/bridge/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* add sanity check to only include valid inherents

* stub: encapsulate block production mechanics instead of copying them

The goal is to end up with something like what's in
validation::block_production::*, which encapsulates
basic block production mechanics. This is a better idea than
just straight-up copying those mechanics.

* partial implementation of propose fn

Doesn't actually compile yet; need to bring in some other
commits to ensure ProvisionerMessage is a thing, and also
figure out how to get the block hash given the current
context.

* fix compilation

* clear a few more compile errors

* finish fn propose

* broken: add timeout to proposal

* add timeout to proposal

* guide: provisioner is responsible for selecting parachain candidates

* implement ProvisionerMessage::RequestInherentData & update fn propose

* impl CreateProposer::init; clean up

* impl std::error::Error for Error

* document error-handling rationale

* cause polkadot-service-new to compile correctly

* Move potentially-blocking call from fn init -> fn propose

This means that we can wrap the delayed call into the same
timeout check used elsewhere.

* document struct Proposer

* extract provisioner data fetch

This satisfies two requirements:

- only applies the timeout to actually fetching the provisioner data,
  not to constructing the block after
- simplifies the problem of injecting default data if we could not
  get the real provisioner data in time.

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

* Switch branch

* Return chain ops parts in new_chain_ops

* Remove where param from new_chain_ops

* Add task manager to new_chain_ops return

* Revert branch switch

* Revert "Revert branch switch"

This reverts commit 7c7900c0

.

* network/test/src/lib: Adjust network worker polling

Companion for https://github.com/paritytech/substrate/pull/6552.

* Fix adder parachain

* Fix collator tests

* Revert branch switch

* Bump everything

- Remove old migration code
- Reduce deposit requried for westend

* Reapply fixes

* Bump locl

* Fix for #6550

* Fix message

Co-authored-by: Ashley Ruglys <[email protected]>
Co-authored-by: Max Inden <[email protected]>

* Downward messages, the front-end.

* Move types around to make them accessible from Parachains

* Fix compilation

* Fix branch

* Make it compile for Cumulus

* Update the branch names

* Add default generic parameter

* Implement `Partialeq`

* Move upward messages into the `ValidationResult`

* Support disabling of the runtime api

* Update branch

* Adds support for handling downward messages

* Implement sending XCMP messages as up/downward messages

* service: update to latest ServiceBuilder changes

* Make it compile

* Initial commit

Forked at: b2c9c149


Parent branch: origin/master

* Update substrate branch to cecton-update-polkadot-substrate

* Update substrate & polkadot to cumulus-branch

* Reset branch

* Update primitives/src/parachain.rs

Co-authored-by: Robert Habermeier <[email protected]>

* Update runtime/common/src/parachains.rs

Co-authored-by: Robert Habermeier <[email protected]>

* Update runtime/common/src/parachains.rs

Co-authored-by: Robert Habermeier <[email protected]>

* Minor fixes

* Fix wasm build

Co-authored-by: Gav Wood <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: Cecile Tonglet <[email protected]>
Co-authored-by: Robert Habermeier <[email protected]>

* impl ProvideInherent for InclusionInherent

* reduce import churn; correct expect message

* move inclusion inherent identifier into primitives

It's not clear precisely why this is desired, but it's a pattern
I've seen in several places, so I'm going this to be on the
safe side. Worst case, we can revert this commit pretty easily.

* bump kusama spec_version to placate CI

* add license header

* empty commit; maybe github will notice the one with changes

* add sanity check to only include valid inherents

* Companion for #6503

* Remove old filtering

* Bump tx version

* Bump version

* Bump

* Update staking reward curve for pre-parachain situation.

* Bump runtime.

* Bump versions

* add signed wrapper, typedef SignedStatement

* typedef SignedAvailabilityBitfield

* implement Signed wrapper

This is strictly an addition as of this commit; nothing is yet
changed in existing behavior.

* inline getters, remove review comment

* move EncodeAs, Signed from node::primitives to primitives::parachain

* Refactor SignedAvailabilityBitfield to use Signed

* don't double-encode real payload

This isn't an ideal solution, because it depends on the
implementation details of how SCALE encodes tuples, but OTOH
that behavior seems unlikely to change anytime soon.

* fix build errors

* cause the runtime to build properly with the new changes

Not sure why cargo check didn't catch this earlier; oh well.

* fix runtime tests and separate SignedStatement from SignedFullStatement

* better explain why CompactStatement exists

Co-authored-by: Robert Habermeier <[email protected]>

Co-authored-by: Robert Habermeier <[email protected]>

* add availability bitfield types to primitives

* begin inclusion module

* use GitHub issue link for limitation

* fix some compiler errors

* integrate validators into initializer

* add generic signing context

* make signing-context more generic

* fix issues with inclusion module

* add TODO

* guide: add validators and session index to inclusion

* guide: add session index to change notification

* implement session change logic

* add BackedCandidate type

* guide: refine inclusion pipeline

* guide: rename group_on to group_validators

* guide: add check about collator for parathread

* guide: add last_code_upgrade to paras and use in inclusion

* implement Paras::last_code_upgrade

* implement most checks in process_candidates

* make candidate receipt structs more generic

* make BackedCandidate struct more generic

* use hash param, not block number

* check that candidate is in context of the parent block

* include inclusion module in initializer

* implement enact-candidate

* check that only occupied cores have bits set

* finish implementing bitfield processing

* restructure consistency checks on candidates

* make some more primitives generic

* signature checking logic for backed candidates

* finish implementing process_candidates

* implement collect_pending

* add some trait implementations to primitives

* implement InclusionInherent and squash warnings

* test bitfield signing checks

* rename parachain head to para_head

* fix note_new_head bug in paras

* test bitfield enactment in inclusion

* helpers for candidate checks

* add test for most candidate checks

* add test for backing setting storage

* test session change logic

* remove extraneous type parameter

* remove some allow(unused)s

* extract threshold computation to const fn

* remove some more allow(unused)s

* improve doc

* add debug assertion

* fix primitive test compilation

* tag unanimous variant as unused

* introduce polkadot-node-primitives

* guide: change statement distribution message types

* guide: remove variant from `CandidateSelectionMessage`

* add a few more message types

* add TODOs

* Almost all messages

* NewBackedCandidate notification

* Formatting

* Use AttestedCandidate as BackedCandidate

* Update node/primitives/src/lib.rs

Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Fix the tests

* Bring in types from #1242

* Adds network bridge messages

* More message types from doc

* use fn pointer type

* Fixes from the review

* Add missing Runtime subsystem message

* rename to CandidateValidationMessage and fix tests

Co-authored-by: Fedor Sakharov <[email protected]>
Co-authored-by: Peter Goodspeed-Niklaus <[email protected]>

* Rename old labels.

* Bump runtimes

* Bump versions

* Bump lock

* Bump versions, tweak deposit costs.

* Version

* Lock

* Make test work ok when numbers are not round.

* Bump Substrate

* Lock

* Bump Substrate again.

* update kusama runtmie for ensure origin

* update polkadot runtime for ensure origin

* fix imports

* root only available for runtime benchmarks

* Fix lock file

* Bump Substrate

* Update lock

* Remove questionable feature-gating.

Co-authored-by: Shawn Tabrizi <[email protected]>

* Bump versions.

* Update lock

* Fix for new Substrate

* Fix

* Remove Utility constraint in NonTransfer

* Bump

* Fixes

* Allow limited utilities in proxy filters.

* Fix

* Add SudoBalances proxy restrictions

* Introduce generalised proxies to polkadot

* Introduce proxy to westend

* Add proxy to Kusama.

* Fix

* Identity

* Bump spec

* Update runtime/polkadot/src/lib.rs

Co-authored-by: kaichao <[email protected]>

* Fix runtime

* Bump runtimes

Co-authored-by: kaichao <[email protected]>

* Bump

* Bump

* Bump

* Test CC1 spec

* Adaptive default chainspec

* Strip -pre

* Update to next CC1 iteration

* Raw chainspec

* Update claim statements

* Update URLs

* Minor tweak to make enum agree with URL

* final html multihashes

* New hashes

* Version

* updated SAFT statement hash

* Update runtimes; this relies on substrate #6131

* Bump

* Share transaction filterer and fix Kusama

* Warning

* Gah!

* Tidy

* Westend fixes

* Westend fix

* Tweak constants

* Implement TCF stuff

* Fix

* Warning

* 15 minute lookahead

* 15 minute lookahead in Polkadot

* update max transactions

* Enable utility, disable vested_transfer

* Update runtime/common/src/lib.rs

Co-authored-by: Tomasz Drwięga <[email protected]>

* Filter calls for validate unsigned also

* Deduplicate

* Fix

* Introduce Polkadot (mainnet) chainspec

* Fix naming.

* Enable indices and fix comment.

* Fix compilation

* Enable indices and fix comment.

* polkadot: babe: enable secondary VRF slots

* Test JSON

* Allow set_heads

* Fix

* Raw chain spec added

Co-authored-by: keorn <[email protected]>
Co-authored-by: NikVolf <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Introduce mandatory statement signing into claims

* Introduce SignedExtension

* Tests passing

* Bump runtime version

* Bump version, fix test

* Test for validate

* Another couple of tests

* Enable PrevalidateAttests on Polkadot

* Enable PrevalidateAttests on Polkadot

* Fix build

* Fixes

* More fixes

* Fix bench tests

* Fix & test Preclaim clobbering.

* Fix for errant claim logic

* Add test

* Update tests, always use Vec as input

* mint_claim can add signature, some_benchmarks

* Add claim with statement test

* finish benchmarks

* put the correct number of claims with benchmarks

* fix compiler warning

* Update weights

* Weight comments for validation

Co-authored-by: Shawn Tabrizi <[email protected]>

* grandpa: report equivocations on all runtimes

* runtime: fix test-runtime compilation

* runtime: enable historical session manager on all runtimes

* runtime: fix indentation

* runtime: use strong key types in KeyOwnerProofSystem definitions

* update substrate

* bump spec_version of runtimes

* Bump Westend runtime version

* Bump Substrate

* Prep for release of 0.7.32

* Update lock