Currently validation data is shared by using a well known key between
the parachain system pallet and the validate block implementation. This
pr changes this by passing the parachain system directly to the validate
block implementation to make use of it. Besides that, we also store the
validation params in some thread local variable to make it inspectable
by parachain system. This moves the validation of validation data and
validation params to the parachain system pallet directly, instead of
having this hidden inside the validate block implementation.

Fixes: https://github.com/paritytech/cumulus/issues/217

* Move consensus to consensus-common crate

* Move the parachain consensus out of the collator

* Add first relay chain consensus stuff

* Remove some warnings

* Fix more stuff

* Fix collator test

* Change `ParachainConsensus` to take a mutable self

* Make everything compile

* Feedback

* Update Substrate & Polkadot

* Update chainspecs

* Update again to fix test

* Move pallets to pallets folder and rename them

* Move genesis file to service

* Rename primitives to primitives-core

* Delete cumulus-runtime

* Move stuff to client folder and rename

* Update polkadot

* Migrate all uses of MQC heads to merkle proofs

* Mass rename `relay_parent_storage_root`

* Restore parachain-system tests

* Update polkadot and libp2p swarm for testing

* Collapse match into an if let

Co-authored-by: Bastian Köcher <[email protected]>

* Fix compilation error in test-service

Co-authored-by: Bastian Köcher <[email protected]>

* Fix HRMP Location

* Add call to xcm handler

* Complete telemetry for parachain & relaychain

* Update Substrate & Polkadot

* rename parachain-{upgrade -> system}

* Merge message-broker into parachain-system

* Remove message-broker and clean up

* Update docs

* Test upward messages sending

And also update the relay-sproof-builder so that it allows to set the
relay dispatch queue size for the given parachain.

* Test horizontal message sending

* Remove old inherent definitions

* Update polkadot

* Extend cumulus primitives with some relay chain exports

Follow https://github.com/paritytech/polkadot/pull/2194

 to see the
polkadot PR

* collator: collect the state proof

This commit changes cumulus-collator so that it takes the relay chain
state at the relay parent and creates a storage proof that contains all
the required data for PVF.

* parachain-upgrade: use the proofs instead

This change is needed to make cumulus logic to not longer depend on the
transient validation data. As part of this change, in order to preserve
the current behavior `code_upgrade_allowed` now is computed on the
parachain side, rather than provided by polkadot.

Turned out that this requires to know the self parachain id so it was
added where needed.

* message-broker: use relay state to track limits

this should make sending messages safe from accidentally running over
the relay chain limits that were previously unknown.

* Update polkadot

So that `relay_storage_root` is available through `ValidationParams`

* Check `relay_storage_root` matches expected

Check that `relay_storage_root` submitted by the collator matches the
one that we receive in `validate_block` through `ValidationParams`

* Add a missing check for `dmq_mqc_head` while we are at it

* Update polkadot

* Fix tests that use the relay storage root

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Update message-broker/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Remove unneeded (&_)

* Fix unwraps

* Polish basti's suggestion

* Fix merge

* Bring back the System::can_set_code check

Co-authored-by: Bastian Köcher <[email protected]>

* Update yeah

* Update spec_version

* Wait for block import in parachain consensus

There was a bug in the parachain consensus that when importing a relay
chain block that sets a new best parachain block, but the required
parachain block was not yet imported. This pr fixes this by waiting for
the block to be imported.

* Finish docs

* Update to latest Substrate & Polkadot

* Update the chain specs

* Update polkadot properly...

* Update chainspecs again...

* Again...

* upgrade to latest polkadot

* ss58 in tests

* format

* removed unused import

* Update

* Update the chain specs

* Update to latest master & master

* add current tick bootnodes

* Update again

* Add track bootnodes

* add trick and track bootnodes

* Update the chain specs

* Update

Co-authored-by: Erin Grasmick <[email protected]>

* The problem in the nutshell

* Make it compile

* make tests pass

* Update the parachain-upgrade module

* Fix collator tests

Co-authored-by: Bastian Köcher <[email protected]>

* initial mock

* integrate xcm-handler into runtime

* expose xcm send error

* oops

* better comment

* HRMP message ingestion

* Plumb hrmp_watermark to build_collation

* Plumb hrmp_watermark to ValidationResult

* Plumb hrmp outbound messages

* Implement message-broker part of HRMP

* Kill UPWARD_MESSAGES as well

Otherwise, they will get resent each block

* Add sudo versions for easier testing

* Remove the xcmp module

Not useful for the moment

* Doc for HRMP message handler

* Estimate the weight upper bound for on_finalize

* Remove a redundant type annotation

* fix spelling of a method

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Deabbreviate dmp and hrmp in the message ingestion type

* Don't use binary_search since it's broken by a following rotate

Instead use the linear search. We can afford linear search here since
due to limited scalability of HRMP we can only have at most a couple of
dozens of channels.

* Fix the watermark

Co-authored-by: Bastian Köcher <[email protected]>

* Upgdate to latest polkadot & substrate

* Fix code formatting (cargo fmt)

* Fix unit tests

* update cumulus to latest polkadot

* s/Trait/Config

To be more consistent with the new naming.

* Update Cargo.lock

* fix network tests

* Start with something

* Whatever

* Update

* MOARE

* Make cumulus-network compile and tests work

* Update more and fixes

* More stuff

* More fixes

* Make collator build

* Make test almost work

* Remove contracts runtime

* More test work

* Make service compile

* Fix test-service

* Fix test client

* More fixes

* Fix collator test

* Fix network tests (again)

* Make everything compile, finally

* Fix tests

* Write test that should fail

* Add `WaitOnRelayChainBlock`

* Update git versions

* Make it all work

* Update logging

* Switch to provided method for pushing an extrinsic

* Try to debug CI

* Aaaa

* Only use Debug

* Updates

* Use native execution to hopefully make CI happy...

This is mostly a copy of the predating version with exception of some
renaming and alterations (e.g. the message handler takes an inbound
downward message by value, not by reference).

* Add --raw flag to export-genesis-state

* Switch export-genesis-wasm to hex by default

Also add --raw flag. This makes it similar to `export-genesis-state`.

* Switch to polkadot master branch

* Fix compilation

* Start with something

* Whatever

* Update

* MOARE

* Make cumulus-network compile and tests work

* Update more and fixes

* More stuff

* More fixes

* Make collator build

* Make test almost work

* Remove contracts runtime

* More test work

* Make service compile

* Fix test-service

* Fix test client

* More fixes

* Fix collator test

* Fix network tests (again)

* Make everything compile, finally

* Fix tests

* Update to latest masters

* Remove ignore

* Switch to different branch in polkadot for now

* Update reference

* Make it compile with latest changes

* Update collator/src/lib.rs

Co-authored-by: Robert Habermeier <[email protected]>

* Update to latest upstream

* Update to latest master

* Fix test

Co-authored-by: Robert Habermeier <[email protected]>

* Initial commit

Forked at: 56753b77
Parent branch: origin/master

* Copy runtime module from rococo

Forked at: 56753b77
Parent branch: origin/master

* Also copy dependencies pallets and primitives

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* test-service

* Move integration test

* CLEANUP

Forked at: 56753b77
Parent branch: origin/master

* Not sure what went wrong...

* WIP

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* CLEANUP

Forked at: 56753b77
Parent branch: origin/master

* fmt

* CLEANUP

Forked at: 56753b77
Parent branch: origin/master

* CLEANUP

Forked at: 56753b77
Parent branch: origin/master

* Remove pallet contracts (not used)

* Remove pallet parachain-info and token-dealer (not used)

* Sort dependencies alphabetically

* CLEANUP

Forked at: 56753b77
Parent branch: origin/master

* CumulusTestNode for testing

* Speed up block generation

* Fix improper shutdown

* rustfmt

* runtime: replace const by storage

* Fix for previous commit

* Remove some generics

* Move generate_genesis_state to cumulus-primitives

* fmt

* Remove message_example

* fixup! Remove message_example

* WIP

Forked at: 56753b77
Parent branch: origin/master

* Half the solution to previous commit :(

* Revert "Fix for previous commit"

This reverts commit 60010bab6797487093ac8c790b3a536f7ca0895b.

* Revert "runtime: replace const by storage"

This reverts commit c64b3a46f0325a98922015e0cbf3570e2e431774.

Not working for some reason...

* Use helper

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* Remove test-primitives

* Revert "Half the solution to previous commit :("

This reverts commit 9a8f89f9f06252198e6405057043c6b313f1aea4.

* Revert "Revert "Half the solution to previous commit :(""

This reverts commit 6a93f0f09d74ccdc3738dd78a777c483427c03ce.

* Test with some extra extrinsics

* WIP

Forked at: 56753b77
Parent branch: origin/master

* CLEANUP

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* WIP

Forked at: 56753b77
Parent branch: origin/master

* CLEANUP

Forked at: 56753b77
Parent branch: origin/master

* Remove message broker

* Initial commit

Forked at: 56753b77


Parent branch: origin/master

* Add --collator flag as alternative to --validator

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>