* Same procedure as not every day

* Remove warning

* Update

* Use DecodeLimit for decoding XCM messages

* Use decode_all_with_depth_limit in appropriate places

* Use decode_all while decoding byte vector

* cargo fmt

* ci: add quick-check with clippy and rustfmt

* chore: rustfmt round

* chore: set the same rustfmt config than substrate

* chore: fix formatting

* cI: remove clippy

* ci: switch to nightly for the checks

* ci: fix toolchains and naming

* ci: Limit the check to formatting

* chore: fix formatting

* Update .rustfmt.toml

* Update .rustfmt.toml

Co-authored-by: Bastian Köcher <[email protected]>

* Remove event pallet::metadata attributes

* Add scale-info deps, TypeInfo derives, update call variants

* Update metadata runtime APIs

* Add missing scale_info dependency, update rococo runtime API

* Add missing scale_info dependency

* Remove pushed diener patches

* Cargo.lock

* Add missing scale-info dependencies

* Fixes

* Statemint runtime fixes

* Call struct variant empty matches

* Add missing scale-info dependency

* Fixes

* scale-info 1.0

* cargo update -p xcm

* update lock

* Update Cargo.lock

* update to latest polkadot

* remove rpc_http_threads

https://github.com/paritytech/substrate/pull/9737

* replace task executor with tokio handler

https://github.com/paritytech/substrate/pull/9737

* fix test compilation?

* Update Cargo.lock

* cargo update

* remove unused

* Update substrate and polkadot

* Update test/client/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Update .gitignore and bring into line with XCM v2's SendError

* type

* Some work

* Weighed responses

* extra fields to xcm pallet

* Fixes

* Bump

* Use new MultiLocation syntax everywhere

* Make tests compile

* Remove Filter and use Contains instead

* Fixes

* Remove patch

* Bump Polkadot

* bump

* support for XCM v1

* Fixes

* Fix.

* Use `xcm::latest`

* Bump Polkadot

Co-authored-by: Bastian Köcher <[email protected]>

* Use definitions from polkadot.

* Remove unused imports.

* Rerun CI.

* Bring in code from moonbeam

* move it to mock.rs

* Actually build it 🤦

* Adds support for checking the timestamp inherent while validating a block

This adds support for checking the timestamp inherent while validating a
block. This will use the relay chain slot number * relay chain slot
duration to calculate a timestamp. This timestamp is used to check the
timestamp in the timestamp inherent.

* Update polkadot-parachains/rococo-runtime/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Update polkadot-parachains/statemine-runtime/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Update primitives/timestamp/src/lib.rs

Co-authored-by: André Silva <[email protected]>

* Fix warnings

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Start

* More work

* Add proc-macro for `validate_block`

* Make everything compile

* Add some test

* compact, need to be made optional and look into/compress child trie
roots/state.

* proto with child trie support

* Missing set_offchain_storage overload.

* right name

* Ignore offchain indexing in validation function.

* patch trie-db

* decompress from iter

* use compressed proof

* remove wasm blob from proof (no inject plugged yet)

* change lock

* update trie

* change in toml

* Revert "change in toml"

This reverts commit e588583b79e1a64e2a54157b40968aa6b586fbbe.

* use patch to branches

* i

* i:wq

* switch branch

* ii

* ok, needed to patch the runtime by putting substrate patch in polkadot
project.

* test passing with this conf

* actual lazy code fetch

* patch issue

* Code reorg

* restore commented tests.

* update deps.

* remove polka patch

* fixes

* remove patch

* revert cargo.lock

* cargo update -p sp-trie polkadot-service

* fix collator test (using parent state root).

* Update pallets/parachain-system/src/validate_block/implementation.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Remove encode_witness test function.

* Update pallets/parachain-system/src/validate_block/implementation.rs

* Fix compilation

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Start with a failing integration test & some refactorings

* More work

* Make it "work"

* Add `NullConsensus` for the test

* More refactorings

* Move stuff over to its own crate

* Refactorings

* Integrate it into `service` and make the test working

* Docs and some exit condition

* Use the real import queue

* Fix tests

* Update client/pov-recovery/src/active_candidate_recovery.rs

Co-authored-by: Bernhard Schuster <[email protected]>

* Fetch slot duration from the relay chain

* Docs

* Fixes

Co-authored-by: Bernhard Schuster <[email protected]>

* Introduce `CollectCollationInfo` runtime api

Instead of using well known keys to communicate information about a
collation between the runtime and the collator, we now use a runtime api
for this.

* Fixes bug

* Apply suggestions from code review

Co-authored-by: Sergei Shulepov <[email protected]>

* Doc update

Co-authored-by: Sergei Shulepov <[email protected]>

* Update polkadot

* Migrate all uses of MQC heads to merkle proofs

* Mass rename `relay_parent_storage_root`

* Restore parachain-system tests

* Update polkadot and libp2p swarm for testing

* Collapse match into an if let

Co-authored-by: Bastian Köcher <[email protected]>

* Start with something

* Update Substrate & Polkadot

* Start to make it compile

* Make it compile

* Begin with something

* Yep

* I'm a hacker

* Bring back the builder

* Make it work in some way

* Compile

* Parachains use their own "slot"

* Adds cumulus-pallet-aura

* Wrap AuRa import queue to disable equivocation checking by default

* Pass slot duration

* Check the seal when validating a block

* Adds missing file

* Try to make the seal working

* Fix it

* Some fixes

* Bring in the latest features to cleanup the code

* Update and make it compile

* Improve the import

* Start fixing

* More work

* Fix fix fix

* Make everything compile

* Small cleanups

* Rename and more docs

* Docs

* Fixes fixes fixes

* Update rococo-parachains/src/chain_spec.rs

* Update client/consensus/aura/src/lib.rs

Co-authored-by: André Silva <[email protected]>

* Update client/consensus/aura/src/lib.rs

Co-authored-by: André Silva <[email protected]>

* Update primitives/parachain-inherent/Cargo.toml

Co-authored-by: André Silva <[email protected]>

* Update primitives/parachain-inherent/Cargo.toml

* Update primitives/parachain-inherent/Cargo.toml

* Update primitives/parachain-inherent/Cargo.toml

Co-authored-by: Sergei Shulepov <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Introduce the converter into the hub

* Parachain recognises Rococo governance body as admin

* Whitespace

* Use UsingComponents for fee payment in XCM

* Fixes

* Fixes for XCM permissions

* Remove encode_call test

* Fixes

* Rococo Collator supports Shell runtime

* Fixes

* Fixes

* Initial draft of DMP Queue pallet

* DMP Queue builds.

* Companion for Polkadot gav-allow-xcm-exec

* Bump

* Fix std

* Fixes

* fix and improve docs

* fix compile errors in tests

* add test for try_service_message

* update cargo.lock

* Fixes

* Make test name read well

* Fixes

* Add a couple of simple tests

* Tests

* Tests

* Update pallets/dmp-queue/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update pallets/dmp-queue/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update pallets/dmp-queue/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update pallets/dmp-queue/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update pallets/dmp-queue/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update pallets/dmp-queue/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update pallets/dmp-queue/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Chain ID and ParaID don't collide

* Fixes

* Update pallets/dmp-queue/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Update pallets/dmp-queue/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Fixes

Co-authored-by: Alexander Popiak <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Add spambot

* Fixes

* Add some extra functions to spambot, bump version

* Lock..

* Aggregate HRMP (XCMP/HMP) messages. Payloads for spambot.

* Fix tests, bump Polkadot.

* Fix HMP tests

* Rename Hrmp -> Xcmp for handler/sender

* Use master branch

* Test Xcm message passing & rename away from HMP

* Docs

* Introduce fee payment mechanics into XCM.

* Rename spambot -> ping

* Lock

* XCMP message dispatch system reimagining

- Moved most of the logic into xcm-handler pallet
- Altered the outgoing XCMP API from push to pull
- Changed underlying outgoing queue data structures to avoid multi-page read/writes
- Introduced queuing for incoming messages
- Introduced signal messages as a flow-control sub-stream
- Introduced flow-control with basic threshold back-pressure
- Introduced overall weight limitation on messages executed
- Additonal alterations to XCM APIs for the new system

* Should process any remaining XCM messages when we're not doing anything else.

* Update API usage and preparation for the big build.

* Some build fixes

* Build fixes

* xcm-handler builds

* Fix warnings

* Docs

* Parachains system builds

* Parachain runtime building

* Fix build

* Introduce transfer_asset specialisation.

* Fixes

* Two-stage upgrade for parachains.

* Fixes

* Fixes

* Updates for message sending.

* Repotting/renaming. Add primitives/utility.

* Remove real-overseer and bump refs

* Configure & document Rococo XCM runtime.

* Add shell runtime, some companion changes for #8589

* Bumps & fixes

* Fix test

* Build fix

* Update pallets/xcmp-queue/src/lib.rs

Co-authored-by: Amar Singh <[email protected]>

* Make tests compile

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* remove unused

* remove unused event stuff

* Adds proper validation-worker to make integration tests work

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* import saturating

* remove panic test

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Amar Singh <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Add a command to purge the relay chain only

* WIP

* Update rococo-parachains/src/cli.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Move cli stuff to its own crate

* Copyright dates

* Test not working for some reason...

* WIP

* Revert "WIP"

This reverts commit f97cd63742c7df822e4a6e52a29db5e0f56b7bfa.

* Fix test to use provided relay chain

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Add hint about which database could not be purged

Co-authored-by: Bastian Köcher <[email protected]>

Currently validation data is shared by using a well known key between
the parachain system pallet and the validate block implementation. This
pr changes this by passing the parachain system directly to the validate
block implementation to make use of it. Besides that, we also store the
validation params in some thread local variable to make it inspectable
by parachain system. This moves the validation of validation data and
validation params to the parachain system pallet directly, instead of
having this hidden inside the validate block implementation.

Fixes: https://github.com/paritytech/cumulus/issues/217

This renames and moves the `SystemInherentData` into its own crate.
The struct is now called `ParachainInherentData`. Besides moving the
struct, this also moves the code for creating this struct into this crate.

* Move pallets to pallets folder and rename them

* Move genesis file to service

* Rename primitives to primitives-core

* Delete cumulus-runtime

* Move stuff to client folder and rename

* MQC auth

Update polkadot

WIP

* Update polkadot

* Silly syntax errors

* Fix typo

* Leave some comments and docs

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Introduce the MessageQueueChain structure

* Move the HRMP channel relevance check below

* Fix the `receive_hrmp_after_pause` test

* ValidationData is passed by reference

* Replace "to cumulus" with "to the collator"

* Update the test so that they are same as in polkadot

Co-authored-by: Bastian Köcher <[email protected]>

* Complete telemetry for parachain & relaychain

* Update Substrate & Polkadot

* rename parachain-{upgrade -> system}

* Merge message-broker into parachain-system

* Remove message-broker and clean up

* Update docs

* Test upward messages sending

And also update the relay-sproof-builder so that it allows to set the
relay dispatch queue size for the given parachain.

* Test horizontal message sending

* Remove old inherent definitions

* Replace ValidationData with PersistedValidationData

* Update polkadot

* Update polkadot

* Extend cumulus primitives with some relay chain exports

Follow https://github.com/paritytech/polkadot/pull/2194 to see the
polkadot PR

* collator: collect the state proof

This commit changes cumulus-collator so that it takes the relay chain
state at the relay parent and creates a storage proof that contains all
the required data for PVF.

* parachain-upgrade: use the proofs instead

This change is needed to make cumulus logic to not longer depend on the
transient validation data. As part of this change, in order to preserve
the current behavior `code_upgrade_allowed` now is computed on the
parachain side, rather than provided by polkadot.

Turned out that this requires to know the self parachain id so it was
added where needed.

* message-broker: use relay state to track limits

this should make sending messages safe from accidentally running over
the relay chain limits that were previously unknown.

* Update polkadot

So that `relay_storage_root` is available through `ValidationParams`

* Check `relay_storage_root` matches expected

Check that `relay_storage_root` submitted by the collator matches the
one that we receive in `validate_block` through `ValidationParams`

* Add a missing check for `dmq_mqc_head` while we are at it

* Update polkadot

* Fix tests that use the relay storage root

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Update message-broker/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Remove unneeded (&_)

* Fix unwraps

* Polish basti's suggestion

* Fix merge

* Bring back the System::can_set_code check

Co-authored-by: Bastian Köcher <[email protected]>

* The problem in the nutshell

* Make it compile

* make tests pass

* Update the parachain-upgrade module

* Fix collator tests

Co-authored-by: Bastian Köcher <[email protected]>

* initial mock

* integrate xcm-handler into runtime

* expose xcm send error

* oops

* better comment

* HRMP message ingestion

* Plumb hrmp_watermark to build_collation

* Plumb hrmp_watermark to ValidationResult

* Plumb hrmp outbound messages

* Implement message-broker part of HRMP

* Kill UPWARD_MESSAGES as well

Otherwise, they will get resent each block

* Add sudo versions for easier testing

* Remove the xcmp module

Not useful for the moment

* Doc for HRMP message handler

* Estimate the weight upper bound for on_finalize

* Remove a redundant type annotation

* fix spelling of a method

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Deabbreviate dmp and hrmp in the message ingestion type

* Don't use binary_search since it's broken by a following rotate

Instead use the linear search. We can afford linear search here since
due to limited scalability of HRMP we can only have at most a couple of
dozens of channels.

* Fix the watermark

Co-authored-by: Bastian Köcher <[email protected]>

* Start with something

* Whatever

* Update

* MOARE

* Make cumulus-network compile and tests work

* Update more and fixes

* More stuff

* More fixes

* Make collator build

* Make test almost work

* Remove contracts runtime

* More test work

* Make service compile

* Fix test-service

* Fix test client

* More fixes

* Fix collator test

* Fix network tests (again)

* Make everything compile, finally

* Fix tests

* Write test that should fail

* Add `WaitOnRelayChainBlock`

* Update git versions

* Make it all work

* Update logging

* Switch to provided method for pushing an extrinsic

* Try to debug CI

* Aaaa

* Only use Debug

* Updates

* Use native execution to hopefully make CI happy...

This is mostly a copy of the predating version with exception of some
renaming and alterations (e.g. the message handler takes an inbound
downward message by value, not by reference).

* Integrate DMP into collation

* Integrate processed_downward_messages as well

* Switch to polkadot master branch

* Fix compilation