Skip to content
Snippets Groups Projects
  1. Nov 15, 2023
  2. Nov 02, 2023
    • Oliver Tale-Yazdi's avatar
      Use `Message Queue` as DMP and XCMP dispatch queue (#1246) · e1c033eb
      Oliver Tale-Yazdi authored
      
      (imported from https://github.com/paritytech/cumulus/pull/2157)
      
      ## Changes
      
      This MR refactores the XCMP, Parachains System and DMP pallets to use
      the [MessageQueue](https://github.com/paritytech/substrate/pull/12485)
      for delayed execution of incoming messages. The DMP pallet is entirely
      replaced by the MQ and thereby removed. This allows for PoV-bounded
      execution and resolves a number of issues that stem from the current
      work-around.
      
      All System Parachains adopt this change.  
      The most important changes are in `primitives/core/src/lib.rs`,
      `parachains/common/src/process_xcm_message.rs`,
      `pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs`
      and the runtime configs.
      
      ### DMP Queue Pallet
      
      The pallet got removed and its logic refactored into parachain-system.
      Overweight message management can be done directly through the MQ
      pallet.
      
      Final undeployment migrations are provided by
      `cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that
      can be configured with an aux config trait like:
      
      ```rust
      parameter_types! {
      	pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME;
      	pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent;
      }
      
      impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime {
      	type PalletName = DmpQueuePalletName;
      	type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>;
      	type DbWeight = <Runtime as frame_system::Config>::DbWeight;
      }
      
      // And adding them to your Migrations tuple:
      pub type Migrations = (
      	...
      	cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>,
      	cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>,
      );
      ```
      
      ### XCMP Queue pallet
      
      Removed all dispatch queue functionality. Incoming XCMP messages are now
      either: Immediately handled if they are Signals, enqueued into the MQ
      pallet otherwise.
      
      New config items for the XCMP queue pallet:
      ```rust
      /// The actual queue implementation that retains the messages for later processing.
      type XcmpQueue: EnqueueMessage<ParaId>;
      
      /// How a XCM over HRMP from a sibling parachain should be processed.
      type XcmpProcessor: ProcessMessage<Origin = ParaId>;
      
      /// The maximal number of suspended XCMP channels at the same time.
      #[pallet::constant]
      type MaxInboundSuspended: Get<u32>;
      ```
      
      How to configure those:
      
      ```rust
      // Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since
      // the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s.
      type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>;
      
      // Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched
      // with origin `Junction::Sibling(…)`.
      type XcmpProcessor = ProcessFromSibling<
      	ProcessXcmMessage<
      		AggregateMessageOrigin,
      		xcm_executor::XcmExecutor<xcm_config::XcmConfig>,
      		RuntimeCall,
      	>,
      >;
      
      // Not really important what to choose here. Just something larger than the maximal number of channels.
      type MaxInboundSuspended = sp_core::ConstU32<1_000>;
      ```
      
      The `InboundXcmpStatus` storage item was replaced by
      `InboundXcmpSuspended` since it now only tracks inbound queue suspension
      and no message indices anymore.
      
      Now only sends the most recent channel `Signals`, as all prio ones are
      out-dated anyway.
      
      ### Parachain System pallet
      
      For `DMP` messages instead of forwarding them to the `DMP` pallet, it
      now pushes them to the configured `DmpQueue`. The message processing
      which was triggered in `set_validation_data` is now being done by the MQ
      pallet `on_initialize`.
      
      XCMP messages are still handed off to the `XcmpMessageHandler`
      (XCMP-Queue pallet) - no change here.
      
      New config items for the parachain system pallet:
      ```rust
      /// Queues inbound downward messages for delayed processing. 
      ///
      /// Analogous to the `XcmpQueue` of the XCMP queue pallet.
      type DmpQueue: EnqueueMessage<AggregateMessageOrigin>;
      ``` 
      
      How to configure:
      ```rust
      /// Use the MQ pallet to store DMP messages for delayed processing.
      type DmpQueue = MessageQueue;
      ``` 
      
      ## Message Flow
      
      The flow of messages on the parachain side. Messages come in from the
      left via the `Validation Data` and finally end up at the `Xcm Executor`
      on the right.
      
      ![Untitled
      (1)](https://github.com/paritytech/cumulus/assets/10380170/6cf8b377-88c9-4aed-96df-baace266e04d)
      
      ## Further changes
      
      - Bumped the default suspension, drop and resume thresholds in
      `QueueConfigData::default()`.
      - `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when
      they would be a noop.
      - Properly validate the `QueueConfigData` before setting it.
      - Marked weight files as auto-generated so they wont auto-expand in the
      MR files view.
      - Move the `hypothetical` asserts to `frame_support` under the name
      `experimental_hypothetically`
      
      Questions:
      - [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in
      the runtimes? Not sure how to best fix. Just having them like this makes
      tests fail that rely on the real message processor when the feature is
      enabled.
      - [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler
      already takes 80% so I put it to 10% but that is quite low.
      
      TODO:
      - [x] Remove c&p code after
      https://github.com/paritytech/polkadot/pull/6271
      - [x] Use `HandleMessage` once it is public in Substrate
      - [x] fix `runtime-benchmarks` feature
      https://github.com/paritytech/polkadot/pull/6966
      - [x] Benchmarks
      - [x] Tests
      - [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended`
      - [x] Possibly cleanup Migrations (DMP+XCMP)
      - [x] optional: create `TransformProcessMessageOrigin` in Substrate and
      replace `ProcessFromSibling`
      - [ ] Rerun weights on ref HW
      
      ---------
      
      Signed-off-by: default avatarOliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
      Co-authored-by: default avatarLiam Aharon <liam.aharon@hotmail.com>
      Co-authored-by: default avatarjoe petrowski <25483142+joepetrowski@users.noreply.github.com>
      Co-authored-by: default avatarKian Paimani <5588131+kianenigma@users.noreply.github.com>
      Co-authored-by: command-bot <>
      e1c033eb
  3. Oct 26, 2023
  4. Oct 04, 2023
    • Bradley Olson's avatar
      Updating glutton for async backing (#1619) · 0a6dfdf9
      Bradley Olson authored
      Applied changes from the [User Update
      Guide](https://docs.google.com/document/d/1WQijD3bZTCsudOyPcDvugv659nCa2hEp2b_8eRU0h-Q),
      diverging in the node side where service.rs is different for
      `polkadot-parachain` than in the parachain template.
      0a6dfdf9
  5. Sep 27, 2023
  6. Sep 18, 2023
  7. Aug 30, 2023
  8. Aug 23, 2023
    • juangirini's avatar
      Companion: restructure macro related exports (#3015) · 8349c8d1
      juangirini authored
      * restructure macro related exports
      
      * restructure macro related exports
      
      * wip
      
      * wip
      
      * update cargo lock
      
      * refactor RuntimeDebug on unincluded segment
      
      * fmt
      
      * Companion: restructure `benchmarking` macro related exports (#3039)
      
      * wip
      
      * wip
      
      * restructure benchmarking macro related exports
      
      * add cargo lock
      
      ---------
      
      Co-authored-by: parity-processbot <>
      8349c8d1
  9. Aug 22, 2023
  10. Aug 18, 2023
    • Chris Sosnin's avatar
      Asynchronous backing PR (#2300) · 6ef11176
      Chris Sosnin authored
      
      * Update substrate & polkadot
      
      * min changes to make async backing compile
      
      * (async backing) parachain-system: track limitations for unincluded blocks (#2438)
      
      * unincluded segment draft
      
      * read para head from storage proof
      
      * read_para_head -> read_included_para_head
      
      * Provide pub interface
      
      * add errors
      
      * fix unincluded segment update
      
      * BlockTracker -> Ancestor
      
      * add a dmp limit
      
      * Read para head depending on the storage switch
      
      * doc comments
      
      * storage items docs
      
      * add a sanity check on block initialize
      
      * Check watermark
      
      * append to the segment on block finalize
      
      * Move segment update into set_validation_data
      
      * Resolve para head todo
      
      * option watermark
      
      * fix comment
      
      * Drop dmq check
      
      * fix weight
      
      * doc-comments on inherent invariant
      
      * Remove TODO
      
      * add todo
      
      * primitives tests
      
      * pallet tests
      
      * doc comments
      
      * refactor unincluded segment length into a ConsensusHook (#2501)
      
      * refactor unincluded segment length into a ConsensusHook
      
      * add docs
      
      * refactor bandwidth_out calculation
      
      Co-authored-by: default avatarChris Sosnin <48099298+slumber@users.noreply.github.com>
      
      * test for limits from impl
      
      * fmt
      
      * make tests compile
      
      * update comment
      
      * uncomment test
      
      * fix collator test by adding parent to state proof
      
      * patch HRMP watermark rules for unincluded segment
      
      * get consensus-common tests to pass, using unincluded segment
      
      * fix unincluded segment tests
      
      * get all tests passing
      
      * fmt
      
      * rustdoc CI
      
      * aura-ext: limit the number of authored blocks per slot (#2551)
      
      * aura_ext consensus hook
      
      * reverse dependency
      
      * include weight into hook
      
      * fix tests
      
      * remove stray println
      
      Co-authored-by: default avatarChris Sosnin <48099298+slumber@users.noreply.github.com>
      
      * fix test warning
      
      * fix doc link
      
      ---------
      
      Co-authored-by: default avatarChris Sosnin <48099298+slumber@users.noreply.github.com>
      Co-authored-by: default avatarChris Sosnin <chris125_@live.com>
      
      * parachain-system: ignore go ahead signal once upgrade is processed (#2594)
      
      * handle goahead signal for unincluded segment
      
      * doc comment
      
      * add test
      
      * parachain-system: drop processed messages from inherent data (#2590)
      
      * implement `drop_processed_messages`
      
      * drop messages based on relay parent number
      
      * adjust tests
      
      * drop changes to mqc
      
      * fix comment
      
      * drop test
      
      * drop more dead code
      
      * clippy
      
      * aura-ext: check slot in consensus hook and remove all `CheckInherents` logic (#2658)
      
      * aura-ext: check slot in consensus hook
      
      * convert relay chain slot
      
      * Make relay chain slot duration generic
      
      * use fixed velocity hook for pallets with aura
      
      * purge timestamp inherent
      
      * fix warning
      
      * adjust runtime tests
      
      * fix slots in tests
      
      * Make `xcm-emulator` test pass for new consensus hook (#2722)
      
      * add pallets on_initialize
      
      * tests pass
      
      * add AuraExt on_init
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      ---------
      
      Co-authored-by: command-bot <>
      
      ---------
      
      Co-authored-by: default avatarIgnacio Palacios <ignacio.palacios.santos@gmail.com>
      
      * update polkadot git refs
      
      * CollationGenerationConfig closure is now optional (#2772)
      
      * CollationGenerationConfig closure is now optional
      
      * fix test
      
      * propagate network-protocol-staging feature (#2899)
      
      * Feature Flagging Consensus Hook Type Parameter (#2911)
      
      * First pass
      
      * fmt
      
      * Added as default feature in tomls
      
      * Changed to direct dependency feature
      
      * Dealing with clippy error
      
      * Update pallets/parachain-system/src/lib.rs
      
      Co-authored-by: default avatarasynchronous rob <rphmeier@gmail.com>
      
      ---------
      
      Co-authored-by: default avatarasynchronous rob <rphmeier@gmail.com>
      
      * fmt
      
      * bump deps and remove warning
      
      * parachain-system: update RelevantMessagingState according to the unincluded segment (#2948)
      
      * mostly address 2471 with a bug introduced
      
      * adjust relevant messaging state after computing total
      
      * fmt
      
      * max -> min
      
      * fix test implementation of xcmp source
      
      * add test
      
      * fix test message sending logic
      
      * fix + test
      
      * add more to unincluded segment test
      
      * fmt
      
      ---------
      
      Co-authored-by: default avatarChris Sosnin <chris125_@live.com>
      
      * Integrate new Aura / Parachain Consensus Logic in Parachain-Template / Polkadot-Parachain (#2864)
      
      * add a comment
      
      * refactor client/service utilities
      
      * deprecate start_collator
      
      * update parachain-template
      
      * update test-service in the same way
      
      * update polkadot-parachain crate
      
      * fmt
      
      * wire up new SubmitCollation message
      
      * some runtime utilities for implementing unincluded segment runtime APIs
      
      * allow parachains to configure their level of sybil-resistance when starting the network
      
      * make aura-ext compile
      
      * update to specify sybil resistance levels
      
      * fmt
      
      * specify relay chain slot duration in milliseconds
      
      * update Aura to explicitly produce Send futures
      
      also, make relay_chain_slot_duration a Duration
      
      * add authoring duration to basic collator and document params
      
      * integrate new basic collator into parachain-template
      
      * remove assert_send used for testing
      
      * basic-aura: only author when parent included
      
      * update polkadot-parachain-bin
      
      * fmt
      
      * some fixes
      
      * fixes
      
      * add a RelayNumberMonotonicallyIncreases
      
      * add a utility function for initializing subsystems
      
      * some logging for timestamp adjustment
      
      * fmt
      
      * some fixes for lookahead collator
      
      * add a log
      
      * update `find_potential_parents` to account for sessions
      
      * bound the loop
      
      * restore & deprecate old start_collator and start_full_node functions.
      
      * remove unnecessary await calls
      
      * fix warning
      
      * clippy
      
      * more clippy
      
      * remove unneeded logic
      
      * ci
      
      * update comment
      
      Co-authored-by: default avatarMarcin S. <marcin@bytedude.com>
      
      * (async backing) restore `CheckInherents` for backwards-compatibility (#2977)
      
      * bring back timestamp
      
      * Restore CheckInherents
      
      * revert to empty CheckInherents
      
      * make CheckInherents optional
      
      * attempt
      
      * properly end system blocks
      
      * add some more comments
      
      * ignore failing system parachain tests
      
      * update refs after main feature branch merge
      
      * comment out the offending tests because CI runs ignored tests
      
      * fix warnings
      
      * fmt
      
      * revert to polkadot master
      
      * cargo update -p polkadot-primitives -p sp-io
      
      ---------
      
      Co-authored-by: default avatarasynchronous rob <rphmeier@gmail.com>
      Co-authored-by: default avatarIgnacio Palacios <ignacio.palacios.santos@gmail.com>
      Co-authored-by: default avatarBradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
      Co-authored-by: default avatarMarcin S. <marcin@bytedude.com>
      Co-authored-by: default avatareskimor <eskimor@users.noreply.github.com>
      Co-authored-by: default avatarAndronik <write@reusable.software>
      6ef11176
  11. Aug 16, 2023
  12. Jul 14, 2023
  13. Jul 13, 2023
    • gupnik's avatar
      Moves `Block` to `frame_system` instead of `construct_runtime` and removes... · 24d6e46a
      gupnik authored
      Moves `Block` to `frame_system` instead of `construct_runtime` and removes `Header` and `BlockNumber` (#2790)
      
      * Fixes
      
      * Removes unused import
      
      * Uses Block and removes BlockNumber/Header from Chain
      
      * Fixes bridges
      
      * Fixes
      
      * Removes unused import
      
      * Fixes build
      
      * Uses correct RelayBlock
      
      * Minor fix
      
      * Fixes glutton-kusama
      
      * Uses correct RelayBlock
      
      * Minor fix
      
      * Fixes benchmark for pallet-bridge-parachains
      
      * Adds appropriate constraints
      
      * Minor fixes
      
      * Removes unused import
      
      * Fixes integrity tests
      
      * Minor fixes
      
      * Updates trait bounds
      
      * Uses custom bound for AsPrimitive
      
      * Fixes trait bounds
      
      * Revert "Fixes trait bounds"
      
      This reverts commit 0b0f42f583f3a616a88afe45fcd06d31e7d9a06f.
      
      * Revert "Uses custom bound for AsPrimitive"
      
      This reverts commit 838e5281adf8b6e9632a2abb9cd550db4ae24126.
      
      * No AsPrimitive trait bound for now
      
      * Removes bounds on Number
      
      * update lockfile for {"substrate", "polkadot"}
      
      * Formatting
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      * Minor fix
      
      ---------
      
      Co-authored-by: parity-processbot <>
      24d6e46a
  14. Jul 12, 2023
    • Michal Kucharczyk's avatar
      `GenesisBuild<T,I>` deprecated. `BuildGenesisConfig` added (#2757) · 94d2e4bc
      Michal Kucharczyk authored
      * GenesisBuild<T,I> deprecated. BuildGenesisConfig added
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      * integration-tests/emulated: ..Default::default added to genesis configs
      
      * Cargo.lock updated
      
      * Cargo.lock updated
      
      * update lockfile for {"polkadot", "substrate"}
      
      * clippy fixes
      
      * clippy fixes
      
      * clippy fixes again
      
      ---------
      
      Co-authored-by: command-bot <>
      94d2e4bc
  15. Jun 23, 2023
    • Branislav Kontur's avatar
      Companion for substrate#14435 (BenchmarkHelper for frame-system) (#2766) · 9486fe97
      Branislav Kontur authored
      * Temporary fix for frame_system::set_code benchmark
      
      * Removed temprary fix
      
      * BenchmarkHelper for frame-system (TODO: add stuff for ParachainSystem OnSetCode)
      
      * BenchmarkHelper for frame-system (TODO: add stuff for ParachainSystem OnSetCode)
      
      * Glutton
      
      * Fix benchmarks for `set_code`
      
      * Changed dummy values to non-zero
      
      * update lockfile for {"polkadot", "substrate"}
      
      ---------
      
      Co-authored-by: parity-processbot <>
      9486fe97
  16. Jun 08, 2023
  17. May 31, 2023
  18. May 23, 2023
  19. May 17, 2023
  20. May 06, 2023
  21. Mar 15, 2023
  22. Jan 11, 2023
  23. Dec 08, 2022
  24. Oct 05, 2022
  25. Oct 03, 2022
  26. Sep 20, 2022
  27. Sep 14, 2022
  28. Sep 12, 2022
    • Sergej Sakac's avatar
      Companion for #11981 (#1563) · 68ba7e54
      Sergej Sakac authored
      
      * Companion for #11981
      
      * rename
      
      * Event to RuntimeEvent in imports
      
      * missed rename
      
      * undo
      
      * revert
      
      * rename type Call & Event
      
      * commit
      
      * ...
      
      * fix
      
      * fix errors
      
      * fixes
      
      * fmt
      
      * fix imports
      
      * final fix?
      
      * fmt
      
      * fix?
      
      * fixes after merge
      
      * small fix
      
      * cargo update -p polkadot-runtime-common
      
      * cargo +nightly fmt
      
      * update lockfile for {"polkadot", "substrate"}
      
      * fix
      
      Co-authored-by: default avatarShawn Tabrizi <shawntabrizi@gmail.com>
      Co-authored-by: parity-processbot <>
      68ba7e54
  29. Sep 01, 2022
  30. Aug 31, 2022
    • Shawn Tabrizi's avatar
      Companion for Weight v1.5 (#1581) · 48d4f1c5
      Shawn Tabrizi authored
      * cargo test -p cumulus-primitives-utility
      
      * cargo test -p cumulus-pallet-xcmp-queue
      
      * cargo test -p cumulus-pallet-xcm
      
      * cargo test -p cumulus-pallet-dmp-queue
      
      * cargo test -p pallet-template
      
      * cargo test -p cumulus-test-runtime
      
      * fix weights
      
      * fix more weights
      
      * cargo test -p parachains-common
      
      * cargo test -p parachain-template-runtime
      
      * fix weights import
      
      * cargo test -p collectives-polkadot-runtime
      
      * cargo test -p contracts-rococo-runtime
      
      * more
      
      * unused
      
      * fixes
      
      * Update benchmarking.rs
      
      * Update lib.rs
      
      * Update lib.rs
      
      * fix
      
      * fix bug in conversion
      
      * update lockfile for {"polkadot", "substrate"}
      
      Co-authored-by: parity-processbot <>
      48d4f1c5
  31. Aug 16, 2022
    • Branislav Kontur's avatar
      Removed dependance on `rococo_parachain_runtime` for unrelated stuff (#1500) · ffe19fd0
      Branislav Kontur authored
      * Clean misleading imports `rococo_parachain_runtime::{AccountId, AuraId}`
      
      * Remove dependance on `rococo_parachain_runtime` for unrelated stuff
      
      * fmt + rebase fix
      
      * Addressing review comments
      
      * Addressing review comments
      
      * Fix for `purge_chain_works` works now with `rococo-local` and purge command needs to allow `rococo-native`
      ffe19fd0
  32. Jun 05, 2022
  33. May 24, 2022
  34. May 20, 2022
  35. Mar 07, 2022
  36. Feb 11, 2022
  37. Jan 21, 2022
    • Keith Yeung's avatar
      Companion for paritytech/polkadot#4712 (#901) · 8c28d4c6
      Keith Yeung authored
      * Rename ParentIsDefault to ParentIsAllZeroes
      
      * Fixes
      
      * Create ParentAccounts for respective networks
      
      * Fixes
      
      * Use b"Parent" as the basis for generating parent AccountId
      
      * Fixes
      
      * Use preset parent account ID
      
      * update lockfile for {"polkadot"}
      
      Co-authored-by: parity-processbot <>
      8c28d4c6
  38. Jan 14, 2022
  39. Jan 03, 2022
  40. Dec 24, 2021
    • cheme's avatar
      Companion for substrate#9732 (#678) · 0256fe73
      cheme authored
      * state-update4 branch
      
      * new ref
      
      * Update to latest.
      
      * update deps
      
      * switch to host state version
      
      * update
      
      * fmt
      
      * up
      
      * remove trie patch
      
      * remove patch
      
      * fmt
      
      * update
      
      * set state_versions in runtimes
      
      * state version from storage
      
      * state version from storage
      
      * seedling compat
      
      * restore lock
      
      * update lockfile for substrate
      
      * update lockfile for polkadot
      
      Co-authored-by: parity-processbot <>
      0256fe73