Skip to content
  1. Apr 25, 2024
  2. Jan 16, 2024
    • Francisco Aguirre's avatar
      XCMv4 (#1230) · 8428f678
      Francisco Aguirre authored
      
      
      # Note for reviewer
      
      Most changes are just syntax changes necessary for the new version.
      Most important files should be the ones under the `xcm` folder.
      
      # Description 
      
      Added XCMv4.
      
      ## Removed `Multi` prefix
      The following types have been renamed:
      - MultiLocation -> Location
      - MultiAsset -> Asset
      - MultiAssets -> Assets
      - InteriorMultiLocation -> InteriorLocation
      - MultiAssetFilter -> AssetFilter
      - VersionedMultiAsset -> VersionedAsset
      - WildMultiAsset -> WildAsset
      - VersionedMultiLocation -> VersionedLocation
      
      In order to fix a name conflict, the `Assets` in `xcm-executor` were
      renamed to `HoldingAssets`, as they represent assets in holding.
      
      ## Removed `Abstract` asset id
      
      It was not being used anywhere and this simplifies the code.
      
      Now assets are just constructed as follows:
      
      ```rust
      let asset: Asset = (AssetId(Location::new(1, Here)), 100u128).into();
      ```
      
      No need for specifying `Concrete` anymore.
      
      ## Outcome is now a named fields struct
      
      Instead of
      
      ```rust
      pub enum Outcome {
        Complete(Weight),
        Incomplete(Weight, Error),
        Error(Error),
      }
      ```
      
      we now have
      
      ```rust
      pub enum Outcome {
        Complete { used: Weight },
        Incomplete { used: Weight, error: Error },
        Error { error: Error },
      }
      ```
      
      ## Added Reanchorable trait
      
      Now both locations and assets implement this trait, making it easier to
      reanchor both.
      
      ## New syntax for building locations and junctions
      
      Now junctions are built using the following methods:
      
      ```rust
      let location = Location {
          parents: 1,
          interior: [Parachain(1000), PalletInstance(50), GeneralIndex(1984)].into()
      };
      ```
      
      or
      
      ```rust
      let location = Location::new(1, [Parachain(1000), PalletInstance(50), GeneralIndex(1984)]);
      ```
      
      And they are matched like so:
      
      ```rust
      match location.unpack() {
        (1, [Parachain(id)]) => ...
        (0, Here) => ...,
        (1, [_]) => ...,
      }
      ```
      
      This syntax is mandatory in v4, and has been also implemented for v2 and
      v3 for easier migration.
      
      This was needed to make all sizes smaller.
      
      # TODO
      - [x] Scaffold v4
      - [x] Port github.com/paritytech/polkadot/pull/7236
      - [x] Remove `Multi` prefix
      - [x] Remove `Abstract` asset id
      
      ---------
      
      Co-authored-by: command-bot <>
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      8428f678
  3. Dec 26, 2023
  4. Nov 02, 2023
    • Oliver Tale-Yazdi's avatar
      Use `Message Queue` as DMP and XCMP dispatch queue (#1246) · e1c033eb
      Oliver Tale-Yazdi authored
      (imported from https://github.com/paritytech/cumulus/pull/2157)
      
      ## Changes
      
      This MR refactores the XCMP, Parachains System and DMP pallets to use
      the [MessageQueue](https://github.com/paritytech/substrate/pull/12485)
      for delayed execution of incoming messages. The DMP pallet is entirely
      replaced by the MQ and thereby removed. This allows for PoV-bounded
      execution and resolves a number of issues that stem from the current
      work-around.
      
      All System Parachains adopt this change.  
      The most important changes are in `primitives/core/src/lib.rs`,
      `parachains/common/src/process_xcm_message.rs`,
      `pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs`
      and the runtime configs.
      
      ### DMP Queue Pallet
      
      The pallet got removed and its logic refactored into parachain-system.
      Overweight message management can be done directly through the MQ
      pallet.
      
      Final undeployment migrations are provided by
      `cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that
      can be configured with an aux config trait like:
      
      ```rust
      parameter_types! {
      	pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME;
      	pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent;
      }
      
      impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime {
      	type PalletName = DmpQueuePalletName;
      	type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>;
      	type DbWeight = <Runtime as frame_system::Config>::DbWeight;
      }
      
      // And adding them to your Migrations tuple:
      pub type Migrations = (
      	...
      	cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>,
      	cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>,
      );
      ```
      
      ### XCMP Queue pallet
      
      Removed all dispatch queue functionality. Incoming XCMP messages are now
      either: Immediately handled if they are Signals, enqueued into the MQ
      pallet otherwise.
      
      New config items for the XCMP queue pallet:
      ```rust
      /// The actual queue implementation that retains the messages for later processing.
      type XcmpQueue: EnqueueMessage<ParaId>;
      
      /// How a XCM over HRMP from a sibling parachain should be processed.
      type XcmpProcessor: ProcessMessage<Origin = ParaId>;
      
      /// The maximal number of suspended XCMP channels at the same time.
      #[pallet::constant]
      type MaxInboundSuspended: Get<u32>;
      ```
      
      How to configure those:
      
      ```rust
      // Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since
      // the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s.
      type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>;
      
      // Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched
      // with origin `Junction::Sibling(…)`.
      type XcmpProcessor = ProcessFromSibling<
      	ProcessXcmMessage<
      		AggregateMessageOrigin,
      		xcm_executor::XcmExecutor<xcm_config::XcmConfig>,
      		RuntimeCall,
      	>,
      >;
      
      // Not really important what to choose here. Just something larger than the maximal number of channels.
      type MaxInboundSuspended = sp_core::ConstU32<1_000>;
      ```
      
      The `InboundXcmpStatus` storage item was replaced by
      `InboundXcmpSuspended` since it now only tracks inbound queue suspension
      and no message indices anymore.
      
      Now only sends the most recent channel `Signals`, as all prio ones are
      out-dated anyway.
      
      ### Parachain System pallet
      
      For `DMP` messages instead of forwarding them to the `DMP` pallet, it
      now pushes them to the configured `DmpQueue`. The message processing
      which was triggered in `set_validation_data` is now being done by the MQ
      pallet `on_initialize`.
      
      XCMP messages are still handed off to the `XcmpMessageHandler`
      (XCMP-Queue pallet) - no change here.
      
      New config items for the parachain system pallet:
      ```rust
      /// Queues inbound downward messages for delayed processing. 
      ///
      /// Analogous to the `XcmpQueue` of the XCMP queue pallet.
      type DmpQueue: EnqueueMessage<AggregateMessageOrigin>;
      ``` 
      
      How to configure:
      ```rust
      /// Use the MQ pallet to store DMP messages for delayed processing.
      type DmpQueue = MessageQueue;
      ``` 
      
      ## Message Flow
      
      The flow of messages on the parachain side. Messages come in from the
      left via the `Validation Data` and finally end up at the `Xcm Executor`
      on the right.
      
      ![Untitled
      (1)](https://github.com/paritytech/cumulus/assets/10380170/6cf8b377-88c9-4aed-96df-baace266e04d)
      
      ## Further changes
      
      - Bumped the default suspension, drop and resume thresholds in
      `QueueConfigData::default()`.
      - `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when
      they would be a noop.
      - Properly validate the `QueueConfigData` before setting it.
      - Marked weight files as auto-generated so they wont auto-expand in the
      MR files view.
      - Move the `hypothetical` asserts to `frame_support` under the name
      `experimental_hypothetically`
      
      Questions:
      - [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in
      the runtimes? Not sure how to best fix. Just having them like this makes
      tests fail that rely on the real message processor when the feature is
      enabled.
      - [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler
      already takes 80% so I put it to 10% but that is quite low.
      
      TODO:
      - [x] Remove c&p code after
      https://github.com/paritytech/polkadot/pull/6271
      - [x] Use `HandleMessage` once it is public in Substrate
      - [x] fix `runtime-benchmarks` feature
      https://github.com/paritytech/polkadot/pull/6966
      
      
      - [x] Benchmarks
      - [x] Tests
      - [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended`
      - [x] Possibly cleanup Migrations (DMP+XCMP)
      - [x] optional: create `TransformProcessMessageOrigin` in Substrate and
      replace `ProcessFromSibling`
      - [ ] Rerun weights on ref HW
      
      ---------
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: default avatarLiam Aharon <[email protected]>
      Co-authored-by: default avatarjoe petrowski <[email protected]>
      Co-authored-by: default avatarKian Paimani <[email protected]>
      Co-authored-by: command-bot <>
      e1c033eb
  5. Sep 01, 2023
  6. Aug 30, 2023
  7. Aug 29, 2023
  8. Jul 19, 2023
    • Francisco Aguirre's avatar
      Change Fixed to WeightInfoBounds for Polkadot (#7077) · cc9f8129
      Francisco Aguirre authored
      
      
      * Add polkadot XCM benchmarks
      
      * Add temp
      
      * ".git/.scripts/commands/bench/bench.sh" xcm polkadot pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm polkadot pallet_xcm_benchmarks::generic
      
      * Add weights to XCM on Polkadot
      
      * Make CI fail on old files
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Update template
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Add reserve_asset_deposited benchmark
      
      * ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::generic
      
      * Update weights
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Change initiate_reserve_deposit in runtime weights
      
      * Update weights
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Remove trusted reserves from runtimes
      
      * Fix pallet-xcm-benchmarks mock
      
      * Fix test
      
      * Change pallet xcm weigher in kusama
      
      * Fix
      
      * Remove merge conflict artifact
      
      * Remove initiate_reserve_withdraw from generic benchmarks
      
      * Add missing implementation to XCM benchmark
      
      * Fix failing karura test
      
      * Remove dbg!
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * Fix fmt
      
      * Revert "Fix fmt"
      
      This reverts commit 676f2d8db07d7427750c79f95494d4988d06fda5.
      
      * Fix fmt
      
      * Remove duplicated template code
      
      * Add back part of the template
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible
      
      * Don't skip reserve asset deposited benchmark
      
      * Remove call to non-generated benchmark yet
      
      * Underscore unused parameter
      
      * Skip not supported benchmarks and hardcode value
      
      * Remove ReserveAssetDeposited benchmark
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible
      
      * Add back ReserveAssetDeposited
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible
      
      * Use default benchmark for ReserveAssetDeposited
      
      * Add missing parameter
      
      * Revert reserve asset deposited benchmark
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm kusama pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm westend pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::fungible
      
      * Add 'real' benchmarks
      
      * Add TrustedReserve to actual XcmConfig
      
      * Add TrustedReserve to actual XcmConfig (fix)
      
      * Whitelist from benchmarking XCM storage keys read each block (#6871)
      
      * Whitelist from benchmarking XCM storage keys read each block
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime westend pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime rococo pallet_xcm
      
      * Remove XcmPallet SupportedVersion from the benchmark whitelist
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime kusama pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime westend pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime rococo pallet_xcm
      
      * WIP
      
      * Add necessary traits, remove unnecessary whitelisted keys
      
      * Fix tests
      
      * Remove unused file
      
      * Remove unused import
      
      ---------
      
      Co-authored-by: command-bot <>
      
      * ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm westend pallet_xcm_benchmarks::fungible
      
      * Fix spellchecker issues
      
      * Remove unused migration code
      
      ---------
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: command-bot <>
      Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      cc9f8129
  9. Jul 18, 2023
  10. May 25, 2023
    • Gavin Wood's avatar
      XCM: Tools for uniquely referencing messages (#7234) · 85dfadff
      Gavin Wood authored
      
      
      * Tools for unique topic references
      
      * Formatting
      
      * Naming
      
      * Repot into routing.rs.
      
      * More things done
      
      * Universal Exporter supports topic-as-reference
      
      * Some tests for the topic routing
      
      * More tests
      
      * Paid bridge tests
      
      * Add message ID to sending events
      
      * Formatting
      
      * fix and integrate into test nets
      
      * Move DenyThenTry and friend from Cumulus
      
      * Append SetTopic rather than prepend
      
      * Docs
      
      * Docs
      
      * Work with new ProcessMessage ID API
      
      * Formatting
      
      * Fix build
      
      * Fixes
      
      * Formatting
      
      * Update xcm/xcm-builder/src/barriers.rs
      
      Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
      
      * Update xcm/xcm-builder/src/routing.rs
      
      Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
      
      * Docs
      
      * Rename message_hash
      
      * Formatting
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      * Rename
      
      * Another Rename
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      * Update xcm/xcm-builder/src/routing.rs
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      ---------
      
      Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
      Co-authored-by: command-bot <>
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      85dfadff
  11. May 21, 2023
  12. May 19, 2023