1. Nov 13, 2023
    • Adrian Catangiu's avatar
      pallet-xcm: enhance `reserve_transfer_assets` to support remote reserves (#1672) · 18257373
      Adrian Catangiu authored
      ## Motivation
      
      `pallet-xcm` is the main user-facing interface for XCM functionality,
      including assets manipulation functions like `teleportAssets()` and
      `reserve_transfer_assets()` calls.
      
      While `teleportAsset()` works both ways, `reserve_transfer_assets()`
      works only for sending reserve-based assets to a remote destination and
      beneficiary when the reserve is the _local chain_.
      
      ## Solution
      
      This PR enhances `pallet_xcm::(limited_)reserve_withdraw_assets` to
      support transfers when reserves are other chains.
      This will allow complete, **bi-directional** reserve-based asset
      transfers user stories using `pallet-xcm`.
      
      Enables following scenarios:
      - transferring assets with local reserve (was previously supported iff
      asset used as fee also had local reserve - now it works in all cases),
      - transferring assets with reserve on destination,
      - transferring assets with reserve on remote/third-party chain (iff
      assets and fees have same remote reserve),
      - transferring assets with reserve different than the reserve of the
      asset to be used as fees - meaning can be used to transfer random asset
      with local/dest reserve while using DOT for fees on all involved chains,
      even if DOT local/dest reserve doesn't match asset reserve,
      - transferring assets with any type of local/dest reserve while using
      fees which can be teleported between involved chains.
      
      All of the above is done by pallet inner logic without the user having
      to specify which scenario/reserves/teleports/etc. The correct scenario
      and corresponding XCM programs are identified, and respectively, built
      automatically based on runtime configuration of trusted teleporters and
      trusted reserves.
      
      #### Current limitations:
      - while `fees` and "non-fee" `assets` CAN have different reserves (or
      fees CAN be teleported), the remaining "non-fee" `assets` CANNOT, among
      themselves, have different reserve locations (this is also implicitly
      enforced by `MAX_ASSETS_FOR_TRANSFER=2`, but this can be safely
      increased in the future).
      - `fees` and "non-fee" `assets` CANNOT have **different remote**
      reserves (this could also be supported in the future, but adds even more
      complexity while possibly not being worth it - we'll see what the future
      holds).
      
      Fixes https://github.com/paritytech/polkadot-sdk/issues/1584
      Fixes https://github.com/paritytech/polkadot-sdk/issues/2055
      
      
      
      ---------
      
      Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
      Co-authored-by: default avatarBranislav Kontur <[email protected]>
      18257373
  2. Nov 10, 2023
    • PG Herveou's avatar
      Contracts: Add XCM traits to interface with contracts (#2086) · 6b7be115
      PG Herveou authored
      We are introducing a new set of `XcmController` traits (final name yet
      to be determined).
      These traits are implemented by `pallet-xcm` and allows other pallets,
      such as `pallet_contracts`, to rely on these traits instead of tight
      coupling them to `pallet-xcm`.
      
      Using only the existing Xcm traits would mean duplicating the logic from
      `pallet-xcm` in these other pallets, which we aim to avoid. Our
      objective is to ensure that when these APIs are called from
      `pallet-contracts`, they produce the exact same outcomes as if called
      directly from `pallet-xcm`.
      
      The other benefits is that we can also expose return values to
      `pallet-contracts` instead of just calling `pallet-xcm` dispatchable and
      getting a `DispatchResult` back.
      
      See traits integration in this PR
      https://github.com/paritytech/polkadot-sdk/pull/1248
      
      , where the traits
      are used as follow to define and implement `pallet-contracts` Config.
      ```rs
      // Contracts config:
      pub trait Config: frame_system::Config {
        // ...
      
        /// A type that exposes XCM APIs, allowing contracts to interact with other parachains, and
        /// execute XCM programs.
        type Xcm: xcm_executor::traits::Controller<
      	  OriginFor<Self>,
      	  <Self as frame_system::Config>::RuntimeCall,
      	  BlockNumberFor<Self>,
        >;
      }
      
      // implementation
      impl pallet_contracts::Config for Runtime {
              // ...
      
      	type Xcm = pallet_xcm::Pallet<Self>;
      }
      ```
      
      ---------
      
      Co-authored-by: default avatarAlexander Theißen <[email protected]>
      Co-authored-by: command-bot <>
      6b7be115
  3. Nov 09, 2023
  4. Nov 02, 2023
    • Oliver Tale-Yazdi's avatar
      Use `Message Queue` as DMP and XCMP dispatch queue (#1246) · e1c033eb
      Oliver Tale-Yazdi authored
      (imported from https://github.com/paritytech/cumulus/pull/2157)
      
      ## Changes
      
      This MR refactores the XCMP, Parachains System and DMP pallets to use
      the [MessageQueue](https://github.com/paritytech/substrate/pull/12485)
      for delayed execution of incoming messages. The DMP pallet is entirely
      replaced by the MQ and thereby removed. This allows for PoV-bounded
      execution and resolves a number of issues that stem from the current
      work-around.
      
      All System Parachains adopt this change.  
      The most important changes are in `primitives/core/src/lib.rs`,
      `parachains/common/src/process_xcm_message.rs`,
      `pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs`
      and the runtime configs.
      
      ### DMP Queue Pallet
      
      The pallet got removed and its logic refactored into parachain-system.
      Overweight message management can be done directly through the MQ
      pallet.
      
      Final undeployment migrations are provided by
      `cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that
      can be configured with an aux config trait like:
      
      ```rust
      parameter_types! {
      	pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME;
      	pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent;
      }
      
      impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime {
      	type PalletName = DmpQueuePalletName;
      	type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>;
      	type DbWeight = <Runtime as frame_system::Config>::DbWeight;
      }
      
      // And adding them to your Migrations tuple:
      pub type Migrations = (
      	...
      	cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>,
      	cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>,
      );
      ```
      
      ### XCMP Queue pallet
      
      Removed all dispatch queue functionality. Incoming XCMP messages are now
      either: Immediately handled if they are Signals, enqueued into the MQ
      pallet otherwise.
      
      New config items for the XCMP queue pallet:
      ```rust
      /// The actual queue implementation that retains the messages for later processing.
      type XcmpQueue: EnqueueMessage<ParaId>;
      
      /// How a XCM over HRMP from a sibling parachain should be processed.
      type XcmpProcessor: ProcessMessage<Origin = ParaId>;
      
      /// The maximal number of suspended XCMP channels at the same time.
      #[pallet::constant]
      type MaxInboundSuspended: Get<u32>;
      ```
      
      How to configure those:
      
      ```rust
      // Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since
      // the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s.
      type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>;
      
      // Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched
      // with origin `Junction::Sibling(…)`.
      type XcmpProcessor = ProcessFromSibling<
      	ProcessXcmMessage<
      		AggregateMessageOrigin,
      		xcm_executor::XcmExecutor<xcm_config::XcmConfig>,
      		RuntimeCall,
      	>,
      >;
      
      // Not really important what to choose here. Just something larger than the maximal number of channels.
      type MaxInboundSuspended = sp_core::ConstU32<1_000>;
      ```
      
      The `InboundXcmpStatus` storage item was replaced by
      `InboundXcmpSuspended` since it now only tracks inbound queue suspension
      and no message indices anymore.
      
      Now only sends the most recent channel `Signals`, as all prio ones are
      out-dated anyway.
      
      ### Parachain System pallet
      
      For `DMP` messages instead of forwarding them to the `DMP` pallet, it
      now pushes them to the configured `DmpQueue`. The message processing
      which was triggered in `set_validation_data` is now being done by the MQ
      pallet `on_initialize`.
      
      XCMP messages are still handed off to the `XcmpMessageHandler`
      (XCMP-Queue pallet) - no change here.
      
      New config items for the parachain system pallet:
      ```rust
      /// Queues inbound downward messages for delayed processing. 
      ///
      /// Analogous to the `XcmpQueue` of the XCMP queue pallet.
      type DmpQueue: EnqueueMessage<AggregateMessageOrigin>;
      ``` 
      
      How to configure:
      ```rust
      /// Use the MQ pallet to store DMP messages for delayed processing.
      type DmpQueue = MessageQueue;
      ``` 
      
      ## Message Flow
      
      The flow of messages on the parachain side. Messages come in from the
      left via the `Validation Data` and finally end up at the `Xcm Executor`
      on the right.
      
      ![Untitled
      (1)](https://github.com/paritytech/cumulus/assets/10380170/6cf8b377-88c9-4aed-96df-baace266e04d)
      
      ## Further changes
      
      - Bumped the default suspension, drop and resume thresholds in
      `QueueConfigData::default()`.
      - `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when
      they would be a noop.
      - Properly validate the `QueueConfigData` before setting it.
      - Marked weight files as auto-generated so they wont auto-expand in the
      MR files view.
      - Move the `hypothetical` asserts to `frame_support` under the name
      `experimental_hypothetically`
      
      Questions:
      - [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in
      the runtimes? Not sure how to best fix. Just having them like this makes
      tests fail that rely on the real message processor when the feature is
      enabled.
      - [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler
      already takes 80% so I put it to 10% but that is quite low.
      
      TODO:
      - [x] Remove c&p code after
      https://github.com/paritytech/polkadot/pull/6271
      - [x] Use `HandleMessage` once it is public in Substrate
      - [x] fix `runtime-benchmarks` feature
      https://github.com/paritytech/polkadot/pull/6966
      
      
      - [x] Benchmarks
      - [x] Tests
      - [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended`
      - [x] Possibly cleanup Migrations (DMP+XCMP)
      - [x] optional: create `TransformProcessMessageOrigin` in Substrate and
      replace `ProcessFromSibling`
      - [ ] Rerun weights on ref HW
      
      ---------
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: default avatarLiam Aharon <[email protected]>
      Co-authored-by: default avatarjoe petrowski <[email protected]>
      Co-authored-by: default avatarKian Paimani <[email protected]>
      Co-authored-by: command-bot <>
      e1c033eb
  5. Nov 01, 2023
  6. Oct 24, 2023
  7. Oct 20, 2023
    • Bastian Köcher's avatar
      `xcm`: Change `TypeInfo::path` to not include `staging` (#1948) · f3bf5c1a
      Bastian Köcher authored
      The `xcm` crate was renamed to `staging-xcm` to be able to publish it to
      crates.io as someone as squatted `xcm`. The problem with this rename is
      that the `TypeInfo` includes the crate name which ultimately lands in
      the metadata. The metadata is consumed by downstream users like
      `polkadot-js` or people building on top of `polkadot-js`. These people
      are using the entire `path` to find the type in the type registry. Thus,
      their code would break as the type path would now be [`staging_xcm`,
      `VersionedXcm`] instead of [`xcm`, `VersionedXcm`]. This pull request
      fixes this by renaming the path segment `staging_xcm` to `xcm`.
      
      This requires: https://github.com/paritytech/scale-info/pull/197
      
      
      
      ---------
      
      Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
      f3bf5c1a
  8. Oct 18, 2023
    • Keith Yeung's avatar
      Introduce XcmFeesToAccount fee manager (#1234) · 3dece311
      Keith Yeung authored
      
      
      Combination of paritytech/polkadot#7005, its addon PR
      paritytech/polkadot#7585 and its companion paritytech/cumulus#2433.
      
      This PR introduces a new XcmFeesToAccount struct which implements the
      `FeeManager` trait, and assigns this struct as the `FeeManager` in the
      XCM config for all runtimes.
      
      The struct simply deposits all fees handled by the XCM executor to a
      specified account. In all runtimes, the specified account is configured
      as the treasury account.
      
      XCM __delivery__ fees are now being introduced (unless the root origin
      is sending a message to a system parachain on behalf of the originating
      chain).
      
      # Note for reviewers
      
      Most file changes are tests that had to be modified to account for the
      new fees.
      Main changes are in:
      - cumulus/pallets/xcmp-queue/src/lib.rs <- To make it track the delivery
      fees exponential factor
      - polkadot/xcm/xcm-builder/src/fee_handling.rs <- Added. Has the
      FeeManager implementation
      - All runtime xcm_config files <- To add the FeeManager to the XCM
      configuration
      
      # Important note
      
      After this change, instructions that create and send a new XCM (Query*,
      Report*, ExportMessage, InitiateReserveWithdraw, InitiateTeleport,
      DepositReserveAsset, TransferReserveAsset, LockAsset and RequestUnlock)
      will require the corresponding origin account in the origin register to
      pay for transport delivery fees, and the onward message will fail to be
      sent if the origin account does not have the required amount. This
      delivery fee is on top of what we already collect as tx fees in
      pallet-xcm and XCM BuyExecution fees!
      
      Wallet UIs that want to expose the new delivery fee can do so using the
      formula:
      
      ```
      delivery_fee_factor * (base_fee + encoded_msg_len * per_byte_fee)
      ```
      
      where the delivery fee factor can be obtained from the corresponding
      pallet based on which transport you are using (UMP, HRMP or bridges),
      the base fee is a constant, the encoded message length from the message
      itself and the per byte fee is the same as the configured per byte fee
      for txs (i.e. `TransactionByteFee`).
      
      ---------
      
      Co-authored-by: default avatarBranislav Kontur <[email protected]>
      Co-authored-by: default avatarjoe petrowski <[email protected]>
      Co-authored-by: default avatarGiles Cope <[email protected]>
      Co-authored-by: command-bot <>
      Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
      Co-authored-by: default avatarLiam Aharon <[email protected]>
      Co-authored-by: default avatarKian Paimani <[email protected]>
      3dece311
  9. Oct 17, 2023
  10. Oct 12, 2023
  11. Oct 01, 2023
  12. Sep 20, 2023
    • joe petrowski's avatar
      Disable Calls to Identity Pallet (#1476) · 771c3fbd
      joe petrowski authored
      This PR filters calls from the Identity pallet from all Relay Chain
      runtimes as preparation to move the identity state and logic to a system
      parachain within each network.
      
      After this change is deployed to a runtime, no more changes such as
      adding new sub-identities will be possible. The frozen state will be
      part of the genesis state of the system chain. After the system chain
      launches, the pallet and all state will be removed from each Relay
      Chain.
      
      Applications and UIs that render display information from this pallet
      will need to read from the system chain when it launches.
      771c3fbd
  13. Sep 19, 2023
  14. Sep 18, 2023
    • Muharem Ismailov's avatar
      xcm-builder: PayOverXcm supports fallible convertors for asset kind and... · a8e82a36
      Muharem Ismailov authored
      xcm-builder: PayOverXcm supports fallible convertors for asset kind and beneficiary conversion  (#1572)
      
      `PayOverXcm` type accepts two converters to transform the `AssetKind`
      and `Beneficiary` parameter types into recognized `xcm` types. In this
      PR, we've modified the bounds for these converters, transitioning from
      `Convert` to `TryConvert`.
      
      One such use case for this adjustment is when dealing with versioned xcm
      types for `AssetKind` and `Beneficiary`. These types might be not
      convertible to the latest xcm version, hence the need for fallible
      conversion.
      
      This changes required for
      https://github.com/paritytech/polkadot-sdk/pull/1333
      a8e82a36
  15. Sep 14, 2023
    • Branislav Kontur's avatar
      [xcm_builder]: Do not consume `msg` on `NotApplicable` for remote exporters (#1519) · 76724ce9
      Branislav Kontur authored
      ## Summary
      
      Implementations of `SendXcm`'s `validate` should not consume `dest`
      and/or `msg` parameters in case of `NotApplicable` error.
      This commit aligns expected behavior for `UnpaidRemoteExporter` and
      `SovereignPaidRemoteExporter`.
      
      ## Testing
      
      Added `remote_exporters_does_not_consume_dest_or_msg_on_not_applicable`
      test which checks two possible cases:
      - `dest` is local
      - no configured exporter for `dest`
      76724ce9
  16. Sep 01, 2023
  17. Aug 31, 2023
  18. Aug 30, 2023
  19. Aug 29, 2023
  20. Aug 28, 2023
  21. Aug 25, 2023
  22. Aug 22, 2023
  23. Aug 19, 2023
    • Francisco Aguirre's avatar
      Add tests for PayOverXcm (#7319) · 85115402
      Francisco Aguirre authored
      * PayOverXcm: Add test skeleton
      
      * PayOverXcm: Add tests checking PayOverXcm generates correct message
      
      * Add PayOverXcm and salary pallet integration test
      
      * Fix XCM execution issue
      
      * Fix documentation issues
      
      * Add missing period at the end of comment
      
      * Fix mock problems
      
      * Add missing licenses
      
      * Fix lints
      
      * Fix more lints
      
      * Add docs to mock
      
      * Add transfer assertions
      
      * Remove magic numbers
      
      ---------
      
      Co-authored-by: parity-processbot <>
      85115402
  24. Aug 16, 2023
  25. Aug 14, 2023
  26. Aug 07, 2023
  27. Aug 04, 2023
  28. Aug 03, 2023
  29. Aug 01, 2023
  30. Jul 19, 2023
    • Francisco Aguirre's avatar
      Change Fixed to WeightInfoBounds for Polkadot (#7077) · cc9f8129
      Francisco Aguirre authored
      
      
      * Add polkadot XCM benchmarks
      
      * Add temp
      
      * ".git/.scripts/commands/bench/bench.sh" xcm polkadot pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm polkadot pallet_xcm_benchmarks::generic
      
      * Add weights to XCM on Polkadot
      
      * Make CI fail on old files
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Update template
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Add reserve_asset_deposited benchmark
      
      * ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::generic
      
      * Update weights
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Change initiate_reserve_deposit in runtime weights
      
      * Update weights
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Remove trusted reserves from runtimes
      
      * Fix pallet-xcm-benchmarks mock
      
      * Fix test
      
      * Change pallet xcm weigher in kusama
      
      * Fix
      
      * Remove merge conflict artifact
      
      * Remove initiate_reserve_withdraw from generic benchmarks
      
      * Add missing implementation to XCM benchmark
      
      * Fix failing karura test
      
      * Remove dbg!
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * Fix fmt
      
      * Revert "Fix fmt"
      
      This reverts commit 676f2d8db07d7427750c79f95494d4988d06fda5.
      
      * Fix fmt
      
      * Remove duplicated template code
      
      * Add back part of the template
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible
      
      * Don't skip reserve asset deposited benchmark
      
      * Remove call to non-generated benchmark yet
      
      * Underscore unused parameter
      
      * Skip not supported benchmarks and hardcode value
      
      * Remove ReserveAssetDeposited benchmark
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible
      
      * Add back ReserveAssetDeposited
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible
      
      * Use default benchmark for ReserveAssetDeposited
      
      * Add missing parameter
      
      * Revert reserve asset deposited benchmark
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm kusama pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm westend pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::fungible
      
      * Add 'real' benchmarks
      
      * Add TrustedReserve to actual XcmConfig
      
      * Add TrustedReserve to actual XcmConfig (fix)
      
      * Whitelist from benchmarking XCM storage keys read each block (#6871)
      
      * Whitelist from benchmarking XCM storage keys read each block
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime westend pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime rococo pallet_xcm
      
      * Remove XcmPallet SupportedVersion from the benchmark whitelist
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime kusama pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime westend pallet_xcm
      
      * ".git/.scripts/commands/bench/bench.sh" runtime rococo pallet_xcm
      
      * WIP
      
      * Add necessary traits, remove unnecessary whitelisted keys
      
      * Fix tests
      
      * Remove unused file
      
      * Remove unused import
      
      ---------
      
      Co-authored-by: command-bot <>
      
      * ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::fungible
      
      * ".git/.scripts/commands/bench/bench.sh" xcm westend pallet_xcm_benchmarks::fungible
      
      * Fix spellchecker issues
      
      * Remove unused migration code
      
      ---------
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: command-bot <>
      Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      cc9f8129