- Nov 02, 2023
-
-
Oliver Tale-Yazdi authored(imported from https://github.com/paritytech/cumulus/pull/2157) ## Changes This MR refactores the XCMP, Parachains System and DMP pallets to use the [MessageQueue](https://github.com/paritytech/substrate/pull/12485) for delayed execution of incoming messages. The DMP pallet is entirely replaced by the MQ and thereby removed. This allows for PoV-bounded execution and resolves a number of issues that stem from the current work-around. All System Parachains adopt this change. The most important changes are in `primitives/core/src/lib.rs`, `parachains/common/src/process_xcm_message.rs`, `pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs` and the runtime configs. ### DMP Queue Pallet The pallet got removed and its logic refactored into parachain-system. Overweight message management can be done directly through the MQ pallet. Final undeployment migrations are provided by `cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that can be configured with an aux config trait like: ```rust parameter_types! { pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME; pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent; } impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime { type PalletName = DmpQueuePalletName; type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>; type DbWeight = <Runtime as frame_system::Config>::DbWeight; } // And adding them to your Migrations tuple: pub type Migrations = ( ... cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>, cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>, ); ``` ### XCMP Queue pallet Removed all dispatch queue functionality. Incoming XCMP messages are now either: Immediately handled if they are Signals, enqueued into the MQ pallet otherwise. New config items for the XCMP queue pallet: ```rust /// The actual queue implementation that retains the messages for later processing. type XcmpQueue: EnqueueMessage<ParaId>; /// How a XCM over HRMP from a sibling parachain should be processed. type XcmpProcessor: ProcessMessage<Origin = ParaId>; /// The maximal number of suspended XCMP channels at the same time. #[pallet::constant] type MaxInboundSuspended: Get<u32>; ``` How to configure those: ```rust // Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since // the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s. type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>; // Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched // with origin `Junction::Sibling(…)`. type XcmpProcessor = ProcessFromSibling< ProcessXcmMessage< AggregateMessageOrigin, xcm_executor::XcmExecutor<xcm_config::XcmConfig>, RuntimeCall, >, >; // Not really important what to choose here. Just something larger than the maximal number of channels. type MaxInboundSuspended = sp_core::ConstU32<1_000>; ``` The `InboundXcmpStatus` storage item was replaced by `InboundXcmpSuspended` since it now only tracks inbound queue suspension and no message indices anymore. Now only sends the most recent channel `Signals`, as all prio ones are out-dated anyway. ### Parachain System pallet For `DMP` messages instead of forwarding them to the `DMP` pallet, it now pushes them to the configured `DmpQueue`. The message processing which was triggered in `set_validation_data` is now being done by the MQ pallet `on_initialize`. XCMP messages are still handed off to the `XcmpMessageHandler` (XCMP-Queue pallet) - no change here. New config items for the parachain system pallet: ```rust /// Queues inbound downward messages for delayed processing. /// /// Analogous to the `XcmpQueue` of the XCMP queue pallet. type DmpQueue: EnqueueMessage<AggregateMessageOrigin>; ``` How to configure: ```rust /// Use the MQ pallet to store DMP messages for delayed processing. type DmpQueue = MessageQueue; ``` ## Message Flow The flow of messages on the parachain side. Messages come in from the left via the `Validation Data` and finally end up at the `Xcm Executor` on the right.  ## Further changes - Bumped the default suspension, drop and resume thresholds in `QueueConfigData::default()`. - `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when they would be a noop. - Properly validate the `QueueConfigData` before setting it. - Marked weight files as auto-generated so they wont auto-expand in the MR files view. - Move the `hypothetical` asserts to `frame_support` under the name `experimental_hypothetically` Questions: - [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in the runtimes? Not sure how to best fix. Just having them like this makes tests fail that rely on the real message processor when the feature is enabled. - [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler already takes 80% so I put it to 10% but that is quite low. TODO: - [x] Remove c&p code after https://github.com/paritytech/polkadot/pull/6271 - [x] Use `HandleMessage` once it is public in Substrate - [x] fix `runtime-benchmarks` feature https://github.com/paritytech/polkadot/pull/6966 - [x] Benchmarks - [x] Tests - [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended` - [x] Possibly cleanup Migrations (DMP+XCMP) - [x] optional: create `TransformProcessMessageOrigin` in Substrate and replace `ProcessFromSibling` - [ ] Rerun weights on ref HW --------- Signed-off-by:
Oliver Tale-Yazdi <[email protected]> Co-authored-by:
Liam Aharon <[email protected]> Co-authored-by:
joe petrowski <[email protected]> Co-authored-by:
Kian Paimani <[email protected]> Co-authored-by: command-bot <>
-
- Oct 11, 2023
-
-
Branislav Kontur authored
# Desription ## Summary This PR introduces several nits and tweaks to xcm emulator tests for system parachains. ## Explanation **Deduplicate `XcmPallet::send(` with root origin code** - Introduced `send_transact_to_parachain` which could be easily reuse for scenarios like _governance call from relay chain to parachain_. **Refactor `send_transact_sudo_from_relay_to_system_para_works`** - Test covered just one use-case which was moved to the `do_force_create_asset_from_relay_to_system_para`, so now we can extend this test with more _governance-like_ senarios. - Renamed to `send_transact_as_superuser_from_relay_to_system_para_works`. **Remove `send_transact_native_from_relay_to_system_para_fails` test** - This test and/or description is kind of misleading, because system paras support Native from relay chain by `RelayChainAsNative` with correct xcm origin. - It tested only sending on relay chain which should go directly to the relay chain unit-tests (does not even need to be in xcm emulator level). ## Future directions Check restructure parachains integration tests [issue](https://github.com/paritytech/polkadot-sdk/issues/1389) and [PR with more TODOs](https://github.com/paritytech/polkadot-sdk/pull/1693 ). --------- Co-authored-by:
Ignacio Palacios <[email protected]>
-
- Oct 10, 2023
-
-
Branislav Kontur authored
[xcm] Use `Weight::MAX` for `reserve_asset_deposited`, `receive_teleported_asset` benchmarks (#1726) # Description ## Summary Previously, the `pallet_xcm::do_reserve_transfer_assets` and `pallet_xcm::do_teleport_assets` functions relied on weight estimation for remote chain execution, which was based on guesswork derived from the local chain. This approach led to complications for runtimes that did not provide or support specific [XCM configurations](https://github.com/paritytech/polkadot-sdk/blob/7cbe0c76ef8fd2aabf9f07de0156941ce3ed44b0/polkadot/xcm/xcm-executor/src/config.rs#L43-L47) for `IsReserve` or `IsTeleporter`. Consequently, such runtimes had to resort to implementing hard-coded weights for XCM instructions like `reserve_asset_deposited` or `receive_teleported_asset` to support extrinsics such as `pallet_xcm::reserve_transfer_assets` and `pallet_xcm::teleport_assets`, which depended on remote weight estimation. The issue of remote weight estimation was addressed and resolved by [Pull Request #1645](https://github.com/paritytech/polkadot-sdk/pull/1645), which removed the need for remote weight estimation. ## Solution As a continuation of this improvement, the current PR proposes further cleanup by removing unnecessary hard-coded values and rectifying benchmark results with `Weight::MAX` that previously used `T::BlockWeights::get().max_block` as an override for unsupported XCM instructions like `ReserveAssetDeposited` and `ReceiveTeleportedAsset`. ## Questions - [x] Can we remove now also `Hardcoded till the XCM pallet is fixed` for `deposit_asset`? E.g. for AssetHubKusama [here](https://github.com/paritytech/polkadot-sdk/blob/7cbe0c76ef8fd2aabf9f07de0156941ce3ed44b0/cumulus/parachains/runtimes/assets/asset-hub-kusama/src/weights/xcm/mod.rs#L129-L134) - [x] Are comments like [this](https://github.com/paritytech/polkadot-sdk/blob/7cbe0c76ef8fd2aabf9f07de0156941ce3ed44b0/polkadot/runtime/kusama/src/weights/xcm/mod.rs#L94) `// Kusama doesn't support ReserveAssetDeposited, so this benchmark has a default weight` still relevant? Shouldnt be removed/changed? ## TODO - [x] `bench bot` regenerate xcm weights for all runtimes - [x] remove hard-coded stuff from system parachain weight files - [ ] when merged, open `polkadot-fellow/runtimes` PR ## References Fixes #1132 Closes #1132 Old polkadot repo [PR](https://github.com/paritytech/polkadot/pull/7546) --------- Co-authored-by: command-bot <>
-
- Sep 29, 2023
-
-
Sebastian Kunert authored
-
- Sep 05, 2023
-
-
joe petrowski authored
* add common libs * asset hubs * add westend * bridge hubs * collectives * contracts * emulated tests * parachain bin * delete collectives constants and update docs * integration tests should have apache license (some missing, some needed changing) * propagate features * fmt
-
- Sep 01, 2023
-
-
Ignacio Palacios authored* improve hygiene & clean up * improve hygiene & clean up - xcm-emulator * improve hygiene & clean up - common * improve hygiene & clean up - tests * improve hygiene & clean up - tests 2 * last hygiene bits * ".git/.scripts/commands/fmt/fmt.sh" --------- Co-authored-by: command-bot <>
-
- Aug 30, 2023
-
-
Przemek Rzad authored* Add missing Cumulus licenses * Typo * Add missing Substrate licenses * Single job checking the sub-repos in steps * Remove dates * Remove dates * Add missing (C) * Update FRAME UI tests Signed-off-by:
-
- Aug 10, 2023
-
-
Ignacio Palacios authored
* Allow functions to work over both parachains and relay chains * additional references * import * backup * refactoring para and relay traits * use runtime crates to build types * decouple ProcessMessage * decouple ProcessMessage 2 * dmp and xcmp handlers decouple * backup * refactor done * common int values working * added global ext with mutex * works for two mutex * single mutex and remove condvar * global test ext done * failing moving test_ext because relay block num * relay_block_number issue fixed * backup * Test working with assertions * assertions get Test as arg * DispatchArgs as generic * clean up * backup * teleports for asset-hub-kusama done * improve assert_expected_events macro * rename Test generics * check assertions for tuples * test assertions redone * reserve_transfer_assets done * send transact done * hrmp test for paras * hrmp channels test done * hrmp channels test done 2 * before modifying test dispatch * reserve tests done & Test dispatch fixed * reserve transfer local asset * force_create_and_mint_asset * force create and mint done * tests done * fix imports in common * common events refactored * add option to events attributes * asset-hub-polkadot tests done * asset-hub-westend half done * relay chain events move to common * remove failing send tests for asset-hub-westend * added events to bridge-hub-rococo * added events to collectives-polkadot * cargo clean up * fix asset-hub-westend tests * ".git/.scripts/commands/fmt/fmt.sh" * fix clippy * ".git/.scripts/commands/fmt/fmt.sh" * Removed unnecessary deps * Extracted some commonality for Kusama/Polkadot (which will be reused also for BridgeHubs) (#2971) * Extracted some commonality for Kusama/Polkadot (which will be reused also for BridgeHubs) * AssetHubRococo should better use AssetHubKusama runtime * add fund_account --------- Co-authored-by:
Giles Cope <[email protected]> Co-authored-by: command-bot <> Co-authored-by:
Branislav Kontur <[email protected]>
-
- Jul 05, 2023
-
-
Ignacio Palacios authored
* rename bridge hub + BridgeMessages type * bridge base * bridge string approach * add decl_test_bridges macro * outbound lanes on the way * get & dispatch mock bridge done * fix bridge errors + log::debug for messages * clean up * update source OutboundLaneData * rococo & wococo added * sender_receiver_accounts_parameter_types macro * sender_receiver_accounts_parameter_types macro 2 * fixed multi parachain + example on the way * working but router error * bridge working * refactor NetworkComponent * make it generic * working as generic * clean up * last bit * ".git/.scripts/commands/fmt/fmt.sh" * fix bridge hub handler name * ".git/.scripts/commands/fmt/fmt.sh" * add AssetConversion back for AssetHubWestend * Update xcm/xcm-emulator/src/lib.rs Co-authored-by:
-
