- Dec 07, 2023
-
-
Oliver Tale-Yazdi authoredCloses https://github.com/paritytech/polkadot-sdk/issues/2319 Changes: - Ensure that only `enqueue_message(s)` is callable from within the message processor. This prevents messed up storage that can currently happen when the pallet is called into recursively. - Use `H256` instead of `[u8; 32]` for clearer API. ## Details The re-entracy check is done with the `environmental` crate by adding a `with_service_mutex(f)` function that runs the closure exclusively. This works since the MQ pallet is not instantiable. --------- Signed-off-by:
Oliver Tale-Yazdi <[email protected]> Co-authored-by:
Francisco Aguirre <[email protected]>
-
- Nov 28, 2023
-
-
gupnik authored
Step in https://github.com/paritytech/polkadot-sdk/issues/171 This PR adds `derive_impl` on all `frame_system` config impls for mock runtimes. The overridden configs are maintained as of now to ensure minimal changes. --------- Co-authored-by:
-
- Nov 02, 2023
-
-
Oliver Tale-Yazdi authored
(imported from https://github.com/paritytech/cumulus/pull/2157) ## Changes This MR refactores the XCMP, Parachains System and DMP pallets to use the [MessageQueue](https://github.com/paritytech/substrate/pull/12485) for delayed execution of incoming messages. The DMP pallet is entirely replaced by the MQ and thereby removed. This allows for PoV-bounded execution and resolves a number of issues that stem from the current work-around. All System Parachains adopt this change. The most important changes are in `primitives/core/src/lib.rs`, `parachains/common/src/process_xcm_message.rs`, `pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs` and the runtime configs. ### DMP Queue Pallet The pallet got removed and its logic refactored into parachain-system. Overweight message management can be done directly through the MQ pallet. Final undeployment migrations are provided by `cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that can be configured with an aux config trait like: ```rust parameter_types! { pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME; pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent; } impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime { type PalletName = DmpQueuePalletName; type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>; type DbWeight = <Runtime as frame_system::Config>::DbWeight; } // And adding them to your Migrations tuple: pub type Migrations = ( ... cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>, cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>, ); ``` ### XCMP Queue pallet Removed all dispatch queue functionality. Incoming XCMP messages are now either: Immediately handled if they are Signals, enqueued into the MQ pallet otherwise. New config items for the XCMP queue pallet: ```rust /// The actual queue implementation that retains the messages for later processing. type XcmpQueue: EnqueueMessage<ParaId>; /// How a XCM over HRMP from a sibling parachain should be processed. type XcmpProcessor: ProcessMessage<Origin = ParaId>; /// The maximal number of suspended XCMP channels at the same time. #[pallet::constant] type MaxInboundSuspended: Get<u32>; ``` How to configure those: ```rust // Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since // the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s. type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>; // Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched // with origin `Junction::Sibling(…)`. type XcmpProcessor = ProcessFromSibling< ProcessXcmMessage< AggregateMessageOrigin, xcm_executor::XcmExecutor<xcm_config::XcmConfig>, RuntimeCall, >, >; // Not really important what to choose here. Just something larger than the maximal number of channels. type MaxInboundSuspended = sp_core::ConstU32<1_000>; ``` The `InboundXcmpStatus` storage item was replaced by `InboundXcmpSuspended` since it now only tracks inbound queue suspension and no message indices anymore. Now only sends the most recent channel `Signals`, as all prio ones are out-dated anyway. ### Parachain System pallet For `DMP` messages instead of forwarding them to the `DMP` pallet, it now pushes them to the configured `DmpQueue`. The message processing which was triggered in `set_validation_data` is now being done by the MQ pallet `on_initialize`. XCMP messages are still handed off to the `XcmpMessageHandler` (XCMP-Queue pallet) - no change here. New config items for the parachain system pallet: ```rust /// Queues inbound downward messages for delayed processing. /// /// Analogous to the `XcmpQueue` of the XCMP queue pallet. type DmpQueue: EnqueueMessage<AggregateMessageOrigin>; ``` How to configure: ```rust /// Use the MQ pallet to store DMP messages for delayed processing. type DmpQueue = MessageQueue; ``` ## Message Flow The flow of messages on the parachain side. Messages come in from the left via the `Validation Data` and finally end up at the `Xcm Executor` on the right.  ## Further changes - Bumped the default suspension, drop and resume thresholds in `QueueConfigData::default()`. - `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when they would be a noop. - Properly validate the `QueueConfigData` before setting it. - Marked weight files as auto-generated so they wont auto-expand in the MR files view. - Move the `hypothetical` asserts to `frame_support` under the name `experimental_hypothetically` Questions: - [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in the runtimes? Not sure how to best fix. Just having them like this makes tests fail that rely on the real message processor when the feature is enabled. - [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler already takes 80% so I put it to 10% but that is quite low. TODO: - [x] Remove c&p code after https://github.com/paritytech/polkadot/pull/6271 - [x] Use `HandleMessage` once it is public in Substrate - [x] fix `runtime-benchmarks` feature https://github.com/paritytech/polkadot/pull/6966 - [x] Benchmarks - [x] Tests - [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended` - [x] Possibly cleanup Migrations (DMP+XCMP) - [x] optional: create `TransformProcessMessageOrigin` in Substrate and replace `ProcessFromSibling` - [ ] Rerun weights on ref HW --------- Signed-off-by:Liam Aharon <[email protected]> Co-authored-by:
joe petrowski <[email protected]> Co-authored-by:
Kian Paimani <[email protected]> Co-authored-by: command-bot <>
-
- Oct 19, 2023
-
-
Oliver Tale-Yazdi authored
Changes: - Use a sensible limit for the overweight-cutoff of a single messages instead of the full configured `ServiceWeight`. - Add/Update tests --------- Signed-off-by:
-
- Sep 05, 2023
-
-
Marcin S. authoredCo-authored-by:
Keith Yeung <[email protected]> Co-authored-by:
-
- Sep 01, 2023
-
-
gupnik authored
Co-authored-by:Javier Viola <[email protected]>
-
- Aug 14, 2023
-
-
Deepanshu Hooda authored
* feat: Add try_state for message_queue * Update frame/message-queue/src/lib.rs change if let to while let and modify bump_service_head function Co-authored-by:
Juan <[email protected]>
-
- Jul 18, 2023
-
-
Oliver Tale-Yazdi authored
* Rename WeightMeter functions * Fixes Signed-off-by:
🤦 Signed-off-by:Bastian Köcher <[email protected]> * Update primitives/weights/src/weight_meter.rs Co-authored-by:
-
- Jul 14, 2023
-
-
juangirini authored
* replace Index by Nonce * replace Index by Nonce * replace Index by Nonce * replace Index by Nonce * replace Index by Nonce * wip * remove index in lieu of nonce * wip * remove accountnonce in lieu of nonce * add minor improvement * rebase and merge conflicts
-
- Jul 13, 2023
-
-
gupnik authored
Moves `Block` to `frame_system` instead of `construct_runtime` and removes `Header` and `BlockNumber` (#14437) * Initial setup * Adds node block * Uses UncheckedExtrinsic and removes Where section * Updates frame_system to use Block * Adds deprecation warning * Fixes pallet-timestamp * Removes Header and BlockNumber * Addresses review comments * Addresses review comments * Adds comment about compiler bug * Removes where clause * Refactors code * Fixes errors in cargo check * Fixes errors in cargo check * Fixes warnings in cargo check * Formatting * Fixes construct_runtime tests * Uses import instead of full path for BlockNumber * Uses import instead of full path for Header * Formatting * Fixes construct_runtime tests * Fixes imports in benchmarks * Formatting * Fixes construct_runtime tests * Formatting * Minor updates * Fixes construct_runtime ui tests * Fixes construct_runtime ui tests with 1.70 * Fixes docs * Fixes docs * Adds u128 mock block type * Fixes split example * fixes for cumulus * ".git/.scripts/commands/fmt/fmt.sh" * Updates new tests * Fixes fully-qualified path in few places * Formatting * Update frame/examples/default-config/src/lib.rs Co-authored-by:
-
- Jul 12, 2023
-
-
Michal Kucharczyk authored* frame::support: GenesisConfig types for Runtime enabled * frame::support: macro generating GenesisBuild::build for RuntimeGenesisConfig * frame: ambiguity BuildStorage vs GenesisBuild fixed * fix * RuntimeGenesisBuild added * Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed" This reverts commit 950f3d019d0e21c55a739c44cc19cdabd3ff0293. * Revert "fix" This reverts commit a2f76dd24e9a16cf9230d45825ed28787211118b. * Revert "RuntimeGenesisBuild added" This reverts commit 3c131b618138ced29c01ab8d15d8c6410c9e128b. * Revert "Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed"" This reverts commit 2b1ecd467231eddec69f8d328039ba48a380da3d. * Revert "Revert "fix"" This reverts commit fd7fa629adf579d83e30e6ae9fd162637fc45e30. * Code review suggestions * frame: BuildGenesisConfig added, BuildGenesis deprecated * frame: some pallets updated with BuildGenesisConfig * constuct_runtime: support for BuildGenesisConfig * frame::support: genesis_build macro supports BuildGenesisConfig * frame: BuildGenesisConfig added, BuildGenesis deprecated * Cargo.lock update * test-runtime: fixes * Revert "fix" This reverts commit a2f76dd24e9a16cf9230d45825ed28787211118b. * Revert "frame: ambiguity BuildStorage vs GenesisBuild fixed" This reverts commit 950f3d019d0e21c55a739c44cc19cdabd3ff0293. * self review * doc fixed * ui tests fixed * fmt * tests fixed * genesis_build macrto fixed for non-generic GenesisConfig * BuildGenesisConfig constraints added * warning fixed * some duplication removed * fmt * fix * doc tests fix * doc fix * cleanup: remove BuildModuleGenesisStorage * self review comments * fix * Update frame/treasury/src/tests.rs Co-authored-by:
Sebastian Kunert <[email protected]> * Update frame/support/src/traits/hooks.rs Co-authored-by:
Davide Galassi <[email protected]> * frame: phantom fields moved to the end of structs * chain-spec: Default::default cleanup * test-runtime: phantom at the end * merge master fixes * fix * fix * fix * fix * fix (facepalm) * Update frame/support/procedural/src/pallet/expand/genesis_build.rs Co-authored-by:
-
- Jun 28, 2023
-
-
Oliver Tale-Yazdi authored
* pallet-message-queue: add queue pausing Signed-off-by:
-
- Jun 19, 2023
-
-
Oleg Plakida authored
Co-authored-by: command-bot <>
-
- May 21, 2023
-
-
Gavin Wood authored
-
- Apr 13, 2023
-
-
Oliver Tale-Yazdi authored
* Align log Signed-off-by:
-
- Mar 16, 2023
-
-
Oliver Tale-Yazdi authored
* Empty commit * ".git/.scripts/commands/bench/bench.sh" all --------- Co-authored-by:Alexander Theißen <[email protected]> Co-authored-by: command-bot <>
-
- Mar 13, 2023
-
-
Vivek Pandya authored
* Remove use of trait Store from staking pallet * Remove use of trait Store from bounties pallet * Remove use of trait Store from collective pallet * Remove use of trait Store from babe pallet * Remove use of trait Store from assets pallet * Remove use of trait Store from grandpa pallet * Remove use of trait Store from balances pallet * Remove use of trait Store from authorship pallet * Remove use of trait Store from authority-discovery pallet * Remove use of trait Store from atomic-swap pallet * Remove use of trait Store from sudo pallet * Remove use of trait Store from scheduler pallet * Remove use of trait Store from scored-pool pallet * Remove use of trait Store from society pallet * Remove use of trait Store from lottery pallet * Remove use of trait Store from executive pallet * Remove use of trait Store from democracy pallet * Remove use of trait Store from elections-phragmen pallet * Remove use of trait Store from indices pallet * Remove use of trait Store from identity pallet * Remove use of trait Store from multisig pallet * Remove use of trait Store from merkle-mountain-range pallet * Remove use of trait Store from im-online pallet * Remove use of trait Store from membership pallet * Remove use of trait Store from nicks pallet * Remove use of trait Store from session pallet * Remove use of trait Store from transaction-payment pallet * Remove use of trait Store from utility pallet * Remove use of trait Store from child-bounties pallet * Remove use of trait Store from nis pallet * Remove use of trait Store from nfts pallet * Remove use of trait Store from conviction-voting pallet * Remove use of trait Store from treasury pallet * Remove use of trait Store from vesting pallet * Remove use of trait Store from preimage pallet * Remove use of trait Store from uniques pallet * Remove use of trait Store from ranked-collective pallet * Remove use of trait Store from beefy-mmr pallet * Remove use of trait Store from referenda pallet * Remove use of trait Store from whitelist pallet * Remove use of trait Store from alliance pallet * Remove use of trait Store from nomination-pools pallet * Remove use of trait Store from state-trie-migration pallet * Remove use of trait Store from message-queue pallet * Remove use of trait Store from root-offences pallet * Remove use of trait Store from root-testing pallet * Remove use of trait Store from timestamps pallet * Remove use of trait Store from system pallet * Remove use of trait Store from offences pallet * Remove use of trait Store from recovery pallet * Remove use of trait Store from node-authorization pallet * Remove use of trait Store from proxy pallet * Remove use of trait Store from benchmarking pallet * Remove use of trait Store from bags-list pallet * Add deprecated warning in store_trait * Change warning message * Run cargo fmt * Fix warning and update tests * Remove unnecessary allow deprecated * Remove use of trait Store * Fix mismatch in expected output * Minor update to warning message for deprecation of generate_store with Store trait attribute * Fixes as per review comments * Fixes as per review suggestions * Remove use of Store trait from core-fellowship pallet * Fix type in store_trait.rs * Fixes as pre review comment
-
- Mar 07, 2023
-
-
Oliver Tale-Yazdi authored
* Unknit permanently overweight books A book with only permanently overweight messages should be unkit from the ready ring. This does currently not happen since perm. overweight messages are not counted as "processed" and therefore not increase the "total_processed" counter. This is only a problem when the next and only message that is processed is overweight. Eventually this should resolve itself when another non-overweight message is enqueued and processed. But for correctness it should be unknitted. Signed-off-by:
-
- Mar 04, 2023
-
-
Gavin Wood authored
* Typo * Update Salary weight Signed-off-by:
-
- Mar 02, 2023
-
-
Oliver Tale-Yazdi authored
* Deprecate Weight::from_{ref_time, proof_size} Signed-off-by:🤦 Signed-off-by:
-
- Feb 28, 2023
-
-
Oliver Tale-Yazdi authored
* Bump default 'additional_trie_layers' to two The default here only works for extremely small runtimes, which have no more than 16 storage prefices. This is changed to a "sane" default of 2, which is save for runtimes with up to 4096 storage prefices (eg StorageValue). Signed-off-by:
-
- Feb 25, 2023
-
-
Oliver Tale-Yazdi authored
* Add Yield message processing error Signed-off-by:
-
- Feb 21, 2023
-
-
Vivek Pandya authored
* Change copyright year to 2023 from 2022 * Fix incorrect update of copyright year * Remove years from copy right header * Fix remaining files * Fix typo in a header and remove update-copyright.sh
-
- Jan 27, 2023
-
-
Sam Johnson authored
migrate new benchmarking syntax from `frame_support::benchmarking` to `frame_benchmarking::v2` (#13235) * * re-export frame_support::benchmarking in frame_benchmarking:: * prefer use frame_benchmarking::*; in examples, etc * switch to frame_benchmarking::v2 * completely migrate new benchmarking code out of frame_support * fix doc links * remove unneeded return Co-authored-by:
-
- Jan 26, 2023
-
-
Shawn Tabrizi authored
* initial impl * add template test * linear fit proof size * always record proof when tracking storage * calculate worst case pov * remove duplicate worst case * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs * more comment output * add cli for worst case map size * update name * clap does not support underscores * rename * expose worst case map values * improve some comments * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs * update template * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark pallet --chain=dev --steps=50 --repeat=20 --pallet=pallet_assets --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/assets/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix fmt * more fmt * more fmt * Dont panic when there is no proof Signed-off-by:
🤦 Signed-off-by:🤦 . Changing it to support that since otherwise the `pov_mode` would conflict with the others. Signed-off-by:Parity Bot <[email protected]> Co-authored-by:
Your Name <[email protected]>
-
- Jan 23, 2023
-
-
Sam Johnson authored
* add stub for new benchmark macro * benchmark syntax * add #[extrinsic call] separator * parse #[benchmark] item as a function * proper emission of error when #[extrinsic_call] annotation is missing * clean up * enclosing module via benchmarks! { } working * use an attribute macro on the module instead of benchmarks! { } * cargo fmt * working component implementation * WIP * working * add syntax for Linear<A, B> * parsing of param ranges (still need to build tuple though) * params parsing WIP * clean up (don't need extrinsic call name) * use proper Result syntax for BenchmarkDef parsing * proper parsing of Linear<0, 1> style args * successfully parse and make use of linear component ranges💥 * rename support variable => home because eventually will be moved * compile-time check that param range types implement ParamRange * switch to using balances as example, failing on instance pallet * successfully set up __origin and __call with balances💥 * clean up * use a module * don't need a variable for transfer * rename benchmark_transfer -> transfer because no longer conflicts * clean up * working with transfer_increasing_users as well💥 * re-add BareBlock * add comments for undocumented structs+functions+traits * refactor in preparation for removing module requirements * switch to a block instead of a module * use the outer macro pattern to to enable #[benchmarks] aggregation * successfully generate SelectedBenchmark💥 * implement components for SelectedBenchmark * implement instance for SelectedBenchmark * properly track #[extra] * working impl for fn benchmarks() * run_benchmarks WIP * finish run_benchmark! impl💥 * import balances transfer_best_case benchmark * import transfer_keep_alive balances pallet benchmark * import set_balance_creating balances pallet benchmark * import set_balance_killing balances pallet benchmark * import force_transfer balances pallet benchmark * add #[extra] annotation and docs to transfer_increasing_users * import transfer_all balances pallet benchmark * import force_unreserve balances pallet benchmark * prepare to implement impl_benchmark_test_suite! * ensure tests cover #[extra] before and after #[benchmark] tag * refactor * clean up * fix * move to outer * switch to benchmarks/instance_benchmarks * test impl almost done, strange compiler error * benchmark test suites working💥 * clean up * add stub and basic parsing for where_clause * working except where clause and extrinsic calls containing method chains * assume option (2) for now wrt https://github.com/paritytech/substrate/pull/12924#issuecomment-1372938718 * clean up * switch to attribute-style * properly handle where clauses * fix subtle missing where clause, now just MessageQueue issues * fix block formatting in message-queue pallet * switch to block vs non-block parsing of extrinsic call * working now but some benchmark tests failing * message-queue tests working (run order issue fixed)🎉 * add comments and internal docs for fame_support_procedural::benchmark * fix license years * docs for lib.rs * add docs to new support procedural macros * don't allow #[benchmark] outside of benchmarking module * add docs * use benchmark(extra, skip_meta) style args * update docs accordingly * appease clippy * bump ci * add notes about `extra` and `skip_meta` * fix doc tests * re-run CI * use `ignore` instead of `no_run` on doc examples * bump CI * replace some if-lets with if-elses * more refactoring of if-let statements * fix remaining if-lets in BenchmarkDef::from() * fix if-lets in benchmarks() * fix remaining if-lets, use nested find_map for extrinsic call * switch to use #[extrinsic_call] or #[block] situationally * refactor ExtrinsicCallDef => BenchmarkCallDef * update docs with info about #[block] * add macro stub for #[extrinsic_call] * fix docs and add stub for #[block] as well * remove unused extern crate line * fix clippy nits * Use V2 bench syntax in pallet-example-basic Just testing the dev-ex... Signed-off-by:
-
- Jan 07, 2023
-
-
Oliver Tale-Yazdi authored
Signed-off-by:
-
- Dec 12, 2022
-
-
Oliver Tale-Yazdi authored
* frame-system: explicit call index Signed-off-by:
-
Oliver Tale-Yazdi authored
* Fix license Signed-off-by:
-
- Dec 09, 2022
-
-
Gavin Wood authored
* The message queue * Make fully generic * Refactor * Docs * Refactor * Use iter not slice * Per-origin queues * Multi-queue processing * Introduce MaxReady * Remove MaxReady in favour of ready ring * Cleanups * ReadyRing and tests * Stale page reaping * from_components -> from_parts Signed-off-by:
Robert Klotzner <[email protected]> Signed-off-by:
-
