- Mar 04, 2024
-
-
Gavin Wood authored
Closes #2160 First part of [Extrinsic Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415) Introduces a new trait `TransactionExtension` to replace `SignedExtension`. Introduce the idea of transactions which obey the runtime's extensions and have according Extension data (né Extra data) yet do not have hard-coded signatures. Deprecate the terminology of "Unsigned" when used for transactions/extrinsics owing to there now being "proper" unsigned transactions which obey the extension framework and "old-style" unsigned which do not. Instead we have __*General*__ for the former and __*Bare*__ for the latter. (Ultimately, the latter will be phased out as a type of transaction, and Bare will only be used for Inherents.) Types of extrinsic are now therefore: - Bare (no hardcoded signature, no Extra data; used to be known as "Unsigned") - Bare transactions (deprecated): Gossiped, validated with `ValidateUnsigned` (deprecated) and the `_bare_compat` bits of `TransactionExtension` (deprecated). - Inherents: Not gossiped, validated with `ProvideInherent`. - Extended (Extra data): Gossiped, validated via `TransactionExtension`. - Signed transactions (with a hardcoded signature). - General transactions (without a hardcoded signature). `TransactionExtension` differs from `SignedExtension` because: - A signature on the underlying transaction may validly not be present. - It may alter the origin during validation. - `pre_dispatch` is renamed to `prepare` and need not contain the checks present in `validate`. - `validate` and `prepare` is passed an `Origin` rather than a `AccountId`. - `validate` may pass arbitrary information into `prepare` via a new user-specifiable type `Val`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. It is encoded *for the entire transaction* and passed in to each extension as a new argument to `validate`. This facilitates the ability of extensions to acts as underlying crypto. There is a new `DispatchTransaction` trait which contains only default function impls and is impl'ed for any `TransactionExtension` impler. It provides several utility functions which reduce some of the tedium from using `TransactionExtension` (indeed, none of its regular functions should now need to be called directly). Three transaction version discriminator ("versions") are now permissible: - 0b000000100: Bare (used to be called "Unsigned"): contains Signature or Extra (extension data). After bare transactions are no longer supported, this will strictly identify an Inherents only. - 0b100000100: Old-school "Signed" Transaction: contains Signature and Extra (extension data). - 0b010000100: New-school "General" Transaction: contains Extra (extension data), but no Signature. For the New-school General Transaction, it becomes trivial for authors to publish extensions to the mechanism for authorizing an Origin, e.g. through new kinds of key-signing schemes, ZK proofs, pallet state, mutations over pre-authenticated origins or any combination of the above. ## Code Migration ### NOW: Getting it to build Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be accompanied by renaming your aggregate type in line with the new terminology. E.g. Before: ```rust /// The SignedExtension to the basic transaction logic. pub type SignedExtra = ( /* snip */ MySpecialSignedExtension, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>; ``` After: ```rust /// The extension to the basic transaction logic. pub type TxExtension = ( /* snip */ AsTransactionExtension<MySpecialSignedExtension>, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>; ``` You'll also need to alter any transaction building logic to add a `.into()` to make the conversion happen. E.g. Before: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let extra: SignedExtra = ( /* snip */ MySpecialSignedExtension::new(/* snip */), ); let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), extra, ) } ``` After: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let tx_ext: TxExtension = ( /* snip */ MySpecialSignedExtension::new(/* snip */).into(), ); let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), tx_ext, ) } ``` ### SOON: Migrating to `TransactionExtension` Most `SignedExtension`s can be trivially converted to become a `TransactionExtension`. There are a few things to know. - Instead of a single trait like `SignedExtension`, you should now implement two traits individually: `TransactionExtensionBase` and `TransactionExtension`. - Weights are now a thing and must be provided via the new function `fn weight`. #### `TransactionExtensionBase` This trait takes care of anything which is not dependent on types specific to your runtime, most notably `Call`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. - Weight must be returned by implementing the `weight` function. If your extension is associated with a pallet, you'll probably want to do this via the pallet's existing benchmarking infrastructure. #### `TransactionExtension` Generally: - `pre_dispatch` is now `prepare` and you *should not reexecute the `validate` functionality in there*! - You don't get an account ID any more; you get an origin instead. If you need to presume an account ID, then you can use the trait function `AsSystemOriginSigner::as_system_origin_signer`. - You get an additional ticket, similar to `Pre`, called `Val`. This defines data which is passed from `validate` into `prepare`. This is important since you should not be duplicating logic from `validate` to `prepare`, you need a way of passing your working from the former into the latter. This is it. - This trait takes two type parameters: `Call` and `Context`. `Call` is the runtime call type which used to be an associated type; you can just move it to become a type parameter for your trait impl. `Context` is not currently used and you can safely implement over it as an unbounded type. - There's no `AccountId` associated type any more. Just remove it. Regarding `validate`: - You get three new parameters in `validate`; all can be ignored when migrating from `SignedExtension`. - `validate` returns a tuple on success; the second item in the tuple is the new ticket type `Self::Val` which gets passed in to `prepare`. If you use any information extracted during `validate` (off-chain and on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can pass it through with this. For the tuple's last item, just return the `origin` argument. Regarding `prepare`: - This is renamed from `pre_dispatch`, but there is one change: - FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM `validate`!! - (This is different to `SignedExtension` which was required to run the same checks in `pre_dispatch` as in `validate`.) Regarding `post_dispatch`: - Since there are no unsigned transactions handled by `TransactionExtension`, `Pre` is always defined, so the first parameter is `Self::Pre` rather than `Option<Self::Pre>`. If you make use of `SignedExtension::validate_unsigned` or `SignedExtension::pre_dispatch_unsigned`, then: - Just use the regular versions of these functions instead. - Have your logic execute in the case that the `origin` is `None`. - Ensure your transaction creation logic creates a General Transaction rather than a Bare Transaction; this means having to include all `TransactionExtension`s' data. - `ValidateUnsigned` can still be used (for now) if you need to be able to construct transactions which contain none of the extension data, however these will be phased out in stage 2 of the Transactions Horizon, so you should consider moving to an extension-centric design. ## TODO - [x] Introduce `CheckSignature` impl of `TransactionExtension` to ensure it's possible to have crypto be done wholly in a `TransactionExtension`. - [x] Deprecate `SignedExtension` and move all uses in codebase to `TransactionExtension`. - [x] `ChargeTransactionPayment` - [x] `DummyExtension` - [x] `ChargeAssetTxPayment` (asset-tx-payment) - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment) - [x] `CheckWeight` - [x] `CheckTxVersion` - [x] `CheckSpecVersion` - [x] `CheckNonce` - [x] `CheckNonZeroSender` - [x] `CheckMortality` - [x] `CheckGenesis` - [x] `CheckOnlySudoAccount` - [x] `WatchDummy` - [x] `PrevalidateAttests` - [x] `GenericSignedExtension` - [x] `SignedExtension` (chain-polkadot-bulletin) - [x] `RefundSignedExtensionAdapter` - [x] Implement `fn weight` across the board. - [ ] Go through all pre-existing extensions which assume an account signer and explicitly handle the possibility of another kind of origin. - [x] `CheckNonce` should probably succeed in the case of a non-account origin. - [x] `CheckNonZeroSender` should succeed in the case of a non-account origin. - [x] `ChargeTransactionPayment` and family should fail in the case of a non-account origin. - [ ] - [x] Fix any broken tests. --------- Signed-off-by:georgepisaltu <[email protected]> Signed-off-by:
Alexandru Vasile <[email protected]> Signed-off-by:
dependabot[bot] <[email protected]> Signed-off-by:
Oliver Tale-Yazdi <[email protected]> Signed-off-by:
Alexandru Gheorghe <[email protected]> Signed-off-by:
Andrei Sandu <[email protected]> Co-authored-by:
Nikhil Gupta <[email protected]> Co-authored-by:
georgepisaltu <[email protected]> Co-authored-by:
Chevdor <[email protected]> Co-authored-by:
Bastian Köcher <[email protected]> Co-authored-by:
Maciej <[email protected]> Co-authored-by:
Javier Viola <[email protected]> Co-authored-by:
Marcin S. <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]> Co-authored-by:
Javier Bullrich <[email protected]> Co-authored-by:
Koute <[email protected]> Co-authored-by:
Adrian Catangiu <[email protected]> Co-authored-by:
Vladimir Istyufeev <[email protected]>
Co-authored-by: Ross Bulat <[email protected]> Co-authored-by:
Gonçalo Pestana <[email protected]> Co-authored-by:
Liam Aharon <[email protected]> Co-authored-by:
Svyatoslav Nikolsky <[email protected]> Co-authored-by:
André Silva <[email protected]> Co-authored-by:
s0me0ne-unkn0wn <[email protected]> Co-authored-by:
ordian <[email protected]> Co-authored-by:
Sebastian Kunert <[email protected]> Co-authored-by:
Aaro Altonen <[email protected]> Co-authored-by:
Dmitry Markin <[email protected]> Co-authored-by:
Alexandru Vasile <[email protected]> Co-authored-by:
Alexander Samusev <[email protected]> Co-authored-by:
Julian Eager <[email protected]> Co-authored-by:
Michal Kucharczyk <[email protected]> Co-authored-by:
Davide Galassi <[email protected]> Co-authored-by:
Dónal Murray <[email protected]> Co-authored-by:
yjh <[email protected]> Co-authored-by:
Tom Mi <[email protected]> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by:
Will | Paradox | ParaNodes.io <[email protected]> Co-authored-by:
Bastian Köcher <[email protected]> Co-authored-by:
Joshy Orndorff <[email protected]> Co-authored-by:
Joshy Orndorff <[email protected]> Co-authored-by:
PG Herveou <[email protected]> Co-authored-by:
Alexander Theißen <[email protected]> Co-authored-by:
Kian Paimani <[email protected]> Co-authored-by:
Juan Girini <[email protected]> Co-authored-by:
bader y <[email protected]> Co-authored-by:
James Wilson <[email protected]> Co-authored-by:
joe petrowski <[email protected]> Co-authored-by:
asynchronous rob <[email protected]> Co-authored-by:
Parth <[email protected]> Co-authored-by:
Andrew Jones <[email protected]> Co-authored-by:
Jonathan Udd <[email protected]> Co-authored-by:
Serban Iorga <[email protected]> Co-authored-by:
Egor_P <[email protected]> Co-authored-by:
Branislav Kontur <[email protected]> Co-authored-by:
Evgeny Snitko <[email protected]> Co-authored-by:
Just van Stam <[email protected]> Co-authored-by:
Francisco Aguirre <[email protected]> Co-authored-by:
gupnik <[email protected]> Co-authored-by:
dzmitry-lahoda <[email protected]> Co-authored-by:
zhiqiangxu <[email protected]> Co-authored-by:
Nazar Mokrynskyi <[email protected]> Co-authored-by:
Anwesh <[email protected]> Co-authored-by:
cheme <[email protected]> Co-authored-by:
Sam Johnson <[email protected]> Co-authored-by:
kianenigma <[email protected]> Co-authored-by:
Jegor Sidorenko <[email protected]> Co-authored-by:
Muharem <[email protected]> Co-authored-by:
joepetrowski <[email protected]> Co-authored-by:
Alexandru Gheorghe <[email protected]> Co-authored-by:
Gabriel Facco de Arruda <[email protected]> Co-authored-by:
Squirrel <[email protected]> Co-authored-by:
Andrei Sandu <[email protected]> Co-authored-by:
-
- Mar 01, 2024
-
-
Egor_P authored
This PR backports Node version and `spec_version` bumps to `1.8.0` from the latest release and orders prdoc files related to it.
-
Francisco Aguirre authored
If an XCM execution fails or ends with leftover assets, these will be trapped. In order to claim them, a custom XCM has to be executed, with the `ClaimAsset` instruction. However, arbitrary XCM execution is not allowed everywhere yet and XCM itself is still not easy enough to use for users out there with trapped assets. This new extrinsic in `pallet-xcm` will allow these users to easily claim their assets, without concerning themselves with writing arbitrary XCMs. Part of fixing https://github.com/paritytech/polkadot-sdk/issues/3495 --------- Co-authored-by: command-bot <> Co-authored-by:
-
- Feb 28, 2024
-
-
Oliver Tale-Yazdi authoredThis MR is the merge of https://github.com/paritytech/substrate/pull/14414 and https://github.com/paritytech/substrate/pull/14275. It implements [RFC#13](https://github.com/polkadot-fellows/RFCs/pull/13), closes https://github.com/paritytech/polkadot-sdk/issues/198. ----- This Merge request introduces three major topicals: 1. Multi-Block-Migrations 1. New pallet `poll` hook for periodic service work 1. Replacement hooks for `on_initialize` and `on_finalize` in cases where `poll` cannot be used and some more general changes to FRAME. The changes for each topical span over multiple crates. They are listed in topical order below. # 1.) Multi-Block-Migrations Multi-Block-Migrations are facilitated by creating `pallet_migrations` and configuring `System::Config::MultiBlockMigrator` to point to it. Executive picks this up and triggers one step of the migrations pallet per block. The chain is in lockdown mode for as long as an MBM is ongoing. Executive does this by polling `MultiBlockMigrator::ongoing` and not allowing any transaction in a block, if true. A MBM is defined through trait `SteppedMigration`. A condensed version looks like this: ```rust /// A migration that can proceed in multiple steps. pub trait SteppedMigration { type Cursor: FullCodec + MaxEncodedLen; type Identifier: FullCodec + MaxEncodedLen; fn id() -> Self::Identifier; fn max_steps() -> Option<u32>; fn step( cursor: Option<Self::Cursor>, meter: &mut WeightMeter, ) -> Result<Option<Self::Cursor>, SteppedMigrationError>; } ``` `pallet_migrations` can be configured with an aggregated tuple of these migrations. It then starts to migrate them one-by-one on the next runtime upgrade. Two things are important here: - 1. Doing another runtime upgrade while MBMs are ongoing is not a good idea and can lead to messed up state. - 2. **Pallet Migrations MUST BE CONFIGURED IN `System::Config`, otherwise it is not used.** The pallet supports an `UpgradeStatusHandler` that can be used to notify external logic of upgrade start/finish (for example to pause XCM dispatch). Error recovery is very limited in the case that a migration errors or times out (exceeds its `max_steps`). Currently the runtime dev can decide in `FailedMigrationHandler::failed` how to handle this. One follow-up would be to pair this with the `SafeMode` pallet and enact safe mode when an upgrade fails, to allow governance to rescue the chain. This is currently not possible, since governance is not `Mandatory`. ## Runtime API - `Core`: `initialize_block` now returns `ExtrinsicInclusionMode` to inform the Block Author whether they can push transactions. ### Integration Add it to your runtime implementation of `Core` and `BlockBuilder`: ```patch diff --git a/runtime/src/lib.rs b/runtime/src/lib.rs @@ impl_runtime_apis! { impl sp_block_builder::Core<Block> for Runtime { - fn initialize_block(header: &<Block as BlockT>::Header) { + fn initialize_block(header: &<Block as BlockT>::Header) -> RuntimeExecutiveMode { Executive::initialize_block(header) } ... } ``` # 2.) `poll` hook A new pallet hook is introduced: `poll`. `Poll` is intended to replace mostly all usage of `on_initialize`. The reason for this is that any code that can be called from `on_initialize` cannot be migrated through an MBM. Currently there is no way to statically check this; the implication is to use `on_initialize` as rarely as possible. Failing to do so can result in broken storage invariants. The implementation of the poll hook depends on the `Runtime API` changes that are explained above. # 3.) Hard-Deadline callbacks Three new callbacks are introduced and configured on `System::Config`: `PreInherents`, `PostInherents` and `PostTransactions`. These hooks are meant as replacement for `on_initialize` and `on_finalize` in cases where the code that runs cannot be moved to `poll`. The reason for this is to make the usage of HD-code (hard deadline) more explicit - again to prevent broken invariants by MBMs. # 4.) FRAME (general changes) ## `frame_system` pallet A new memorize storage item `InherentsApplied` is added. It is used by executive to track whether inherents have already been applied. Executive and can then execute the MBMs directly between inherents and transactions. The `Config` gets five new items: - `SingleBlockMigrations` this is the new way of configuring migrations that run in a single block. Previously they were defined as last generic argument of `Executive`. This shift is brings all central configuration about migrations closer into view of the developer (migrations that are configured in `Executive` will still work for now but is deprecated). - `MultiBlockMigrator` this can be configured to an engine that drives MBMs. One example would be the `pallet_migrations`. Note that this is only the engine; the exact MBMs are injected into the engine. - `PreInherents` a callback that executes after `on_initialize` but before inherents. - `PostInherents` a callback that executes after all inherents ran (including MBMs and `poll`). - `PostTransactions` in symmetry to `PreInherents`, this one is called before `on_finalize` but after all transactions. A sane default is to set all of these to `()`. Example diff suitable for any chain: ```patch @@ impl frame_system::Config for Test { type MaxConsumers = ConstU32<16>; + type SingleBlockMigrations = (); + type MultiBlockMigrator = (); + type PreInherents = (); + type PostInherents = (); + type PostTransactions = (); } ``` An overview of how the block execution now looks like is here. The same graph is also in the rust doc. <details><summary>Block Execution Flow</summary> <p>  </p> </details> ## Inherent Order Moved to https://github.com/paritytech/polkadot-sdk/pull/2154 --------------- ## TODO - [ ] Check that `try-runtime` still works - [ ] Ensure backwards compatibility with old Runtime APIs - [x] Consume weight correctly - [x] Cleanup --------- Signed-off-by:Gavin Wood <[email protected]> Co-authored-by:
-
Liam Aharon authoredCloses https://github.com/paritytech/polkadot-sdk-docs/issues/55 - Changes 'current storage version' terminology to less ambiguous 'in-code storage version' (suggestion by @ggwpez ) - Adds a new example pallet `pallet-example-single-block-migrations` - Adds a new reference doc to replace https://docs.substrate.io/maintain/runtime-upgrades/ (temporarily living in the pallet while we wait for developer hub PR to merge) - Adds documentation for the `storage_alias` macro - Improves `trait Hooks` docs - Improves `trait GetStorageVersion` docs - Update the suggested patterns for using `VersionedMigration`, so that version unchecked migrations are never exported - Prevents accidental usage of version unchecked migrations in runtimes https://github.com/paritytech/substrate/pull/14421#discussion_r1255467895 - Unversioned migration code is kept inside `mod version_unchecked`, versioned code is kept in `pub mod versioned` - It is necessary to use modules to limit visibility because the inner migration must be `pub`. See https://github.com/rust-lang/rust/issues/30905 and https://internals.rust-lang.org/t/lang-team-minutes-private-in-public-rules/4504/40 for more. ### todo - [x] move to reference docs to proper place within sdk-docs (now that https://github.com/paritytech/polkadot-sdk/pull/2102 is merged) - [x] prdoc --------- Co-authored-by:
-
- Feb 26, 2024
-
-
Branislav Kontur authored
## Problem During the bumping of the `polkadot-fellows` repository to `[email protected]`, I encountered a situation where the benchmarks `teleport_assets` and `reserve_transfer_assets` in AssetHubKusama started to fail. This issue arose due to a decreased ED balance for AssetHubs introduced [here](https://github.com/polkadot-fellows/runtimes/pull/158/files#diff-80668ff8e793b64f36a9a3ec512df5cbca4ad448c157a5d81abda1b15f35f1daR213), and also because of a [missing CI pipeline](https://github.com/polkadot-fellows/runtimes/issues/197) to check the benchmarks, which went unnoticed. These benchmarks expect the `caller` to have enough: 1. balance to transfer (BTT) 2. balance for paying delivery (BFPD). So the initial balance was calculated as `ED * 100`, which seems reasonable: ``` const ED_MULTIPLIER: u32 = 100; let balance = existential_deposit.saturating_mul(ED_MULTIPLIER.into());` ``` The problem arises when the price for delivery is 100 times higher than the existential deposit. In other words, when `ED * 100` does not cover `BTT` + `BFPD`. I check AHR/AHW/AHK/AHP and this problem has only AssetHubKusama ``` ED: 3333333 calculated price to parent delivery: 1031666634 (from xcm logs from the benchmark) --- 3333333 * 100 - BTT(3333333) - BFPD(1031666634) = −701666667 ``` which results in the error; ``` 2024-02-23 09:19:42 Unable to charge fee with error Module(ModuleError { index: 31, error: [17, 0, 0, 0], message: Some("FeesNotMet") }) Error: Input("Benchmark pallet_xcm::reserve_transfer_assets failed: FeesNotMet") ``` ## Solution The benchmarks `teleport_assets` and `reserve_transfer_assets` were fixed by removing `ED * 100` and replacing it with `DeliveryHelper` logic, which calculates the (almost real) price for delivery and sets it along with the existential deposit as the initial balance for the account used in the benchmark. ## TODO - [ ] patch for 1.6 - https://github.com/paritytech/polkadot-sdk/pull/3466 - [ ] patch for 1.7 - https://github.com/paritytech/polkadot-sdk/pull/3465 - [ ] patch for 1.8 - TODO: PR --------- Co-authored-by:
-
- Feb 20, 2024
-
-
Branislav Kontur authored
-
- Feb 09, 2024
-
-
Egor_P authored
This PR backports version bumps from `1.7.0` release branch and moves related prdoc files to the appropriate folder.
-
- Feb 06, 2024
-
-
Branislav Kontur authored
Relates to: https://github.com/paritytech/polkadot-sdk/issues/3214 ## TODO - [ ] backport to the `1.7.0` release
-
- Feb 02, 2024
-
-
Marcin S. authored
Built on top of https://github.com/paritytech/polkadot-sdk/pull/2826/ which was a trial run. Guide: https://github.com/w3f/polkadot-wiki/blob/master/docs/maintain/maintain-guides-async-backing.md --------- Signed-off-by:
Dmitry Sinyavin <[email protected]> Co-authored-by:
-
- Jan 31, 2024
-
-
Branislav Kontur authored
[frame] `#[pallet::composite_enum]` improved variant count handling + removed `pallet_balances`'s `MaxHolds` config (#2657) I started this investigation/issue based on @liamaharon question [here](https://github.com/paritytech/polkadot-sdk/pull/1801#discussion_r1410452499). ## Problem The `pallet_balances` integrity test should correctly detect that the runtime has correct distinct `HoldReasons` variant count. I assume the same situation exists for RuntimeFreezeReason. It is not a critical problem, if we set `MaxHolds` with a sufficiently large value, everything should be ok. However, in this case, the integrity_test check becomes less useful. **Situation for "any" runtime:** - `HoldReason` enums from different pallets: ```rust /// from pallet_nis #[pallet::composite_enum] pub enum HoldReason { NftReceipt, } /// from pallet_preimage #[pallet::composite_enum] pub enum HoldReason { Preimage, } // from pallet_state-trie-migration #[pallet::composite_enum] pub enum HoldReason { SlashForContinueMigrate, SlashForMigrateCustomTop, SlashForMigrateCustomChild, } ``` - generated `RuntimeHoldReason` enum looks like: ```rust pub enum RuntimeHoldReason { #[codec(index = 32u8)] Preimage(pallet_preimage::HoldReason), #[codec(index = 38u8)] Nis(pallet_nis::HoldReason), #[codec(index = 42u8)] StateTrieMigration(pallet_state_trie_migration::HoldReason), } ``` - composite enum `RuntimeHoldReason` variant count is detected as `3` - we set `type MaxHolds = ConstU32<3>` - `pallet_balances::integrity_test` is ok with `3`(at least 3) However, the real problem can occur in a live runtime where some functionality might stop working. This is due to a total of 5 distinct hold reasons (for pallets with multi-instance support, it is even more), and not all of them can be used because of an incorrect `MaxHolds`, which is deemed acceptable according to the `integrity_test`: ``` // pseudo-code - if we try to call all of these: T::Currency::hold(&pallet_nis::HoldReason::NftReceipt.into(), &nft_owner, deposit)?; T::Currency::hold(&pallet_preimage::HoldReason::Preimage.into(), &nft_owner, deposit)?; T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForContinueMigrate.into(), &nft_owner, deposit)?; // With `type MaxHolds = ConstU32<3>` these two will fail T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForMigrateCustomTop.into(), &nft_owner, deposit)?; T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForMigrateCustomChild.into(), &nft_owner, deposit)?; ``` ## Solutions A macro `#[pallet::*]` expansion is extended of `VariantCount` implementation for the `#[pallet::composite_enum]` enum type. This expansion generates the `VariantCount` implementation for pallets' `HoldReason`, `FreezeReason`, `LockId`, and `SlashReason`. Enum variants must be plain enum values without fields to ensure a deterministic count. The composite runtime enum, `RuntimeHoldReason` and `RuntimeFreezeReason`, now sets `VariantCount::VARIANT_COUNT` as the sum of pallets' enum `VariantCount::VARIANT_COUNT`: ```rust #[frame_support::pallet(dev_mode)] mod module_single_instance { #[pallet::composite_enum] pub enum HoldReason { ModuleSingleInstanceReason1, ModuleSingleInstanceReason2, } ... } #[frame_support::pallet(dev_mode)] mod module_multi_instance { #[pallet::composite_enum] pub enum HoldReason<I: 'static = ()> { ModuleMultiInstanceReason1, ModuleMultiInstanceReason2, ModuleMultiInstanceReason3, } ... } impl self::sp_api_hidden_includes_construct_runtime::hidden_include::traits::VariantCount for RuntimeHoldReason { const VARIANT_COUNT: u32 = 0 + module_single_instance::HoldReason::VARIANT_COUNT + module_multi_instance::HoldReason::<module_multi_instance::Instance1>::VARIANT_COUNT + module_multi_instance::HoldReason::<module_multi_instance::Instance2>::VARIANT_COUNT + module_multi_instance::HoldReason::<module_multi_instance::Instance3>::VARIANT_COUNT; } ``` In addition, `MaxHolds` is removed (as suggested [here](https://github.com/paritytech/polkadot-sdk/pull/2657#discussion_r1443324573)) from `pallet_balances`, and its `Holds` are now bounded to `RuntimeHoldReason::VARIANT_COUNT`. Therefore, there is no need to let the runtime specify `MaxHolds`. ## For reviewers Relevant changes can be found here: - `substrate/frame/support/procedural/src/lib.rs` - `substrate/frame/support/procedural/src/pallet/parse/composite.rs` - `substrate/frame/support/procedural/src/pallet/expand/composite.rs` - `substrate/frame/support/procedural/src/construct_runtime/expand/composite_helper.rs` - `substrate/frame/support/procedural/src/construct_runtime/expand/hold_reason.rs` - `substrate/frame/support/procedural/src/construct_runtime/expand/freeze_reason.rs` - `substrate/frame/support/src/traits/misc.rs` And the rest of the files is just about removed `MaxHolds` from `pallet_balances` ## Next steps Do the same for `MaxFreezes` https://github.com/paritytech/polkadot-sdk/issues/2997. --------- Co-authored-by: command-bot <> Co-authored-by:
-
Branislav Kontur authored
The `parachains-common` contains a lots of constants and type definitions which are used for `polkadot-sdk`'s testnet runtimes and also for `polkadot-fellows`'s production [SP runtimes](https://github.com/polkadot-fellows/runtimes/tree/main/system-parachains/constants). This PR cleans `parachains-common` module to contain only common and generic functionality. Testnet-specific constants have been moved to the separate module dedicated just for testnets: `polkadot-sdk/cumulus/parachains/runtimes/constants/` Part of: https://github.com/paritytech/polkadot-sdk/issues/3054 --------- Co-authored-by:
-
- Jan 30, 2024
-
-
Oliver Tale-Yazdi authored
Reverts paritytech/polkadot-sdk#2302.
🤦 ♂️ should have checked the migration CI first. We either need to reduce the `max_message_size` for the open HRMP channels on the failing chains or increase the `PageSize` of the XCMP queue. Both would be fine on a test-net, but i assume this will also fail before the next SP runtime upgrade so first need to think what best to do. AFAIK its not possible currently to change the `max_message_size` of an open HRMP channel.
-
- Jan 29, 2024
-
-
Oliver Tale-Yazdi authored
Remove `without_storage_info` from the XCMP queue pallet. Part of https://github.com/paritytech/polkadot-sdk/issues/323 Changes: - Limit the number of channels that can be suspended at the same time. - Limit the number of channels that can have messages or signals pending at the same time. A No-OP migration is put in place to ensure that all `BoundedVec`s still decode and not truncate after upgrade. The storage version is thereby bumped to 4 to have our tooling remind us to deploy that migration. --------- Signed-off-by:
-
- Jan 24, 2024
-
-
Just van Stam authored
Moved from: https://github.com/paritytech/polkadot/pull/6951 closes https://github.com/paritytech/polkadot-sdk/issues/490 - [x] update cumulus --- This PR introduces transactional processing of certain xcm instructions. For the list of instructions checkout https://github.com/paritytech/polkadot-sdk/issues/490. The transactional processing is implemented as an xcm-executor config item. The two implementations in this PR are `FrameTransactionalProcessor` and `()`. The `()` implementation does no transactional processing. Each implementation of the `ProcessTransaction` trait has an `IS_TRANSACTIONAL` const that tells the XCVM if transactional processing is actually implemented. If Transactional processing is implemented, changes to touched registers should also be rolled back to prevent inconsistencies. Note for reviewers: Check out the following safety assumption: https://github.com/paritytech/polkadot-sdk/pull/1222/files#diff-4effad7d8c1c9de19fd27e18661cbf2128c8718f3b2420a27d2f816e0749ea53R30 --------- Co-authored-by:
Keith Yeung <[email protected]> Co-authored-by:
-
- Jan 22, 2024
-
-
Marcin S. authored
The goal is to move all system chains on Rococo (+ other testnets?) to use async backing. Starting with `asset-hub-rococo` to get feedback, before I do the rest. ## Related Example: https://github.com/paritytech/polkadot-sdk/pull/1619/ Guide: https://github.com/w3f/polkadot-wiki/blob/master/docs/maintain/maintain-guides-async-backing.md --------- Co-authored-by:
-
joe petrowski authored
Clean up all the old syntax. --------- Co-authored-by: command-bot <> Co-authored-by:
Maksym H <[email protected]>
-
- Jan 19, 2024
-
-
Sergej Sakac authored
This PR introduces the following changes: - [x] Adds a `UniquesTransactor` to asset-hub-rococo - [x] Adds a `UniquesTransactor` to asset-hub-westend We can't add a transactor for `pallet-nfts` like we do for `pallet-uniques` because `pallet-nfts` uses `nonfungibles_v2::Mutate` instead of `nonfungibles::Mutate`, and making that work would be out of scope of this PR. With these modifications, reserve-based NFT cross-chain transfers can be performed on asset-hub. --------- Co-authored-by:
-
- Jan 17, 2024
-
-
Egor_P authored
This PR backports version bumps from release branch `release-polkadot-v1.6.0` back to `master` and also moved `prdoc` files related to the release to the appropriate folder
-
- Jan 16, 2024
-
-
Francisco Aguirre authored
# Note for reviewer Most changes are just syntax changes necessary for the new version. Most important files should be the ones under the `xcm` folder. # Description Added XCMv4. ## Removed `Multi` prefix The following types have been renamed: - MultiLocation -> Location - MultiAsset -> Asset - MultiAssets -> Assets - InteriorMultiLocation -> InteriorLocation - MultiAssetFilter -> AssetFilter - VersionedMultiAsset -> VersionedAsset - WildMultiAsset -> WildAsset - VersionedMultiLocation -> VersionedLocation In order to fix a name conflict, the `Assets` in `xcm-executor` were renamed to `HoldingAssets`, as they represent assets in holding. ## Removed `Abstract` asset id It was not being used anywhere and this simplifies the code. Now assets are just constructed as follows: ```rust let asset: Asset = (AssetId(Location::new(1, Here)), 100u128).into(); ``` No need for specifying `Concrete` anymore. ## Outcome is now a named fields struct Instead of ```rust pub enum Outcome { Complete(Weight), Incomplete(Weight, Error), Error(Error), } ``` we now have ```rust pub enum Outcome { Complete { used: Weight }, Incomplete { used: Weight, error: Error }, Error { error: Error }, } ``` ## Added Reanchorable trait Now both locations and assets implement this trait, making it easier to reanchor both. ## New syntax for building locations and junctions Now junctions are built using the following methods: ```rust let location = Location { parents: 1, interior: [Parachain(1000), PalletInstance(50), GeneralIndex(1984)].into() }; ``` or ```rust let location = Location::new(1, [Parachain(1000), PalletInstance(50), GeneralIndex(1984)]); ``` And they are matched like so: ```rust match location.unpack() { (1, [Parachain(id)]) => ... (0, Here) => ..., (1, [_]) => ..., } ``` This syntax is mandatory in v4, and has been also implemented for v2 and v3 for easier migration. This was needed to make all sizes smaller. # TODO - [x] Scaffold v4 - [x] Port github.com/paritytech/polkadot/pull/7236 - [x] Remove `Multi` prefix - [x] Remove `Abstract` asset id --------- Co-authored-by: command-bot <> Co-authored-by: -
Muharem Ismailov authored
Implements an XCM executor `WeightTrader`, facilitating fee payments in any asset that can be exchanged for a native asset. A few constraints need to be observed: - `buy_weight` and `refund` operations must be atomic, as another weight trader implementation might be attempted in case of failure. - swap credit must be utilized since there isn’t an account to which an asset of some class can be deposited with a guarantee to meet the existential deposit requirement. Also, operating with credits enhances the efficiency of the weight trader - https://github.com/paritytech/polkadot-sdk/pull/1677 related PRs: - (depends) https://github.com/paritytech/polkadot-sdk/pull/2031 - (depends) https://github.com/paritytech/polkadot-sdk/pull/1677 - (caused) https://github.com/paritytech/polkadot-sdk/pull/1847 - (caused) https://github.com/paritytech/polkadot-sdk/pull/1876 // DONE: impl `OnUnbalanced` for a `fungible/s` credit // DONE: make the trader free from a concept of a native currency and drop few fallible conversions. related issue - https://github.com/paritytech/polkadot-sdk/issues/1842 // DONE: tests --------- Co-authored-by:
-
- Dec 20, 2023
-
-
Muharem Ismailov authored
closes https://github.com/paritytech/polkadot-sdk/issues/1842 Decoupling Pallet from the Concept of Native Currency Currently, the pallet is intrinsically linked with the concept of native currency, requiring users to provide implementations of the `fungible::*` and `fungibles::*` traits to interact with native and non native assets. This incapsulates some non-related to the pallet complexity and makes it less adaptable in contexts where the native currency concept is absent. With this PR, the dependence on `fungible::*` for liquidity-supplying assets has been removed. Instead, the native and non-native currencies' handling is now overseen by a single type that implements the `fungibles::*` traits. To simplify this integration, types have been introduced to facilitate the creation of a union between `fungible::*` and `fungibles::*` implementations, producing a unified `fungibles::*` type. One of the reasons driving these changes is the ambition to create a more user-friendly API for the `SwapCredit` implementation. Given that it interacts with two distinct credit types from `fungible` and `fungibles`, a unified type was introduced. Clients now manage potential conversion failures for those credit types. In certain contexts, it's vital to guarantee that operations are fail-safe, like in this impl - [PR](https://github.com/paritytech/polkadot-sdk/pull/1845), place in [code](https://github.com/paritytech/polkadot-sdk/blob/20b85a5f /cumulus/primitives/utility/src/lib.rs#L429). Additional Updates: - abstracted the pool ID and its account derivation logic via trait bounds, along with common implementation offerings; - removed `inc_providers` on a pool creation for the pool account; - benchmarks: -- swap complexity is N, not const; -- removed `From<u128> + Into<u128>` bound from `T::Balance`; -- removed swap/liquidity/.. amount constants, resolve them dynamically based on pallet configuration; -- migrated to v2 API; - `OnUnbalanced` handler for the pool creation fee, replacing direct transfers to a specified account ID; - renamed `MultiAssetId` to `AssetKind` aligning with naming across frame crates; related PRs: - (depends) https://github.com/paritytech/polkadot-sdk/pull/1677 - (caused) https://github.com/paritytech/polkadot-sdk/pull/2033 - (caused) https://github.com/paritytech/polkadot-sdk/pull/1876 --------- Co-authored-by:
-
- Dec 19, 2023
-
-
Muharem Ismailov authored
Introduces a swap implementation that allows the exchange of a credit (aka Negative Imbalance) of one asset for a credit of another asset. This is particularly useful when a credit swap is required but may not have sufficient value to meet the ED constraint, hence cannot be deposited to temp account before. An example use case is when XCM fees are paid using an asset held in the XCM executor registry and has to be swapped for native currency. Additional Updates: - encapsulates the existing `Swap` trait impl within a transactional context, since partial storage mutation is possible when an error occurs; - supplied `Currency` and `Assets` impls must be implemented over the same `Balance` type, the `AssetBalance` generic type is dropped. This helps to avoid numerous type conversion and overflow cases. If those types are different it should be handled outside of the pallet; - `Box` asset kind on a pallet level, unbox on a runtime level - here [why](https://substrate.stackexchange.com/questions/10039/boxed-argument-of-a-dispatchable/10103#10103); - `path` uses `Vec` now, instead of `BoundedVec` since it is never used in PoV; - removes the `Transfer` event due to it's redundancy with the events emitted by `fungible/s` implementations; - modifies the `SwapExecuted` event type; related issue: - https://github.com/paritytech/polkadot-sdk/issues/105 related PRs: - (required for) https://github.com/paritytech/polkadot-sdk/pull/1845 - (caused) https://github.com/paritytech/polkadot-sdk/pull/1717 // DONE make the pallet work only with `fungibles` trait and make it free from the concept of a `native` asset - https://github.com/paritytech/polkadot-sdk/issues/1842 --------- Co-authored-by:
-
- Dec 14, 2023
-
-
Egor_P authored
This PR backports `transaction_version` bump from `1.5.0` release back to `master`
-
- Dec 12, 2023
-
-
Branislav Kontur authored
## Summary This pull request proposes a solution for improved control of the versioned XCM flow over the bridge (across different consensus chains) and resolves the situation where the sending chain/consensus has already migrated to a higher XCM version than the receiving chain/consensus. ## Problem/Motivation The current flow over the bridge involves a transfer from AssetHubRococo (AHR) to BridgeHubRococo (BHR) to BridgeHubWestend (BHW) and finally to AssetHubWestend (AHW), beginning with a reserve-backed transfer on AHR. In this process: 1. AHR sends XCM `ExportMessage` through `XcmpQueue`, incorporating XCM version checks using the `WrapVersion` feature, influenced by `pallet_xcm::SupportedVersion` (managed by `pallet_xcm::force_xcm_version` or version discovery). 2. BHR handles the `ExportMessage` instruction, utilizing the latest XCM version. The `HaulBlobExporter` converts the inner XCM to [`VersionedXcm::from`](https://github.com/paritytech/polkadot-sdk/blob/63ac2471/polkadot/xcm/xcm-builder/src/universal_exports.rs#L465-L467), also using the latest XCM version. However, challenges arise: - Incompatibility when BHW uses a different version than BHR. For instance, if BHR migrates to **XCMv4** while BHW remains on **XCMv3**, BHR's `VersionedXcm::from` uses `VersionedXcm::V4` variant, causing encoding issues for BHW. ``` /// Just a simulation of possible error, which could happen on BHW /// (this code is based on actual master without XCMv4) let encoded = hex_literal::hex!("0400"); println!("{:?}", VersionedXcm::<()>::decode(&mut &encoded[..])); Err(Error { cause: None, desc: "Could not decode `VersionedXcm`, variant doesn't exist" }) ``` - Similar compatibility issues exist between AHR and AHW. ## Solution This pull request introduces the following solutions: 1. **New trait `CheckVersion`** - added to the `xcm` module and exposing `pallet_xcm::SupportedVersion`. This enhancement allows checking the actual XCM version for desired destinations outside of the `pallet_xcm` module. 2. **Version Check in `HaulBlobExporter`** uses `CheckVersion` to check known/configured destination versions, ensuring compatibility. For example, in the scenario mentioned, BHR can store the version `3` for BHW. If BHR is on XCMv4, it will attempt to downgrade the message to version `3` instead of using the latest version `4`. 3. **Version Check in `pallet-xcm-bridge-hub-router`** - this check ensures compatibility with the real destination's XCM version, preventing the unnecessary sending of messages to the local bridge hub if versions are incompatible. These additions aim to improve the control and compatibility of XCM flows over the bridge and addressing issues related to version mismatches. ## Possible alternative solution _(More investigation is needed, and at the very least, it should extend to XCMv4/5. If this proves to be a viable option, I can open an RFC for XCM.)._ Add the `XcmVersion` attribute to the `ExportMessage` so that the sending chain can determine, based on what is stored in `pallet_xcm::SupportedVersion`, the version the destination is using. This way, we may not need to handle the version in `HaulBlobExporter`. ``` ExportMessage { network: NetworkId, destination: InteriorMultiLocation, xcm: Xcm<()> destination_xcm_version: Version, // <- new attritbute }, ``` ``` pub trait ExportXcm { fn validate( network: NetworkId, channel: u32, universal_source: &mut Option<InteriorMultiLocation>, destination: &mut Option<InteriorMultiLocation>, message: &mut Option<Xcm<()>>, destination_xcm_version: Version, , // <- new attritbute ) -> SendResult<Self::Ticket>; ``` ## Future Directions This PR does not fix version discovery over bridge, further investigation will be conducted here: https://github.com/paritytech/polkadot-sdk/issues/2417. ## TODO - [x] `pallet_xcm` mock for tests uses hard-coded XCM version `2` - change to 3 or lastest? - [x] fix `pallet-xcm-bridge-hub-router` - [x] fix HaulBlobExporter with version determination [here](https://github.com/paritytech/polkadot-sdk/blob/2183669d/polkadot/xcm/xcm-builder/src/universal_exports.rs#L465) - [x] add unit-tests to the runtimes - [x] run benchmarks for `ExportMessage` - [x] extend local run scripts about `force_xcm_version(dest, version)` - [ ] when merged, prepare governance calls for Rococo/Westend - [ ] add PRDoc Part of: https://github.com/paritytech/parity-bridges-common/issues/2719 --------- Co-authored-by: command-bot <>
-
Chevdor authored
This PR introduces a script and some templates to use the prdoc involved in a release and build: - the changelog - a simple draft of audience documentation Since the prdoc presence was enforced in the middle of the version 1.5.0, not all PRs did come with a `prdoc` file. This PR creates all the missing `prdoc` files with some minimum content allowing to properly generate the changelog. The generated content is **not** suitable for the audience documentation. The audience documentation will be possible with the next version, when all PR come with a proper `prdoc`. ## Assumptions - the prdoc files for release `vX.Y.Z` have been moved under `prdoc/X.Y.Z` - the changelog requires for now for the prdoc files to contain author + topic. Thos fields are optional. The build script can be called as: ``` VERSION=X.Y.Z ./scripts/release/build-changelogs.sh ``` Related: - #1408 --------- Co-authored-by:
-
- Dec 06, 2023
-
-
Adrian Catangiu authored
# Motivation (+testing) ### Enable easy `ForeignAssets` transfers using `pallet-xcm` We had just previously added capabilities to teleport fees during reserve-based transfers, but what about reserve-transferring fees when needing to teleport some non-fee asset? This PR aligns everything under either explicit reserve-transfer, explicit teleport, or this new flexible `transfer_assets()` which can mix and match as needed with fewer artificial constraints imposed to the user. This will enable, for example, a (non-system) parachain to teleport their `ForeignAssets` assets to AssetHub while using DOT to pay fees. (the assets are teleported - as foreign assets should from their owner chain - while DOT used for fees can only be reserve-based transferred between said parachain and AssetHub). Added `xcm-emulator` tests for this scenario ^. # Description Reverts `(limited_)reserve_transfer_assets` to only allow reserve-based transfers for all `assets` including fees. Similarly `(limited_)teleport_assets` only allows teleports for all `assets` including fees. For complex combinations of asset transfers where assets and fees may have different reserves or different reserve/teleport trust configurations, users can use the newly added `transfer_assets()` extrinsic which is more flexible in allowing more complex scenarios. `assets` (excluding `fees`) must have same reserve location or otherwise be teleportable to `dest`. No limitations imposed on `fees`. - for local reserve: transfer assets to sovereign account of destination chain and forward a notification XCM to `dest` to mint and deposit reserve-based assets to `beneficiary`. - for destination reserve: burn local assets and forward a notification to `dest` chain to withdraw the reserve assets from this chain's sovereign account and deposit them to `beneficiary`. - for remote reserve: burn local assets, forward XCM to reserve chain to move reserves from this chain's SA to `dest` chain's SA, and forward another XCM to `dest` to mint and deposit reserve-based assets to `beneficiary`. - for teleports: burn local assets and forward XCM to `dest` chain to mint/teleport assets and deposit them to `beneficiary`. ## Review notes Only around 500 lines are prod code (see `pallet_xcm/src/lib.rs`), the rest of the PR is new tests and improving existing tests. --------- Co-authored-by: command-bot <>
-
- Dec 05, 2023
-
-
gupnik authored
-
- Dec 04, 2023
-
-
Serban Iorga authored
Removes obsolete fields from the `QueueConfigData` structure. For the remaining fields, if they use the old defaults, we replace them with the new defaults. Resolves: https://github.com/paritytech/polkadot-sdk/issues/1795
-
- Nov 29, 2023
-
-
Liam Aharon authored
Westend SP dmp queue pallet removal is complete. <img width="1499" alt="Screenshot 2023-11-28 at 08 31 27" src="https://github.com/paritytech/polkadot-sdk/assets/16665596/906246fb-3de9-4133-a827-431636a097ad"> <img width="1499" alt="Screenshot 2023-11-28 at 08 32 08" src="https://github.com/paritytech/polkadot-sdk/assets/16665596/bde84891-b044-42c7-9a0b-59125cd24db1"> <img width="1499" alt="Screenshot 2023-11-28 at 08 31 45" src="https://github.com/paritytech/polkadot-sdk/assets/16665596/38337484-0856-45c0-b9ff-8c785bc3c0e3">
-
- Nov 27, 2023
-
-
Chevdor authored
## Overview This PR aligns the `spec_version` formatting to the [recent changes](https://github.com/polkadot-fellows/runtimes/pull/26/files#diff-efa4caeb17487ecb13d8f5eb7863c3241d84afa2e73fbf25909a2ca89df0f362R142) made for the Polkadot/Kusama runtimes. It also backports the latest version `v1.4.0` bumps as `1_004_000`. ## Details During the switch from `v0.9` to `v1.x`, the format of the `spec_version` was modified from: `(M)m_ppp` for a runtime considered on version `M.m.pp`. For instance `0.9.42` had a `spec_version` of `9420`. With the transition to `v1.x`, the format was changed to a bigger number (still `u32`) formatted as `MM_mm_ppp` where `1.2.3` would be stored as `01_02_003`. This PR aligns the format with that has been introduced in the fellowship repo: `MMM_mmm_ppp`. --------- Co-authored-by:
-
- Nov 16, 2023
-
-
joe petrowski authored
Missed in https://github.com/paritytech/polkadot-sdk/pull/1672
-
- Nov 14, 2023
-
-
Liam Aharon authored
Closes https://github.com/paritytech/polkadot-sdk/issues/2241 See issue comments for more details about this storage.
-
- Nov 13, 2023
-
-
Adrian Catangiu authored
## Motivation `pallet-xcm` is the main user-facing interface for XCM functionality, including assets manipulation functions like `teleportAssets()` and `reserve_transfer_assets()` calls. While `teleportAsset()` works both ways, `reserve_transfer_assets()` works only for sending reserve-based assets to a remote destination and beneficiary when the reserve is the _local chain_. ## Solution This PR enhances `pallet_xcm::(limited_)reserve_withdraw_assets` to support transfers when reserves are other chains. This will allow complete, **bi-directional** reserve-based asset transfers user stories using `pallet-xcm`. Enables following scenarios: - transferring assets with local reserve (was previously supported iff asset used as fee also had local reserve - now it works in all cases), - transferring assets with reserve on destination, - transferring assets with reserve on remote/third-party chain (iff assets and fees have same remote reserve), - transferring assets with reserve different than the reserve of the asset to be used as fees - meaning can be used to transfer random asset with local/dest reserve while using DOT for fees on all involved chains, even if DOT local/dest reserve doesn't match asset reserve, - transferring assets with any type of local/dest reserve while using fees which can be teleported between involved chains. All of the above is done by pallet inner logic without the user having to specify which scenario/reserves/teleports/etc. The correct scenario and corresponding XCM programs are identified, and respectively, built automatically based on runtime configuration of trusted teleporters and trusted reserves. #### Current limitations: - while `fees` and "non-fee" `assets` CAN have different reserves (or fees CAN be teleported), the remaining "non-fee" `assets` CANNOT, among themselves, have different reserve locations (this is also implicitly enforced by `MAX_ASSETS_FOR_TRANSFER=2`, but this can be safely increased in the future). - `fees` and "non-fee" `assets` CANNOT have **different remote** reserves (this could also be supported in the future, but adds even more complexity while possibly not being worth it - we'll see what the future holds). Fixes https://github.com/paritytech/polkadot-sdk/issues/1584 Fixes https://github.com/paritytech/polkadot-sdk/issues/2055 --------- Co-authored-by:
-
- Nov 02, 2023
-
-
Oliver Tale-Yazdi authored
(imported from https://github.com/paritytech/cumulus/pull/2157) ## Changes This MR refactores the XCMP, Parachains System and DMP pallets to use the [MessageQueue](https://github.com/paritytech/substrate/pull/12485) for delayed execution of incoming messages. The DMP pallet is entirely replaced by the MQ and thereby removed. This allows for PoV-bounded execution and resolves a number of issues that stem from the current work-around. All System Parachains adopt this change. The most important changes are in `primitives/core/src/lib.rs`, `parachains/common/src/process_xcm_message.rs`, `pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs` and the runtime configs. ### DMP Queue Pallet The pallet got removed and its logic refactored into parachain-system. Overweight message management can be done directly through the MQ pallet. Final undeployment migrations are provided by `cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that can be configured with an aux config trait like: ```rust parameter_types! { pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME; pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent; } impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime { type PalletName = DmpQueuePalletName; type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>; type DbWeight = <Runtime as frame_system::Config>::DbWeight; } // And adding them to your Migrations tuple: pub type Migrations = ( ... cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>, cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>, ); ``` ### XCMP Queue pallet Removed all dispatch queue functionality. Incoming XCMP messages are now either: Immediately handled if they are Signals, enqueued into the MQ pallet otherwise. New config items for the XCMP queue pallet: ```rust /// The actual queue implementation that retains the messages for later processing. type XcmpQueue: EnqueueMessage<ParaId>; /// How a XCM over HRMP from a sibling parachain should be processed. type XcmpProcessor: ProcessMessage<Origin = ParaId>; /// The maximal number of suspended XCMP channels at the same time. #[pallet::constant] type MaxInboundSuspended: Get<u32>; ``` How to configure those: ```rust // Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since // the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s. type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>; // Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched // with origin `Junction::Sibling(…)`. type XcmpProcessor = ProcessFromSibling< ProcessXcmMessage< AggregateMessageOrigin, xcm_executor::XcmExecutor<xcm_config::XcmConfig>, RuntimeCall, >, >; // Not really important what to choose here. Just something larger than the maximal number of channels. type MaxInboundSuspended = sp_core::ConstU32<1_000>; ``` The `InboundXcmpStatus` storage item was replaced by `InboundXcmpSuspended` since it now only tracks inbound queue suspension and no message indices anymore. Now only sends the most recent channel `Signals`, as all prio ones are out-dated anyway. ### Parachain System pallet For `DMP` messages instead of forwarding them to the `DMP` pallet, it now pushes them to the configured `DmpQueue`. The message processing which was triggered in `set_validation_data` is now being done by the MQ pallet `on_initialize`. XCMP messages are still handed off to the `XcmpMessageHandler` (XCMP-Queue pallet) - no change here. New config items for the parachain system pallet: ```rust /// Queues inbound downward messages for delayed processing. /// /// Analogous to the `XcmpQueue` of the XCMP queue pallet. type DmpQueue: EnqueueMessage<AggregateMessageOrigin>; ``` How to configure: ```rust /// Use the MQ pallet to store DMP messages for delayed processing. type DmpQueue = MessageQueue; ``` ## Message Flow The flow of messages on the parachain side. Messages come in from the left via the `Validation Data` and finally end up at the `Xcm Executor` on the right.  ## Further changes - Bumped the default suspension, drop and resume thresholds in `QueueConfigData::default()`. - `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when they would be a noop. - Properly validate the `QueueConfigData` before setting it. - Marked weight files as auto-generated so they wont auto-expand in the MR files view. - Move the `hypothetical` asserts to `frame_support` under the name `experimental_hypothetically` Questions: - [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in the runtimes? Not sure how to best fix. Just having them like this makes tests fail that rely on the real message processor when the feature is enabled. - [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler already takes 80% so I put it to 10% but that is quite low. TODO: - [x] Remove c&p code after https://github.com/paritytech/polkadot/pull/6271 - [x] Use `HandleMessage` once it is public in Substrate - [x] fix `runtime-benchmarks` feature https://github.com/paritytech/polkadot/pull/6966 - [x] Benchmarks - [x] Tests - [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended` - [x] Possibly cleanup Migrations (DMP+XCMP) - [x] optional: create `TransformProcessMessageOrigin` in Substrate and replace `ProcessFromSibling` - [ ] Rerun weights on ref HW --------- Signed-off-by:
-
- Nov 01, 2023
-
-
Branislav Kontur authored
## Summary Asset bridging support for AssetHub**Rococo** <-> AssetHub**Wococo** was added [here](https://github.com/paritytech/polkadot-sdk/pull/1215), so now we aim to bridge AssetHub**Rococo** and AssetHub**Westend**. (And perhaps retire AssetHubWococo and the Wococo chains). ## Solution **bridge-hub-westend-runtime** - added new runtime as a copy of `bridge-hub-rococo-runtime` - added support for bridging to `BridgeHubRococo` - added tests and benchmarks **bridge-hub-rococo-runtime** - added support for bridging to `BridgeHubWestend` - added tests and benchmarks - internal refactoring by splitting bridge configuration per network, e.g., `bridge_to_whatevernetwork_config.rs`. **asset-hub-rococo-runtime** - added support for asset bridging to `AssetHubWestend` (allows to receive only WNDs) - added new xcm router for `Westend` - added tests and benchmarks **asset-hub-westend-runtime** - added support for asset bridging to `AssetHubRococo` (allows to receive only ROCs) - added new xcm router for `Rococo` - added tests and benchmarks ## Deployment All changes will be deployed as a part of https://github.com/paritytech/polkadot-sdk/issues/1988. ## TODO - [x] benchmarks for all pallet instances - [x] integration tests - [x] local run scripts Relates to: https://github.com/paritytech/parity-bridges-common/issues/2602 Relates to: https://github.com/paritytech/polkadot-sdk/issues/1988 --------- Co-authored-by: command-bot <> Co-authored-by:
-
- Oct 26, 2023
-
-
Dastan authored
# Description - What does this PR do? While working with `pallet_nfts` through `nonfungibles_v2` traits `Inspect, Mutate`, I found out that once you have set the collection attribute with `<Nfts as Mutate>::set_collection_attribute()`, it's not possible to read it with `<Nfts as Inspect>::collection_attribute()` since they use different `namespace` values. When setting the attribute, `AttributeNamespace::Pallet` is used, while `AttributeNamespace::CollectionOwner` is used when reading. more context: https://github.com/freeverseio/laos/issues/7#issuecomment-1766137370 This PR makes `item` an optional parameter in `Inspect::system_attribute()`, to be able to read collection attributes. - Why are these changes needed? To be able to read collection level attributes when reading attributes of the collection. It will be possible to read collection attributes by passing `None` for `item` - How were these changes implemented and what do they affect? `NftsApi` is also affected and `NftsApi::system_attribute()` now accepts optional `item` parameter. ## Breaking change Because of the change in the `NftsApi::system_attribute()` method's `item` param, parachains who integrated the `NftsApi` need to update their API code and frontend integrations accordingly. AssetHubs are unaffected since the NftsApi wasn't released on those parachains yet.
-
- Oct 24, 2023
-
-
Kian Paimani authored
closes https://github.com/paritytech/polkadot-sdk/issues/1882 ## Breaking Changes This PR introduces a new item to `pallet_balances::Config`: ```diff trait Config { ++ type RuntimeFreezeReasons; } ``` This value is only used to check it against `type MaxFreeze`. A similar check has been added for `MaxHolds` against `RuntimeHoldReasons`, which is already given to `pallet_balances`. In all contexts, you should pass the real `RuntimeFreezeReasons` generated by `construct_runtime` to `type RuntimeFreezeReasons`. Passing `()` would also work, but it would imply that the runtime uses no freezes at all. --------- Signed-off-by:
-
- Oct 18, 2023
-
-
Keith Yeung authored
Combination of paritytech/polkadot#7005, its addon PR paritytech/polkadot#7585 and its companion paritytech/cumulus#2433. This PR introduces a new XcmFeesToAccount struct which implements the `FeeManager` trait, and assigns this struct as the `FeeManager` in the XCM config for all runtimes. The struct simply deposits all fees handled by the XCM executor to a specified account. In all runtimes, the specified account is configured as the treasury account. XCM __delivery__ fees are now being introduced (unless the root origin is sending a message to a system parachain on behalf of the originating chain). # Note for reviewers Most file changes are tests that had to be modified to account for the new fees. Main changes are in: - cumulus/pallets/xcmp-queue/src/lib.rs <- To make it track the delivery fees exponential factor - polkadot/xcm/xcm-builder/src/fee_handling.rs <- Added. Has the FeeManager implementation - All runtime xcm_config files <- To add the FeeManager to the XCM configuration # Important note After this change, instructions that create and send a new XCM (Query*, Report*, ExportMessage, InitiateReserveWithdraw, InitiateTeleport, DepositReserveAsset, TransferReserveAsset, LockAsset and RequestUnlock) will require the corresponding origin account in the origin register to pay for transport delivery fees, and the onward message will fail to be sent if the origin account does not have the required amount. This delivery fee is on top of what we already collect as tx fees in pallet-xcm and XCM BuyExecution fees! Wallet UIs that want to expose the new delivery fee can do so using the formula: ``` delivery_fee_factor * (base_fee + encoded_msg_len * per_byte_fee) ``` where the delivery fee factor can be obtained from the corresponding pallet based on which transport you are using (UMP, HRMP or bridges), the base fee is a constant, the encoded message length from the message itself and the per byte fee is the same as the configured per byte fee for txs (i.e. `TransactionByteFee`). --------- Co-authored-by:
-
- Oct 02, 2023
-
-
Liam Aharon authored
Makes SPs first class citizens along with the relay chains in the context of our CI runtime upgrade checks. ## Code changes - Sets missing current storage version in `uniques` pallet - Adds multisig V1 migration to run where it was missing - Removes executed migration whos pre/post hooks were failing from collectives runtime - Initializes storage versions for SP pallets added after genesis - Originally I was going to wait for https://github.com/paritytech/polkadot-sdk/pull/1297 to be merged so this wouldn't need to be done manually, but it doesn't seem like it'll be merged any time soon so I've decided to set them manually to unblock this ## CI changes - Removed dependency of `westend` runtime upgrades being complete prior to other ones running. I assume it is supposed to cache the `try-runtime` build for a performance benefit, but it seems it wasn't working. Maybe someone from the CI team can look into this or explain why it needs to be there? - Adds check-runtime-migration jobs for Parity asset-hubs, bridge-hubs and contract chains - Updated VARIABLES to accomodate the `kusama-runtime` package being renamed to `staging-kusama-runtime` in https://github.com/paritytech/polkadot-sdk/pull/1241 - Added `EXTRA_ARGS` variable to `check-runtime-migration`, and set `--no-weight-warnings` to the relay chain runtime upgrade checks (relay chains don't have weight restrictions).
-
