The `query_weight_to_asset_fee` function was trying to convert versions
by using `try_as`, this function [doesn't convert from a versioned to a
concrete
type](https://github.com/paritytech/polkadot-sdk/blob/0156ca8f/polkadot/xcm/src/lib.rs#L131).
This would cause all calls with a lower version to fail.

The correct function to use is the good old
[try_into](https://github.com/paritytech/polkadot-sdk/blob/0156ca8f

/polkadot/xcm/src/lib.rs#L184).
Now those calls work :)

---------

Co-authored-by: command-bot <>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: GitHub Action <action@github.com>

Step in https://github.com/paritytech/polkadot-sdk/issues/3268

This PR adds the ability for these pallets to specify their source of
the block number. This is useful when these pallets are migrated from
the relay chain to a parachain and vice versa.

This change is backwards compatible:
1. If the `BlockNumberProvider` continues to use the system pallet's
block number
2. When a pallet deployed on the relay chain is moved to a parachain,
but still uses the relay chain's block number

However, we would need migrations if the deployed pallets are upgraded
on an existing parachain, and the `BlockNumberProvider` uses the relay
chain block number.

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Added `ExecuteWithOrigin` instruction according to the old XCM RFC 38:
https://github.com/polkadot-fellows/xcm-format/pull/38.

This instruction allows you to descend or clear while going back again.

## TODO
- [x] Implementation
- [x] Unit tests
- [x] Integration tests
- [x] Benchmarks
- [x] PRDoc

## Future work

Modify `WithComputedOrigin` barrier to allow, for example, fees to be
paid with a descendant origin using this instruction.

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Andrii <ndk@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Joseph Zhao <65984904+programskillforverification@users.noreply.github.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: command-bot <>

# Context

This PR aims to introduce XCMv5, for now it's in progress and will be
updated over time.
This branch will serve as a milestone branch for merging in all features
we want to add to XCM, roughly outlined
[here](https://github.com/polkadot-fellows/xcm-format/issues/60).
More features could be added.

## TODO
- [x] Migrate foreign assets from v3 to v4
- [x] Setup v5 skeleton
- [x] Remove XCMv2
- [x] https://github.com/paritytech/polkadot-sdk/pull/5390
- [x] https://github.com/paritytech/polkadot-sdk/pull/5585
- [x] https://github.com/paritytech/polkadot-sdk/pull/5420
- [x] https://github.com/paritytech/polkadot-sdk/pull/5876
- [x] https://github.com/paritytech/polkadot-sdk/pull/5971
- [x] https://github.com/paritytech/polkadot-sdk/pull/6148
- [x] https://github.com/paritytech/polkadot-sdk/pull/6228

Fixes #3434 
Fixes https://github.com/paritytech/polkadot-sdk/issues/4190
Fixes https://github.com/paritytech/polkadot-sdk/issues/5209
Fixes https://github.com/paritytech/polkadot-sdk/issues/5241
Fixes https://github.com/paritytech/polkadot-sdk/issues/4284

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Andrii <ndk@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Joseph Zhao <65984904+programskillforverification@users.noreply.github.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Serban Iorga <serban@parity.io>

Remove `sp_runtime::RuntimeString` and replace with `Cow<'static, str>` or `String` depending on use case (#5693)

# Description

As described in https://github.com/paritytech/polkadot-sdk/issues/4001
`RuntimeVersion` was not encoded consistently using serde. Turned out it
was a remnant of old times and no longer actually needed. As such I
removed it completely in this PR and replaced with `Cow<'static, str>`
for spec/impl names and `String` for error cases.

Fixes https://github.com/paritytech/polkadot-sdk/issues/4001.

## Integration

For downstream projects the upgrade will primarily consist of following
two changes:
```diff
#[sp_version::runtime_version]
pub const VERSION: RuntimeVersion = RuntimeVersion {
-	spec_name: create_runtime_str!("statemine"),
-	impl_name: create_runtime_str!("statemine"),
+	spec_name: alloc::borrow::Cow::Borrowed("statemine"),
+	impl_name: alloc::borrow::Cow::Borrowed("statemine"),
```
```diff
		fn dispatch_benchmark(
			config: frame_benchmarking::BenchmarkConfig
-		) -> Result<Vec<frame_benchmarking::BenchmarkBatch>, sp_runtime::RuntimeString> {
+		) -> Result<Vec<frame_benchmarking::BenchmarkBatch>, alloc::string::String> {
```

SCALE encoding/decoding remains the same as before, but serde encoding
in runtime has changed from bytes to string (it was like this in `std`
environment already), which most projects shouldn't have issues with. I
consider the impact of serde encoding here low due to the type only
being used in runtime version struct and mostly limited to runtime
internals, where serde encoding/decoding of this data structure is quite
unlikely (though we did hit exactly this edge-case ourselves


).

## Review Notes

Most of the changes are trivial and mechanical, the only non-trivial
change is in
`substrate/primitives/version/proc-macro/src/decl_runtime_version.rs`
where macro call expectation in `sp_version::runtime_version`
implementation was replaced with function call expectation.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Guillaume Thiolliere <guillaume.thiolliere@parity.io>
Co-authored-by: Bastian Köcher <info@kchr.de>

This PR refactors `pallet-identity` to decouple usernames from
identities.

Main changes in this PR:
- Separate usernames from identities in storage, allowing for correct
deposit accounting
- Introduce the option for username authorities to put up a deposit to
issue a username
- Allow authorities to remove usernames by declaring the intent to do
so, then removing the username after the grace period expires
- Refactor the authority storage to be keyed by suffix rather than owner
account.
- Introduce the concept of a system provider for a username, different
from a governance allocation, allowing for usernames set by the system
and not a specific authority
- Implement multi-block migration to enable all of the changes described
above

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>

Changed returned type of API methods from `Result<bool,
xcm_runtime_apis::trusted_query::Error>` to a typed one `type
XcmTrustedQueryResult = Result<bool,
xcm_runtime_apis::trusted_query::Error>;`
Follow-up of
[PR-6039](https://github.com/paritytech/polkadot-sdk/pull/6039)

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

Original PR https://github.com/paritytech/polkadot-sdk/pull/2280
reverted in https://github.com/paritytech/polkadot-sdk/pull/3665

This PR reintroduces the reverted functionality with additional changes,
related effort
[here](https://github.com/paritytech/polkadot-sdk/pull/3623).
Description is copied over from the original PR

First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415)

Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for the latter. (Ultimately, the latter will be phased out as
a type of transaction, and Bare will only be used for Inherents.)

Types of extrinsic are now therefore:
- Bare (no hardcoded signature, no Extra data; used to be known as
"Unsigned")
- Bare transactions (deprecated): Gossiped, validated with
`ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
`TransactionExtension` (deprecated).
  - Inherents: Not gossiped, validated with `ProvideInherent`.
- Extended (Extra data): Gossiped, validated via `TransactionExtension`.
  - Signed transactions (with a hardcoded signature) in extrinsic v4.
- General transactions (without a hardcoded signature) in extrinsic v5.

`TransactionExtension` differs from `SignedExtension` because:
- A signature on the underlying transaction may validly not be present.
- It may alter the origin during validation.
- `pre_dispatch` is renamed to `prepare` and need not contain the checks
present in `validate`.
- `validate` and `prepare` is passed an `Origin` rather than a
`AccountId`.
- `validate` may pass arbitrary information into `prepare` via a new
user-specifiable type `Val`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`. It is encoded *for the entire transaction* and
passed in to each extension as a new argument to `validate`. This
facilitates the ability of extensions to acts as underlying crypto.

There is a new `DispatchTransaction` trait which contains only default
function impls and is impl'ed for any `TransactionExtension` impler. It
provides several utility functions which reduce some of the tedium from
using `TransactionExtension` (indeed, none of its regular functions
should now need to be called directly).

Three transaction version discriminator ("versions") are now permissible
(RFC [here](https://github.com/polkadot-fellows/RFCs/pull/84)) in
extrinsic version 5:
- 0b00000100 or 0b00000101: Bare (used to be called "Unsigned"):
contains Signature or Extra (extension data). After bare transactions
are no longer supported, this will strictly identify an Inherents only.
Available in both extrinsic versions 4 and 5.
- 0b10000100: Old-school "Signed" Transaction: contains Signature, Extra
(extension data) and an extension version byte, introduced as part of
[RFC99](https://github.com/polkadot-fellows/RFCs/blob/main/text/0099-transaction-extension-version.md).
Still available as part of extrinsic v4.
- 0b01000101: New-school "General" Transaction: contains Extra
(extension data) and an extension version byte, as per RFC99, but no
Signature. Only available in extrinsic v5.

For the New-school General Transaction, it becomes trivial for authors
to publish extensions to the mechanism for authorizing an Origin, e.g.
through new kinds of key-signing schemes, ZK proofs, pallet state,
mutations over pre-authenticated origins or any combination of the
above.

`UncheckedExtrinsic` still maintains encode/decode backwards
compatibility with extrinsic version 4, where the first byte was encoded
as:
- 0b00000100 - Unsigned transactions
- 0b10000100 - Old-school Signed transactions, without the extension
version byte

Now, `UncheckedExtrinsic` contains a `Preamble` and the actual call. The
`Preamble` describes the type of extrinsic as follows:
```rust
/// A "header" for extrinsics leading up to the call itself. Determines the type of extrinsic and
/// holds any necessary specialized data.
#[derive(Eq, PartialEq, Clone)]
pub enum Preamble<Address, Signature, Extension> {
	/// An extrinsic without a signature or any extension. This means it's either an inherent or
	/// an old-school "Unsigned" (we don't use that terminology any more since it's confusable with
	/// the general transaction which is without a signature but does have an extension).
	///
	/// NOTE: In the future, once we remove `ValidateUnsigned`, this will only serve Inherent
	/// extrinsics and thus can be renamed to `Inherent`.
	Bare(ExtrinsicVersion),
	/// An old-school transaction extrinsic which includes a signature of some hard-coded crypto.
	/// Available only on extrinsic version 4.
	Signed(Address, Signature, ExtensionVersion, Extension),
	/// A new-school transaction extrinsic which does not include a signature by default. The
	/// origin authorization, through signatures or other means, is performed by the transaction
	/// extension in this extrinsic. Available starting with extrinsic version 5.
	General(ExtensionVersion, Extension),
}
```

## Code Migration

### NOW: Getting it to build

Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
accompanied by renaming your aggregate type in line with the new
terminology. E.g. Before:

```rust
/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
	/* snip */
	MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
```

After:

```rust
/// The extension to the basic transaction logic.
pub type TxExtension = (
	/* snip */
	AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
```

You'll also need to alter any transaction building logic to add a
`.into()` to make the conversion happen. E.g. Before:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let extra: SignedExtra = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */),
	);
	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		extra,
	)
}
```

After:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let tx_ext: TxExtension = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */).into(),
	);
	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		tx_ext,
	)
}
```

### SOON: Migrating to `TransactionExtension`

Most `SignedExtension`s can be trivially converted to become a
`TransactionExtension`. There are a few things to know.

- Instead of a single trait like `SignedExtension`, you should now
implement two traits individually: `TransactionExtensionBase` and
`TransactionExtension`.
- Weights are now a thing and must be provided via the new function `fn
weight`.

#### `TransactionExtensionBase`

This trait takes care of anything which is not dependent on types
specific to your runtime, most notably `Call`.

- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`.
- Weight must be returned by implementing the `weight` function. If your
extension is associated with a pallet, you'll probably want to do this
via the pallet's existing benchmarking infrastructure.

#### `TransactionExtension`

Generally:
- `pre_dispatch` is now `prepare` and you *should not reexecute the
`validate` functionality in there*!
- You don't get an account ID any more; you get an origin instead. If
you need to presume an account ID, then you can use the trait function
`AsSystemOriginSigner::as_system_origin_signer`.
- You get an additional ticket, similar to `Pre`, called `Val`. This
defines data which is passed from `validate` into `prepare`. This is
important since you should not be duplicating logic from `validate` to
`prepare`, you need a way of passing your working from the former into
the latter. This is it.
- This trait takes a `Call` type parameter. `Call` is the runtime call
type which used to be an associated type; you can just move it to become
a type parameter for your trait impl.
- There's no `AccountId` associated type any more. Just remove it.

Regarding `validate`:
- You get three new parameters in `validate`; all can be ignored when
migrating from `SignedExtension`.
- `validate` returns a tuple on success; the second item in the tuple is
the new ticket type `Self::Val` which gets passed in to `prepare`. If
you use any information extracted during `validate` (off-chain and
on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
pass it through with this. For the tuple's last item, just return the
`origin` argument.

Regarding `prepare`:
- This is renamed from `pre_dispatch`, but there is one change:
- FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
`validate`!!
- (This is different to `SignedExtension` which was required to run the
same checks in `pre_dispatch` as in `validate`.)

Regarding `post_dispatch`:
- Since there are no unsigned transactions handled by
`TransactionExtension`, `Pre` is always defined, so the first parameter
is `Self::Pre` rather than `Option<Self::Pre>`.

If you make use of `SignedExtension::validate_unsigned` or
`SignedExtension::pre_dispatch_unsigned`, then:
- Just use the regular versions of these functions instead.
- Have your logic execute in the case that the `origin` is `None`.
- Ensure your transaction creation logic creates a General Transaction
rather than a Bare Transaction; this means having to include all
`TransactionExtension`s' data.
- `ValidateUnsigned` can still be used (for now) if you need to be able
to construct transactions which contain none of the extension data,
however these will be phased out in stage 2 of the Transactions Horizon,
so you should consider moving to an extension-centric design.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>

Implemented is_trusted_reserve and is_trusted_teleporter API methods. 
Tested them with regular and chopstick tests.
Fixes #97

---------

Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>

# Description
## What?
Make it possible for other pallets to implement their own logic when a
slash on a balance occurs.

## Why?
In the [introduction of
holds](https://github.com/paritytech/substrate/pull/12951) @gavofyork
said:
> Since Holds are designed to be infallibly slashed, this means that any
logic using a Freeze must handle the possibility of the frozen amount
being reduced, potentially to zero. A permissionless function should be
provided in order to allow bookkeeping to be updated in this instance.

At Polimec we needed to find a way to reduce the vesting schedules of
our users after a slash was made, and after talking to @Kianenigma

 at
the Web3Summit, we realized there was no easy way to implement this with
the current traits, so we came up with this solution.



## How?
- First we abstract the `done_slash` function of holds::Balanced to it's
own trait that any pallet can implement.
- Then we add a config type in pallet-balances that accepts a callback
tuple of all the pallets that implement this trait.
- Finally implement done_slash for pallet-balances such that it calls
the config type.

## Integration
The default implementation of done_slash is still an empty function, and
the new config type of pallet-balances can be set to an empty tuple, so
nothing changes by default.

## Review Notes
- I suggest to focus on the first commit which contains the main logic
changes.
- I also have a working implementation of done_slash for pallet_vesting,
should I add it to this PR?
- If I run `cargo +nightly fmt --all` then I get changes to a lot of
unrelated crates, so not sure if I should run it to avoid the fmt
failure of the CI
- Should I hunt down references to fungible/fungibles documentation and
update it accordingly?

**Polkadot address:** `15fj1UhQp8Xes7y7LSmDYTy349mXvUwrbNmLaP5tQKBxsQY1`

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>

Added `HashedDescription<AccountId,
DescribeFamily<DescribeAllTerminal>>` foreign locations to local
accounts converter to all the parachains.

---------

Co-authored-by: command-bot <>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

This PR backports regular version bumps and prdocs reordering from the
`stable2409` release branch to `master`

---------

Co-authored-by: Morgan Adamiec <morgan@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>

# MBM try-runtime support

This MR adds support to the try-runtime trait such that the
try-runtime-CLI will be able to support MBM testing
[here](https://github.com/paritytech/try-runtime-cli/pull/90). It mainly
adds two feature-gated hooks to the `SteppedMigration` hook to
facilitate testing. These hooks are named `pre_upgrade` and
`post_upgrade` and have the same signature and implications as for
single-block migrations.

## Integration

To make use of this in your Multi-Block-Migration, just implement the
two new hooks and test pre- and post-conditions in them:

```rust
#[cfg(feature = "try-runtime")]
fn pre_upgrade() -> Result<Vec<u8>, frame_support::sp_runtime::TryRuntimeError> {
	// ...
}

#[cfg(feature = "try-runtime")]
fn post_upgrade(prev: Vec<u8>) -> Result<(), frame_support::sp_runtime::TryRuntimeError> {
    // ...
}
```

You may return an error or panic in these functions to indicate failure.
This will then show up in the try-runtime-CLI and can be used in CI for
testing.

Changes:
- Adds `try-runtime` gated methods `pre_upgrade` and `post_upgrade` on
`SteppedMigration`
- Adds `try-runtime` gated methods `nth_pre_upgrade` and
`nth_post_upgrade` on `SteppedMigrations`
- Modifies `pallet_migrations` implementation to run pre_upgrade and
post_upgrade steps at the appropriate times, and panic in the event of
migration failure.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: claravanstaden <claravanstaden64@gmail.com>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>
Co-authored-by: georgepisaltu <george.pisaltu@parity.io>

Partially implements
https://github.com/paritytech/polkadot-sdk/issues/5048

- adds a core selection runtime API to cumulus and a generic way of
configuring it for a parachain
- modifies the slot based collator to utilise the claim queue and the
generic core selection

What's left to be implemented (in a follow-up PR):
- add the UMP signal for core selection into the parachain-system pallet

View the RFC for more context:
https://github.com/polkadot-fellows/RFCs/pull/103

---------

Co-authored-by: command-bot <>

- This will ensure that a correct product name
(Polkadot/Cumulus/Substrate) is referenced in license headers.
- Closes https://github.com/paritytech/license-scanner/issues/49

Proxies are possible in the runtimes for Kusama and Polkadot but this
functionality was not previously available on testnets.

Closes #5453.

Proxies can now be used on `coretime-rococo`, `coretime-westend`,
`people-rococo` and `people-westend` in the same way as they can be on
Kusama and Polkadot chains.

The exact same proxies are configured as the production runtimes for the
respective system parachains.

Update `RuntimeVerison` type and use `system_version` to derive extrinsics root `StateVersion` instead of `V0` (#4257)

This PR 
- Renames `RuntimeVersion::state_version` to `system_version`
- Uses `Runtime::system_version` to derive extrinsics root
`StateVersion` instead of default `StateVersion::V0`

This PR should not be breaking any existing chains so long as they use
same `RuntimeVersion::state_version` for `Runtime::system_version`

Using `RuntimeVersion::system_version = 2` will make the extrinsics root
to use `StateVersion::V1` instead of `V0`

RFC for this change - https://github.com/polkadot-fellows/RFCs/pull/42

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>

This PR adds possibility to set BUILD_OPTIONS to the "Srtool Build" step
in the release pipeline while building runtimes.

Colses: https://github.com/paritytech/release-engineering/issues/213

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: EgorPopelyaev <EgorPopelyaev@users.noreply.github.com>

Closes #4242

XCM programs that deposit assets to some new (empty) account will now
succeed if at least one of the deposited assets satisfies ED. Before
this change, the requirement was that the _first_ asset had to satisfy
ED, but assets order can be changed during reanchoring so it is not
reliable.

With this PR, ordering doesn't matter, any one(s) of them can satisfy ED
for the whole deposit to work.

Kusama address: FkB6QEo8VnV3oifugNj5NeVG3Mvq1zFbrUu4P5YwRoe5mQN

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: command-bot <>

This PR backports regular version bumps and reorganisation of the
`prdoc` files from the stable release branch back to master

Fixes #4960 

Configuring `FeeManager` enforces the boundary `Into<[u8; 32]>` for the
`AccountId` type.

Here is how it works currently: 

Configuration:
```rust
    type FeeManager = XcmFeeManagerFromComponents<
        IsChildSystemParachain<primitives::Id>,
        XcmFeeToAccount<Self::AssetTransactor, AccountId, TreasuryAccount>,
    >;
```

`XcmToFeeAccount` struct:
```rust
/// A `HandleFee` implementation that simply deposits the fees into a specific on-chain
/// `ReceiverAccount`.
///
/// It reuses the `AssetTransactor` configured on the XCM executor to deposit fee assets. If
/// the `AssetTransactor` returns an error while calling `deposit_asset`, then a warning will be
/// logged and the fee burned.
pub struct XcmFeeToAccount<AssetTransactor, AccountId, ReceiverAccount>(
	PhantomData<(AssetTransactor, AccountId, ReceiverAccount)>,
);

impl<
		AssetTransactor: TransactAsset,
		AccountId: Clone + Into<[u8; 32]>,
		ReceiverAccount: Get<AccountId>,
	> HandleFee for XcmFeeToAccount<AssetTransactor, AccountId, ReceiverAccount>
{
	fn handle_fee(fee: Assets, context: Option<&XcmContext>, _reason: FeeReason) -> Assets {
		deposit_or_burn_fee::<AssetTransactor, _>(fee, context, ReceiverAccount::get());

		Assets::new()
	}
}
```

`deposit_or_burn_fee()` function:
```rust
/// Try to deposit the given fee in the specified account.
/// Burns the fee in case of a failure.
pub fn deposit_or_burn_fee<AssetTransactor: TransactAsset, AccountId: Clone + Into<[u8; 32]>>(
	fee: Assets,
	context: Option<&XcmContext>,
	receiver: AccountId,
) {
	let dest = AccountId32 { network: None, id: receiver.into() }.into();
	for asset in fee.into_inner() {
		if let Err(e) = AssetTransactor::deposit_asset(&asset, &dest, context) {
			log::trace!(
				target: "xcm::fees",
				"`AssetTransactor::deposit_asset` returned error: {:?}. Burning fee: {:?}. \
				They might be burned.",
				e, asset,
			);
		}
	}
}
```

---

In order to use **another** `AccountId` type (for example, 20 byte
addresses for compatibility with Ethereum or Bitcoin), one has to
duplicate the code as the following (roughly changing every `32` to
`20`):
```rust
/// A `HandleFee` implementation that simply deposits the fees into a specific on-chain
/// `ReceiverAccount`.
///
/// It reuses the `AssetTransactor` configured on the XCM executor to deposit fee assets. If
/// the `AssetTransactor` returns an error while calling `deposit_asset`, then a warning will be
/// logged and the fee burned.
pub struct XcmFeeToAccount<AssetTransactor, AccountId, ReceiverAccount>(
    PhantomData<(AssetTransactor, AccountId, ReceiverAccount)>,
);
impl<
        AssetTransactor: TransactAsset,
        AccountId: Clone + Into<[u8; 20]>,
        ReceiverAccount: Get<AccountId>,
    > HandleFee for XcmFeeToAccount<AssetTransactor, AccountId, ReceiverAccount>
{
    fn handle_fee(fee: XcmAssets, context: Option<&XcmContext>, _reason: FeeReason) -> XcmAssets {
        deposit_or_burn_fee::<AssetTransactor, _>(fee, context, ReceiverAccount::get());

        XcmAssets::new()
    }
}

pub fn deposit_or_burn_fee<AssetTransactor: TransactAsset, AccountId: Clone + Into<[u8; 20]>>(
    fee: XcmAssets,
    context: Option<&XcmContext>,
    receiver: AccountId,
) {
    let dest = AccountKey20 { network: None, key: receiver.into() }.into();
    for asset in fee.into_inner() {
        if let Err(e) = AssetTransactor::deposit_asset(&asset, &dest, context) {
            log::trace!(
                target: "xcm::fees",
                "`AssetTransactor::deposit_asset` returned error: {:?}. Burning fee: {:?}. \
                They might be burned.",
                e, asset,
            );
        }
    }
}
```

---

This results in code duplication, which can be avoided simply by
relaxing the trait enforced by `XcmFeeToAccount`.

In this PR, I propose to introduce a new trait called `IntoLocation` to
be able to express both `Into<[u8; 32]>` and `Into<[u8; 20]>` should be
accepted (and every other `AccountId` type as long as they implement
this trait).

Currently, `deposit_or_burn_fee()` function converts the `receiver:
AccountId` to a location. I think converting an account to `Location`
should not be the responsibility of `deposit_or_burn_fee()` function.

This trait also decouples the conversion of `AccountId` to `Location`,
from `deposit_or_burn_fee()` function. And exposes `IntoLocation` trait.
Thus, allowing everyone to come up with their `AccountId` type and make
it compatible for configuring `FeeManager`.

---

Note 1: if there is a better file/location to put `IntoLocation`, I'm
all ears

Note 2: making `deposit_or_burn_fee` or `XcmToFeeAccount` generic was
not possible from what I understood, due to Rust currently do not
support a way to express the generic should implement either `trait A`
or `trait B` (since the compiler cannot guarantee they won't overlap).
In this case, they are `Into<[u8; 32]>` and `Into<[u8; 20]>`.
See [this](https://github.com/rust-lang/rust/issues/20400) and
[this](https://github.com/rust-lang/rfcs/pull/1672#issuecomment-262152934).

Note 3: I should also submit a PR to `frontier` that implements
`IntoLocation` for `AccountId20` if this PR gets accepted.


### Summary 
this new trait:
- decouples the conversion of `AccountId` to `Location`, from
`deposit_or_burn_fee()` function
- makes `XcmFeeToAccount` accept every possible `AccountId` type as long
as they they implement `IntoLocation`
- backwards compatible
- keeps the API simple and clean while making it less restrictive


@franciscoaguirre and @gupnik

 are already aware of the issue, so tagging
them here for visibility.

---------

Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: command-bot <>

This should remove nearly all usage of `sp-std` except:
- bridge and bridge-hubs
- a few of frames re-export `sp-std`, keep them for now
- there is a usage of `sp_std::Writer`, I don't have an idea how to move
it

Please review proc-macro carefully. I'm not sure I'm doing it the right
way.

Note: need `/bot fmt`

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>

This PR backports regular version bumps and prdocs reordering from the
1.14.0 release branch to master

Closes: https://github.com/paritytech/polkadot-sdk/issues/4298

This PR also merges `xcm-fee-payment-runtime-api` module to the
`xcm-runtime-api`.


## TODO

- [x] rename `convert` to `convert_location` and add new one
`convert_account` (opposite direction)
- [x] add to the all testnet runtimes
- [x] check polkadot-js if supports that automatically or if needs to be
added manually https://github.com/polkadot-js/api/pull/5917
- [ ] backport/patch for fellows and release (asap)

## Open questions
- [x] should we merge `xcm-runtime-api` and
`xcm-fee-payment-runtime-api` to the one module `xcm-runtime-api` ?

## Usage
Input:
 - `location:  VersionedLocation`
 
Output:
 - account_id bytes


![image](https://github.com/paritytech/polkadot-sdk/assets/8159517/4607b15a-77d2-462b-806c-606107776c0d)

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

After preparing in https://github.com/paritytech/polkadot-sdk/pull/4633,
we can lift also all internal dependencies up to the workspace.

This does not actually change anything, but uses `workspace = true` for
all dependencies. You can check it with:
```bash
git checkout -q $(git merge-base oty-lift-all-deps origin/master)
cargo tree -e features > master.out

git checkout -q oty-lift-all-deps
cargo tree -e features > new.out
diff master.out new.out
```

It did not yet lift 100% of dependencies, some inside of `target.*` or
some that had conflicting aliases introduced recently. But i will do
these together in a follow-up with CI checks.

Can be reproduced with [zepter](https://github.com/ggwpez/zepter/):
`zepter transpose d lift-to-workspace "regex:.*" --version-resolver
highest --skip-package "polkadot-sdk" --ignore-errors --fix`.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

This PR backports regular version bumps and prdocs reordering from the
release branch back to master

Depends on https://github.com/paritytech/polkadot-sdk/pull/4621.

Implemented the
[`XcmPaymentApi`](https://github.com/paritytech/polkadot-sdk/pull/3607)
and [`DryRunApi`](https://github.com/paritytech/polkadot-sdk/pull/3872)
on all system parachains.

More scenarios can be tested on both rococo and westend if all system
parachains implement this APIs.
The objective is for all XCM-enabled runtimes to implement them.
After demonstrating fee estimation in a UI on the testnets, come the
fellowship runtimes.

Step towards https://github.com/paritytech/polkadot-sdk/issues/690.

This PR backports version bumps and reorganisation of the prdoc files
from the `1.12.0` release branch back to `master`

Closes #4366

Re-applying #2302 after increasing the `MaxPageSize`.  

Remove `without_storage_info` from the XCMP queue pallet. Part of
https://github.com/paritytech/polkadot-sdk/issues/323

Changes:
- Limit the number of messages and signals a HRMP channel can have at
most.
- Limit the number of HRML channels.

A No-OP migration is put in place to ensure that all `BoundedVec`s still
decode and not truncate after upgrade. The storage version is thereby
bumped to 5 to have our tooling remind us to deploy that migration.

## Integration

If you see this error in your try-runtime-cli:  
```pre
Max message size for channel is too large. This means that the V5 migration can be front-run and an
attacker could place a large message just right before the migration to make other messages un-decodable.
Please either increase `MaxPageSize` or decrease the `max_message_size` for this channel. Channel max:
102400, MaxPageSize: 65535
```

Then increase the `MaxPageSize` of the `cumulus_pallet_xcmp...

This bump of versions:

https://github.com/paritytech/polkadot-sdk/pull/4409/files#diff-13ee4b2252c9e516a0547f2891aa2105c3ca71c6d7a1e682c69be97998dfc87eR11936

reintroduced a dependency to proc-macro-crate 2.0.0 which is suffering
from: https://github.com/bkchr/proc-macro-crate/pull/42 this, so bump
parity-scale-codec to a newer version to eliminate the bad
proc-macro-crate 2.0.0 dependency.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: command-bot <>

Adds an additional extrinsic call to the `pallet-balances` to _burn_
tokens.
Depending on the `keep_alive` flag, the call might or might not reap the
account.

Required modification of the _fungible's_ `Mutate` trait, `burn_from`
function to allow the `Preservation` argument.

**TODO**
- [x] run benchmarks & update weights
- [x] make sure prdoc is required & properly formatted

Related issue: https://github.com/paritytech/polkadot-sdk/issues/3943

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>

# Context

Estimating fees for XCM execution and sending has been an area with bad
UX.
The addition of the
[XcmPaymentApi](https://github.com/paritytech/polkadot-sdk/pull/3607)
exposed the necessary components to be able to estimate XCM fees
correctly, however, that was not the full story.
The `XcmPaymentApi` works for estimating fees only if you know the
specific XCM you want to execute or send.
This is necessary but most UIs want to estimate the fees for extrinsics,
they don't necessarily know the XCM program that's executed by them.

# Main addition

A new runtime API is introduced, the `XcmDryRunApi`, that given an
extrinsic, or an XCM program, returns its effects:
- Execution result
- Local XCM (in the case of an extrinsic)
- Forwarded XCMs
- List of events

This API can be used on its own for dry-running purposes, for
double-checking or testing, but it mainly shines when used in
conjunction with the `XcmPaymentApi`.
UIs can use these two APIs to estimate transfers.

# How it works

New tests are added to exemplify how to incorporate both APIs.
There's a mock test just to make sure everything works under
`xcm-fee-payment-runtime-api`.
There's a real-world test using Westend and AssetHubWestend under
`cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`.
Added both a test for a simple teleport between chains and a reserve
transfer asset between two parachains going through a reserve.

The steps to follow:
- Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and
forwarded messages
- For each forwarded message
- Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery
fees
- Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the
remote execution weight
- (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE
DESTINATION to know on which assets the execution fees can be paid
- Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to
convert weight to the actual remote execution fees
- Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new
message will be forwarded, if so, continue

# Dear reviewer

The changes in this PR are grouped as follows, and in order of
importance:
- Addition of new runtime API
- Definition, mock and simple tests:
polkadot/xcm/xcm-fee-payment-runtime-api/*
- Implemented on Westend, Asset Hub Westend and Penpal, will implement
on every runtime in a following PR
- Addition of a new config item to the XCM executor for recording xcms
about to be executed
  - Definition: polkadot/xcm/xcm-executor/*
  - Implementation: polkadot/xcm/pallet-xcm/*
- had to update all runtime xcm_config.rs files with `type XcmRecorder =
XcmPallet;`
- Addition of a new trait for inspecting the messages in queues
  - Definition: polkadot/xcm/xcm-builder/src/routing.rs
  - Implemented it on all routers:
    - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs
- ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on
implementation in cumulus/pallets/parachain-system/src/lib.rs)
    - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs
    - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs
- More complicated and useful tests:
-
cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs

## Next steps

With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have
the new API.
UIs can test on these runtimes to create better experiences around
cross-chain operations.

Next:
- Add XcmDryRunApi to all system parachains
- Integrate xcm fee estimation in all emulated tests
- Get this on the fellowship runtimes

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

This PR backports version bumps and reorganization of the `prdocs` from
`1.11.0` release branch back to master

Revert "pallet-xcm: Deprecate `execute` and `send` in favor of
`execute_blob` and `send_blob` (#3749)"

This reverts commit feee773d

.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Javier Bullrich <javier@bullrich.dev>

This PR:
- introduces `AllowHrmpNotificationsFromRelayChain` barrier for allowing
HRMP notifications just from the relay chain (to fulfill safety
assumptions -
[see](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/src/v4/mod.rs#L532))
- sets it up for all testnet SP parachains

Continuation of: https://github.com/paritytech/polkadot-sdk/pull/3696

Fixes https://github.com/paritytech/polkadot-sdk/issues/4206

In #1340 one of the storage types was changed from `Candidates` to
`CandidateList`. Since the actual key includes the hash of this value,
all of the candidates stored here are (a) "missing" and (b) unable to
unreserve their candidacy bond.

This migration kills the storage values and refunds the deposit held for
each candidate.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: georgepisaltu <george.pisaltu@parity.io>

Weights::v3 also accounts for PoV weight so we no longer need the
SafeCallFilter. All calls are allowed as long as they "fit in the
block".

This PR backports `spec_version`, `node_version` bumps and reordering of
the prdocs from the 1.10.0 release branch

We are exploring [allowing this for
Kusama](https://github.com/polkadot-fellows/runtimes/pull/261) as well,
disallowing on test chains seems unnecessarily limiting.