Implemented is_trusted_reserve and is_trusted_teleporter API methods. 
Tested them with regular and chopstick tests.
Fixes #97

---------

Co-authored-by: Francisco Aguirre <[email protected]>

Closes #5705

---------

Co-authored-by: Iulian Barbu <[email protected]>
Co-authored-by: Michal Kucharczyk <[email protected]>

Added error logs in pallet-xcm to help in debugging, fixes #2408 

## TODO

- [x] change `log::error` to `tracing::error` format for `xcm-executor`
- [x] check existing logs, e.g. this one can be extended with more info
`tracing::error!(target: "xcm::reanchor", ?error, "Failed reanchoring
with error");`
- [x] use `tracing` instead of `log` for `pallet-xcm/src/lib.rs`

---------

Co-authored-by: Ayevbeosa Iyamu <[email protected]>
Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

# Description

This PR addresses
https://github.com/paritytech/polkadot-sdk/issues/5878.

After dry running an xcm on asset hub, we had redundant xcms showing up
in the `forwarded_xcms` field of the dry run effects returned.
These were caused by two things:
- The `UpwardMessageSender` router always added an element even if there
were no messages.
- The two routers on asset hub westend related to bridging (to rococo
and sepolia) getting the message from their queues when their queues is
actually the same xcmp queue that was already contemplated.

In order to fix this, we check for no messages in UMP and clear the
implementation of `InspectMessageQueues` for these bridging routers.
Keep in mind that the bridged message is still sent, as normal via the
xcmp-queue to Bridge Hub.
To keep on dry-running the journey of the message, the next hop to
dry-run is Bridge Hub.
That'll be tackled in a different PR.

Added a test in `bridge-hub-westend-integration-tests` and
`bridge-hub-rococo-integration-tests` that show that dry-running a
transfer across the bridge from asset hub results in one and only one
message sent to bridge hub.

## TODO
- [x] Functionality
- [x] Test

---------

Co-authored-by: command-bot <>

# Description
## What?
Make it possible for other pallets to implement their own logic when a
slash on a balance occurs.

## Why?
In the [introduction of
holds](https://github.com/paritytech/substrate/pull/12951) @gavofyork
said:
> Since Holds are designed to be infallibly slashed, this means that any
logic using a Freeze must handle the possibility of the frozen amount
being reduced, potentially to zero. A permissionless function should be
provided in order to allow bookkeeping to be updated in this instance.

At Polimec we needed to find a way to reduce the vesting schedules of
our users after a slash was made, and after talking to @Kianenigma

 at
the Web3Summit, we realized there was no easy way to implement this with
the current traits, so we came up with this solution.



## How?
- First we abstract the `done_slash` function of holds::Balanced to it's
own trait that any pallet can implement.
- Then we add a config type in pallet-balances that accepts a callback
tuple of all the pallets that implement this trait.
- Finally implement done_slash for pallet-balances such that it calls
the config type.

## Integration
The default implementation of done_slash is still an empty function, and
the new config type of pallet-balances can be set to an empty tuple, so
nothing changes by default.

## Review Notes
- I suggest to focus on the first commit which contains the main logic
changes.
- I also have a working implementation of done_slash for pallet_vesting,
should I add it to this PR?
- If I run `cargo +nightly fmt --all` then I get changes to a lot of
unrelated crates, so not sure if I should run it to avoid the fmt
failure of the CI
- Should I hunt down references to fungible/fungibles documentation and
update it accordingly?

**Polkadot address:** `15fj1UhQp8Xes7y7LSmDYTy349mXvUwrbNmLaP5tQKBxsQY1`

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)

---------

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <[email protected]>

The AllowTopLevelPaidExecutionFrom allows ClearOrigin instructions
before the expected BuyExecution instruction, it also allows messages
without any origin altering instructions.

This commit enhances the barrier to also support messages that use
AliasOrigin, or DescendOrigin. This is sometimes desired in asset
transfer XCM programs that need to run the inbound assets instructions
using the origin chain root origin, but then want to drop privileges for
the rest of the program. Currently these programs drop privileges by
clearing the origin completely, but that also unnecessarily limits the
range of actions available to the rest of the program. Using
DescendOrigin or AliasOrigin allows the sending chain to instruct the
receiving chain what the deprivileged real origin is.

See https://github.com/polkadot-fellows/RFCs/pull/109 and
https://github.com/polkadot-fellows/RFCs/pull/122 for more details on
how DescendOrigin and AliasOrigin could be used instead of ClearOrigin.

---------

Signed-off-by: Adrian Catangiu <[email protected]>

cc https://github.com/paritytech/ci_cd/issues/1035

cc https://github.com/paritytech/ci_cd/issues/1023

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Maksym H <[email protected]>
Co-authored-by: gui <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: ggwpez <[email protected]>

This change adds the required validation for stronger UX guarantees when
using `InitiateReserveWithdraw` or `InitiateTeleport` XCM instructions.
Execution of the instructions will fail if the local chain is not
configured to trust the "destination" or "reserve" chain as a
reserve/trusted-teleporter for the provided "assets".

With this change, misuse of `InitiateReserveWithdraw`/`InitiateTeleport`
fails on origin with no overall side-effects, rather than failing on
destination (with side-effects to origin's assets issuance).

The commit also makes the same validations for pallet-xcm transfers, and
adds regression tests.

---------

Signed-off-by: Adrian Catangiu <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>

This PR primarily fixes the issue with
`zombienet-bridges-0001-asset-transfer-works` (see:
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7404903).

The PR looks large, but most of the changes involve splitting `LaneId`
into `LegacyLaneId` and `HashedLaneId`. All pallets now use `LaneId` as
a generic parameter.

The actual bridging pallets are now backward compatible and work with
actual **substrate-relay v1.6.10**, which does not even known anything
about permissionless lanes or the new pallet changes.



## Important

- [x] added migration for `pallet_bridge_relayers` and
`RewardsAccountParams` change order of params, which generates different
accounts

## Deployment follow ups
- [ ] fix monitoring for
`at_{}_relay_{}_reward_for_msgs_from_{}_on_lane_{}`
- [ ] check sovereign reward accounts - because of changed
`RewardsAccountParams`
- [ ] deploy another messages instances for permissionless lanes - on
BHs or AHs?
- [ ] return bac...

It is a first step for switching to the `frame-omni-bencher` for CI.

This PR includes several changes related to generating chain specs plus:

- [x] pallet `assigned_slots` fix missing `#[serde(skip)]` for phantom
- [x] pallet `paras_inherent` benchmark fix - cherry-picked from
https://github.com/paritytech/polkadot-sdk/pull/5688
- [x] migrates `get_preset` to the relevant runtimes
- [x] fixes Rococo genesis presets - does not work
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7317249
- [x] fixes Rococo benchmarks for CI 
- [x] migrate westend genesis
- [x] remove wococo stuff

Closes: https://github.com/paritytech/polkadot-sdk/issues/5680

## Follow-ups
- Fix for frame-omni-bencher
https://github.com/paritytech/polkadot-sdk/pull/5655
- Enable new short-benchmarking CI -
https://github.com/paritytech/polkadot-sdk/pull/5706
- Remove gitlab pipelines for short benchmarking
- refactor all Cumulus runtimes to use `get_preset` -
https://github.com/paritytech/polkadot-sdk/issues/5704
- https://github.com/paritytech/polkadot-sdk/issues/5705
- https://github.com/paritytech/polkadot-sdk/issues/5700
- [ ] Backport to the stable

---------

Co-authored-by: command-bot <>
Co-authored-by: ordian <[email protected]>

- This will ensure that a correct product name
(Polkadot/Cumulus/Substrate) is referenced in license headers.
- Closes https://github.com/paritytech/license-scanner/issues/49

The dry-run shows in `forwarded_xcms` all the messages in the queues at
the time of calling the API.
Each time the API is called, the result could be different.
You could get messages even if you dry-run something that doesn't send a
message, like a `System::remark`.

This PR fixes this by clearing the message queues before doing the
dry-run, so the only messages left are the ones the users of the API
actually care about.

---------

Co-authored-by: Adrian Catangiu <[email protected]>

# Context

Fees can already be paid in other assets locally thanks to the Trader
implementations we have.
This doesn't work when sending messages because delivery fees go through
a different mechanism altogether.
The idea is to fix this leveraging the `AssetExchanger` config item
that's able to turn the asset the user wants to pay fees in into the
asset the router expects for delivery fees.

# Main addition

An adapter was needed to use `pallet-asset-conversion` for exchanging
assets in XCM.
This was created in
https://github.com/paritytech/polkadot-sdk/pull/5130.

The XCM executor was modified to use `AssetExchanger` (when available)
to swap assets to pay for delivery fees.

## Limitations

We can only pay for delivery fees in different assets in intermediate
hops. We can't pay in different assets locally. The first hop will
always need the native token of the chain (or whatever is specified in
the `XcmRouter`).
This is a byproduct of using the `BuyExecution` instruction to know
which asset should be used for delivery fee payment.
Since this instruction is not present when executing an XCM locally, we
are left with this limitation.
To illustrate this limitation, I'll show two scenarios. All chains
involved have pools.

### Scenario 1

Parachain A --> Parachain B

Here, parachain A can use any asset in a pool with its native asset to
pay for local execution fees.
However, as of now we can't use those for local delivery fees.
This means transfers from A to B need some amount of A's native token to
pay for delivery fees.

### Scenario 2

Parachain A --> Parachain C --> Parachain B

Here, Parachain C's remote delivery fees can be paid with any asset in a
pool with its native asset.
This allows a reserve asset transfer between A and B with C as the
reserve to only need A's native token at the starting hop.
After that, it could all be pool assets.

## Future work

The fact that delivery fees go through a totally different mechanism
results in a lot of bugs and pain points.
Unfortunately, this is not so easy to solve in a backwards compatible
manner.
Delivery fees will be integrated into the language in future XCM
versions, following
https://github.com/polkadot-fellows/xcm-format/pull/53.

Old PR: https://github.com/paritytech/polkadot-sdk/pull/4375.

There was no need for it to be `&mut self` since block import can happen
concurrently for different blocks and in many cases it was `&mut Arc<dyn
BlockImport>` anyway :man_shrugging:

Similar in nature to
https://github.com/paritytech/polkadot-sdk/pull/4844

Closes #4242

XCM programs that deposit assets to some new (empty) account will now
succeed if at least one of the deposited assets satisfies ED. Before
this change, the requirement was that the _first_ asset had to satisfy
ED, but assets order can be changed during reanchoring so it is not
reliable.

With this PR, ordering doesn't matter, any one(s) of them can satisfy ED
for the whole deposit to work.

Kusama address: FkB6QEo8VnV3oifugNj5NeVG3Mvq1zFbrUu4P5YwRoe5mQN

---------

Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: command-bot <>

Added a new adapter to xcm-builder, the `SingleAssetExchangeAdapter`.
This adapter makes it easy to use `pallet-asset-conversion` for
configuring the `AssetExchanger` XCM config item.

I also took the liberty of adding a new function to the `AssetExchange`
trait, with the following signature:

```rust
fn quote_exchange_price(give: &Assets, want: &Assets, maximal: bool) -> Option<Assets>;
```

The signature is meant to be fairly symmetric to that of
`exchange_asset`.
The way they interact can be seen in the doc comment for it in the
`AssetExchange` trait.

This is a breaking change but is needed for
https://github.com/paritytech/polkadot-sdk/pull/5131.
Another idea is to create a new trait for this but that would require
setting it in the XCM config which is also breaking.

Old PR: https://github.com/paritytech/polkadot-sdk/pull/4375.

---------

Co-authored-by: Adrian Catangiu <[email protected]>

This PR replaces env_logger with sp_tracing because of an issue with
env_logger and gum #4660

---------

Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: Andrei Eres <[email protected]>

The test name is `test-frame-ui` I don't know if I can also change it to
`test-ui` without breaking other stuff. So I kept the name unchanged.

---------

Co-authored-by: Bastian Köcher <[email protected]>

Fixes #4960 

Configuring `FeeManager` enforces the boundary `Into<[u8; 32]>` for the
`AccountId` type.

Here is how it works currently: 

Configuration:
```rust
    type FeeManager = XcmFeeManagerFromComponents<
        IsChildSystemParachain<primitives::Id>,
        XcmFeeToAccount<Self::AssetTransactor, AccountId, TreasuryAccount>,
    >;
```

`XcmToFeeAccount` struct:
```rust
/// A `HandleFee` implementation that simply deposits the fees into a specific on-chain
/// `ReceiverAccount`.
///
/// It reuses the `AssetTransactor` configured on the XCM executor to deposit fee assets. If
/// the `AssetTransactor` returns an error while calling `deposit_asset`, then a warning will be
/// logged and the fee burned.
pub struct XcmFeeToAccount<AssetTransactor, AccountId, ReceiverAccount>(
	PhantomData<(AssetTransactor, AccountId, ReceiverAccount)>,
);

impl<
		AssetTransactor: TransactAsset,
		AccountId: Clone + Into<[u8; 32]>,
		ReceiverAccount: Get<AccountId>,
	> HandleFee for XcmFeeToAccount<AssetTransactor, AccountId, ReceiverAccount>
{
	fn handle_fee(fee: Assets, context: Option<&XcmContext>, _reason: FeeReason) -> Assets {
		deposit_or_burn_fee::<AssetTransactor, _>(fee, context, ReceiverAccount::get());

		Assets::new()
	}
}
```

`deposit_or_burn_fee()` function:
```rust
/// Try to deposit the given fee in the specified account.
/// Burns the fee in case of a failure.
pub fn deposit_or_burn_fee<AssetTransactor: TransactAsset, AccountId: Clone + Into<[u8; 32]>>(
	fee: Assets,
	context: Option<&XcmContext>,
	receiver: AccountId,
) {
	let dest = AccountId32 { network: None, id: receiver.into() }.into();
	for asset in fee.into_inner() {
		if let Err(e) = AssetTransactor::deposit_asset(&asset, &dest, context) {
			log::trace!(
				target: "xcm::fees",
				"`AssetTransactor::deposit_asset` returned error: {:?}. Burning fee: {:?}. \
				They might be burned.",
				e, asset,
			);
		}
	}
}
```

---

In order to use **another** `AccountId` type (for example, 20 byte
addresses for compatibility with Ethereum or Bitcoin), one has to
duplicate the code as the following (roughly changing every `32` to
`20`):
```rust
/// A `HandleFee` implementation that simply deposits the fees into a specific on-chain
/// `ReceiverAccount`.
///
/// It reuses the `AssetTransactor` configured on the XCM executor to deposit fee assets. If
/// the `AssetTransactor` returns an error while calling `deposit_asset`, then a warning will be
/// logged and the fee burned.
pub struct XcmFeeToAccount<AssetTransactor, AccountId, ReceiverAccount>(
    PhantomData<(AssetTransactor, AccountId, ReceiverAccount)>,
);
impl<
        AssetTransactor: TransactAsset,
        AccountId: Clone + Into<[u8; 20]>,
        ReceiverAccount: Get<AccountId>,
    > HandleFee for XcmFeeToAccount<AssetTransactor, AccountId, ReceiverAccount>
{
    fn handle_fee(fee: XcmAssets, context: Option<&XcmContext>, _reason: FeeReason) -> XcmAssets {
        deposit_or_burn_fee::<AssetTransactor, _>(fee, context, ReceiverAccount::get());

        XcmAssets::new()
    }
}

pub fn deposit_or_burn_fee<AssetTransactor: TransactAsset, AccountId: Clone + Into<[u8; 20]>>(
    fee: XcmAssets,
    context: Option<&XcmContext>,
    receiver: AccountId,
) {
    let dest = AccountKey20 { network: None, key: receiver.into() }.into();
    for asset in fee.into_inner() {
        if let Err(e) = AssetTransactor::deposit_asset(&asset, &dest, context) {
            log::trace!(
                target: "xcm::fees",
                "`AssetTransactor::deposit_asset` returned error: {:?}. Burning fee: {:?}. \
                They might be burned.",
                e, asset,
            );
        }
    }
}
```

---

This results in code duplication, which can be avoided simply by
relaxing the trait enforced by `XcmFeeToAccount`.

In this PR, I propose to introduce a new trait called `IntoLocation` to
be able to express both `Into<[u8; 32]>` and `Into<[u8; 20]>` should be
accepted (and every other `AccountId` type as long as they implement
this trait).

Currently, `deposit_or_burn_fee()` function converts the `receiver:
AccountId` to a location. I think converting an account to `Location`
should not be the responsibility of `deposit_or_burn_fee()` function.

This trait also decouples the conversion of `AccountId` to `Location`,
from `deposit_or_burn_fee()` function. And exposes `IntoLocation` trait.
Thus, allowing everyone to come up with their `AccountId` type and make
it compatible for configuring `FeeManager`.

---

Note 1: if there is a better file/location to put `IntoLocation`, I'm
all ears

Note 2: making `deposit_or_burn_fee` or `XcmToFeeAccount` generic was
not possible from what I understood, due to Rust currently do not
support a way to express the generic should implement either `trait A`
or `trait B` (since the compiler cannot guarantee they won't overlap).
In this case, they are `Into<[u8; 32]>` and `Into<[u8; 20]>`.
See [this](https://github.com/rust-lang/rust/issues/20400) and
[this](https://github.com/rust-lang/rfcs/pull/1672#issuecomment-262152934).

Note 3: I should also submit a PR to `frontier` that implements
`IntoLocation` for `AccountId20` if this PR gets accepted.


### Summary 
this new trait:
- decouples the conversion of `AccountId` to `Location`, from
`deposit_or_burn_fee()` function
- makes `XcmFeeToAccount` accept every possible `AccountId` type as long
as they they implement `IntoLocation`
- backwards compatible
- keeps the API simple and clean while making it less restrictive


@franciscoaguirre and @gupnik

 are already aware of the issue, so tagging
them here for visibility.

---------

Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: command-bot <>

v3 PalletInfo had the fields public, but not v4. Any reason why?
I need the PalletInfo fields public so I can read the values and do some
logic based on that at Polimec
@franciscoaguirre 

If this could be backported would be highly appreciated 🙏🏻



---------

Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

This should remove nearly all usage of `sp-std` except:
- bridge and bridge-hubs
- a few of frames re-export `sp-std`, keep them for now
- there is a usage of `sp_std::Writer`, I don't have an idea how to move
it

Please review proc-macro carefully. I'm not sure I'm doing it the right
way.

Note: need `/bot fmt`

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: command-bot <>

This improves logging in the xcm-executor to have better debugability
when executing a XCM message.

It was added to v4 and v3 but was missing from v2

Fixes #3770 

Added `clear_origin` as an allowed command after commands that load the
holdings register, in the safe xcm builder.

Checklist
- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [x] My PR follows the [labeling
requirements](https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process)
of this project (at minimum one label for T required)
- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works (if applicable)

---------

Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: gupnik <[email protected]>

Introduce an optional auto-increment setup for the IDs of new assets.

---------

Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Closes: https://github.com/paritytech/polkadot-sdk/issues/4298

This PR also merges `xcm-fee-payment-runtime-api` module to the
`xcm-runtime-api`.


## TODO

- [x] rename `convert` to `convert_location` and add new one
`convert_account` (opposite direction)
- [x] add to the all testnet runtimes
- [x] check polkadot-js if supports that automatically or if needs to be
added manually https://github.com/polkadot-js/api/pull/5917
- [ ] backport/patch for fellows and release (asap)

## Open questions
- [x] should we merge `xcm-runtime-api` and
`xcm-fee-payment-runtime-api` to the one module `xcm-runtime-api` ?

## Usage
Input:
 - `location:  VersionedLocation`
 
Output:
 - account_id bytes


![image](https://github.com/paritytech/polkadot-sdk/assets/8159517/4607b15a-77d2-462b-806c-606107776c0d)

---------

Co-authored-by: Bastian Köcher <[email protected]>

Signed-off-by: dashangcun <[email protected]>
Co-authored-by: dashangcun <[email protected]>

After preparing in https://github.com/paritytech/polkadot-sdk/pull/4633,
we can lift also all internal dependencies up to the workspace.

This does not actually change anything, but uses `workspace = true` for
all dependencies. You can check it with:
```bash
git checkout -q $(git merge-base oty-lift-all-deps origin/master)
cargo tree -e features > master.out

git checkout -q oty-lift-all-deps
cargo tree -e features > new.out
diff master.out new.out
```

It did not yet lift 100% of dependencies, some inside of `target.*` or
some that had conflicting aliases introduced recently. But i will do
these together in a follow-up with CI checks.

Can be reproduced with [zepter](https://github.com/ggwpez/zepter/):
`zepter transpose d lift-to-workspace "regex:.*" --version-resolver
highest --skip-package "polkadot-sdk" --ignore-errors --fix`.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

This PR aligns the settings for `MaxFreezes`, `RuntimeFreezeReason`, and
`FreezeIdentifier`.

#### Future work and improvements
https://github.com/paritytech/polkadot-sdk/issues/2997 (remove
`MaxFreezes` and `FreezeIdentifier`)

Inherited workspace dependencies cannot be renamed by the crate using
them (see [1](https://github.com/rust-lang/cargo/issues/12546),
[2](https://stackoverflow.com/questions/76792343/can-inherited-dependencies-in-rust-be-aliased-in-the-cargo-toml-file)).
Since we want to use inherited workspace dependencies everywhere, we
first need to unify all aliases that we use for a dependency throughout
the workspace.
The umbrella crate is currently excluded from this procedure, since it
should be able to export the crates by their original name without much
hassle.

For example: one crate may alias `parity-scale-codec` to `codec`, while
another crate does not alias it at all. After this change, all crates
have to use `codec` as name. The problematic combinations were:
- conflicting aliases: most crates aliases as `A` but some use `B`.
- missing alias: most of the crates alias a dep but some dont.
- superfluous alias: most crates dont alias a dep but some do.

The script that i used first determines whether most crates opted to
alias a dependency or not. From that info it decides whether to use an
alias or not. If it decided to use an alias, the most common one is used
everywhere.

To reproduce, i used
[this](https://github.com/ggwpez/substrate-scripts/blob/master/uniform-crate-alias.py)
python script in combination with
[this](https://github.com/ggwpez/zepter/blob/38ad10585fe98a5a86c1d2369738bc763a77057b/renames.json)
error output from Zepter.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Depends on https://github.com/paritytech/polkadot-sdk/pull/4621.

Implemented the
[`XcmPaymentApi`](https://github.com/paritytech/polkadot-sdk/pull/3607)
and [`DryRunApi`](https://github.com/paritytech/polkadot-sdk/pull/3872)
on all system parachains.

More scenarios can be tested on both rococo and westend if all system
parachains implement this APIs.
The objective is for all XCM-enabled runtimes to implement them.
After demonstrating fee estimation in a UI on the testnets, come the
fellowship runtimes.

Step towards https://github.com/paritytech/polkadot-sdk/issues/690.

Follow-up to the new `XcmDryRunApi` runtime API introduced in
https://github.com/paritytech/polkadot-sdk/pull/3872.

Taking an extrinsic means the frontend has to sign first to dry-run and
once again to submit.
This is bad UX which is solved by taking an `origin` and a `call`.
This also has the benefit of being able to dry-run as any account, since
it needs no signature.

This is a breaking change since I changed `dry_run_extrinsic` to
`dry_run_call`, however, this API is still only on testnets.
The crates are bumped accordingly.

As a part of this PR, I changed the name of the API from `XcmDryRunApi`
to just `DryRunApi`, since it can be used for general dry-running :)

Step towards https://github.com/paritytech/polkadot-sdk/issues/690.

Example of calling the API with PAPI, not the best code, just testing :)

```ts
// We just build a call, the arguments make it look very big though.
const call = localApi.tx.XcmPallet.transfer_assets({
  dest: XcmVersionedLocation.V4({ parents: 0, interior: XcmV4Junctions.X1(XcmV4Junction.Parachain(1000)) }),
  beneficiary: XcmVersionedLocation.V4({ parents: 0, interior: XcmV4Junctions.X1(XcmV4Junction.AccountId32({ network: undefined, id: Binary.fromBytes(encodeAccount(account.address)) })) }),
  weight_limit: XcmV3WeightLimit.Unlimited(),
  assets: XcmVersionedAssets.V4([{
    id: { parents: 0, interior: XcmV4Junctions.Here() },
    fun: XcmV3MultiassetFungibility.Fungible(1_000_000_000_000n) }
  ]),
  fee_asset_item: 0,
});
// We call the API passing in a signed origin 
const result = await localApi.apis.XcmDryRunApi.dry_run_call(
  WestendRuntimeOriginCaller.system(DispatchRawOrigin.Signed(account.address)),
  call.decodedCall
);
if (result.success && result.value.execution_result.success) {
  // We find the forwarded XCM we want. The first one going to AssetHub in this case.
  const xcmsToAssetHub = result.value.forwarded_xcms.find(([location, _]) => (
    location.type === "V4" &&
      location.value.parents === 0 &&
      location.value.interior.type === "X1"
      && location.value.interior.value.type === "Parachain"
      && location.value.interior.value.value === 1000
  ))!;

  // We can even find the delivery fees for that forwarded XCM.
  const deliveryFeesQuery = await localApi.apis.XcmPaymentApi.query_delivery_fees(xcmsToAssetHub[0], xcmsToAssetHub[1][0]);

  if (deliveryFeesQuery.success) {
    const amount = deliveryFeesQuery.value.type === "V4" && deliveryFeesQuery.value.value[0].fun.type === "Fungible" && deliveryFeesQuery.value.value[0].fun.value.valueOf() || 0n;
    // We store them in state somewhere.
    setDeliveryFees(formatAmount(BigInt(amount)));
  }
}
```

---------

Co-authored-by: Bastian Köcher <[email protected]>

Marked XCMv2 as deprecated now that we have XCMv4.
It will be removed sometime around June 2024.

---------

Co-authored-by: Branislav Kontur <[email protected]>

Attempt to avoid specifying `BlockHashCount` for different `mocking::{MockBlock, MockBlockU32, MockBlockU128}` (#4543)

While doing some migration/rebase I came in to the situation, where I
needed to change `mocking::MockBlock` to `mocking::MockBlockU32`:
```
#[derive_impl(frame_system::config_preludes::TestDefaultConfig)]
impl frame_system::Config for TestRuntime {
	type Block = frame_system::mocking::MockBlockU32<TestRuntime>;
	type AccountData = pallet_balances::AccountData<ThisChainBalance>;
}
```
But actual `TestDefaultConfig` for `frame_system` is using `ConstU64`
for `type BlockHashCount = frame_support::traits::ConstU64<10>;`
[here](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/system/src/lib.rs#L303).
Because of this, it force me to specify and add override for `type
BlockHashCount = ConstU32<10>`.

This PR tries to fix this with `TestBlockHashCount` implementation for
`TestDefaultConfig` which supports `u32`, `u64` and `u128` as a
`BlockNumber`.

### How to simulate error
Just by removing `type BlockHashCount = ConstU32<250>;`
[here](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/multisig/src/tests.rs#L44)
```
:~/parity/olkadot-sdk$ cargo test -p pallet-multisig
   Compiling pallet-multisig v28.0.0 (/home/bparity/parity/aaa/polkadot-sdk/substrate/frame/multisig)
error[E0277]: the trait bound `ConstU64<10>: frame_support::traits::Get<u32>` is not satisfied
   --> substrate/frame/multisig/src/tests.rs:41:1
    |
41  | #[derive_impl(frame_system::config_preludes::TestDefaultConfig)]
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `frame_support::traits::Get<u32>` is not implemented for `ConstU64<10>`
    |
    = help: the following other types implement trait `frame_support::traits::Get<T>`:
              <ConstU64<T> as frame_support::traits::Get<u64>>
              <ConstU64<T> as frame_support::traits::Get<std::option::Option<u64>>>
note: required by a bound in `frame_system::Config::BlockHashCount`
   --> /home/bparity/parity/aaa/polkadot-sdk/substrate/frame/system/src/lib.rs:535:24
    |
535 |         type BlockHashCount: Get<BlockNumberFor<Self>>;
    |                              ^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `Config::BlockHashCount`
    = note: this error originates in the attribute macro `derive_impl` which comes from the expansion of the macro `frame_support::macro_magic::forward_tokens_verbatim` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0277`.
error: could not compile `pallet-multisig` (lib test) due to 1 previous error 
```




## For reviewers:

(If there is a better solution, please let me know!)

The first commit contains actual attempt to fix the problem:
https://github.com/paritytech/polkadot-sdk/commit/3c5499e5

.
The second commit is just removal of `BlockHashCount` from all other
places where not needed by default.

Closes: https://github.com/paritytech/polkadot-sdk/issues/1657

---------

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Svyatoslav Nikolsky <[email protected]>

# Context

XCM docs are currently an md book hosted with github pages:
https://paritytech.github.io/xcm-docs/.
While that's fine, it's not in line with the work being done in the
polkadot-sdk docs.

# Main addition

This PR aims to fix that by bringing the docs back to this repo.
This does not have all the information currently present in the mdbook
xcm-docs but aims to be a good chunk of it and fully replace it over
time.

I also added the sections `guides` and `cookbook` which will be very
useful for users wanting to get into XCM.
For now I only added one example to the cookbook, but have ideas for
guides and more examples.
Having this docs be in rust docs is very useful for the cookbook.

# TODO

- [x] Use `FungibleAdapter`
- [x] Improve and relocate mock message queue
- [x] Fix license issue. Why does docs/sdk/ not have this problem? (Just
added the licenses)

# Next steps

- More examples in the cookbook
- End-to-end XCM guide with zombienet testing

---------

Co-authored-by: command-bot <>
Co-authored-by: joe petrowski <[email protected]>

This bump of versions:

https://github.com/paritytech/polkadot-sdk/pull/4409/files#diff-13ee4b2252c9e516a0547f2891aa2105c3ca71c6d7a1e682c69be97998dfc87eR11936

reintroduced a dependency to proc-macro-crate 2.0.0 which is suffering
from: https://github.com/bkchr/proc-macro-crate/pull/42 this, so bump
parity-scale-codec to a newer version to eliminate the bad
proc-macro-crate 2.0.0 dependency.

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>
Co-authored-by: command-bot <>

Adds an additional extrinsic call to the `pallet-balances` to _burn_
tokens.
Depending on the `keep_alive` flag, the call might or might not reap the
account.

Required modification of the _fungible's_ `Mutate` trait, `burn_from`
function to allow the `Preservation` argument.

**TODO**
- [x] run benchmarks & update weights
- [x] make sure prdoc is required & properly formatted

Related issue: https://github.com/paritytech/polkadot-sdk/issues/3943

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

# Context

Estimating fees for XCM execution and sending has been an area with bad
UX.
The addition of the
[XcmPaymentApi](https://github.com/paritytech/polkadot-sdk/pull/3607)
exposed the necessary components to be able to estimate XCM fees
correctly, however, that was not the full story.
The `XcmPaymentApi` works for estimating fees only if you know the
specific XCM you want to execute or send.
This is necessary but most UIs want to estimate the fees for extrinsics,
they don't necessarily know the XCM program that's executed by them.

# Main addition

A new runtime API is introduced, the `XcmDryRunApi`, that given an
extrinsic, or an XCM program, returns its effects:
- Execution result
- Local XCM (in the case of an extrinsic)
- Forwarded XCMs
- List of events

This API can be used on its own for dry-running purposes, for
double-checking or testing, but it mainly shines when used in
conjunction with the `XcmPaymentApi`.
UIs can use these two APIs to estimate transfers.

# How it works

New tests are added to exemplify how to incorporate both APIs.
There's a mock test just to make sure everything works under
`xcm-fee-payment-runtime-api`.
There's a real-world test using Westend and AssetHubWestend under
`cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`.
Added both a test for a simple teleport between chains and a reserve
transfer asset between two parachains going through a reserve.

The steps to follow:
- Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and
forwarded messages
- For each forwarded message
- Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery
fees
- Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the
remote execution weight
- (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE
DESTINATION to know on which assets the execution fees can be paid
- Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to
convert weight to the actual remote execution fees
- Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new
message will be forwarded, if so, continue

# Dear reviewer

The changes in this PR are grouped as follows, and in order of
importance:
- Addition of new runtime API
- Definition, mock and simple tests:
polkadot/xcm/xcm-fee-payment-runtime-api/*
- Implemented on Westend, Asset Hub Westend and Penpal, will implement
on every runtime in a following PR
- Addition of a new config item to the XCM executor for recording xcms
about to be executed
  - Definition: polkadot/xcm/xcm-executor/*
  - Implementation: polkadot/xcm/pallet-xcm/*
- had to update all runtime xcm_config.rs files with `type XcmRecorder =
XcmPallet;`
- Addition of a new trait for inspecting the messages in queues
  - Definition: polkadot/xcm/xcm-builder/src/routing.rs
  - Implemented it on all routers:
    - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs
- ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on
implementation in cumulus/pallets/parachain-system/src/lib.rs)
    - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs
    - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs
- More complicated and useful tests:
-
cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs

## Next steps

With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have
the new API.
UIs can test on these runtimes to create better experiences around
cross-chain operations.

Next:
- Add XcmDryRunApi to all system parachains
- Integrate xcm fee estimation in all emulated tests
- Get this on the fellowship runtimes

---------

Co-authored-by: Adrian Catangiu <[email protected]>