Ensure outbound XCMs are decodable with limits + add `EnsureDecodableXcm` router (for testing purposes) (#4186)

This PR:
- adds `EnsureDecodableXcm` (testing) router that attempts to *encode*
and *decode* passed XCM `message` to ensure that the receiving side will
be able to decode, at least with the same XCM version.
- fixes `pallet_xcm` / `pallet_xcm_benchmarks` assets data generation

Relates to investigation of
https://substrate.stackexchange.com/questions/11288 and missing fix
https://github.com/paritytech/polkadot-sdk/pull/2129 which did not get
into the fellows 1.1.X release.

## Questions/TODOs

- [x] fix XCM benchmarks, which produces undecodable data - new router
catched at least two cases
  - `BoundedVec exceeds its limit`
  - `Fungible asset of zero amount is not allowed`  
- [x] do we need to add `sort` to the `prepend_with` as we did for
reanchor [here](https://github.com/paritytech/polkadot-sdk/pull/2129)?
@serban300 (**created separate/follow-up PR**:
https://github.com/paritytech/polkadot-sdk/pull/4235)
- [x] We added decoding check to `XcmpQueue` -> `validate_xcm_nesting`,
why not to added to the `ParentAsUmp` or `ChildParachainRouter`?
@franciscoaguirre (**created separate/follow-up PR**:
https://github.com/paritytech/polkadot-sdk/pull/4236

)
- [ ] `SendController::send_blob` replace `VersionedXcm::<()>::decode(`
with `VersionedXcm::<()>::decode_with_depth_limit(MAX_XCM_DECODE_DEPTH,
data)` ?

---------

Co-authored-by: Adrian Catangiu <[email protected]>

This PR:
- moves `validate_xcm_nesting` from `XcmpQueue` into the `VersionedXcm`
- adds `validate_xcm_nesting` to the `ParentAsUmp`
- adds `validate_xcm_nesting` to the `ChildParachainRouter`


Based on discussion
[here](https://github.com/paritytech/polkadot-sdk/pull/4186#discussion_r1571344270)
and/or
[here](https://github.com/paritytech/polkadot-sdk/pull/4186#discussion_r1572076666)
and/or [here]()

## Question/TODO

- [x] To the
[comment](https://github.com/paritytech/polkadot-sdk/pull/4186#discussion_r1572072295)
- Why was `validate_xcm_nesting` added just to the `XcmpQueue` router
and nowhere else? What kind of problem `MAX_XCM_DECODE_DEPTH` is
solving? (see
[comment](https://github.com/paritytech/polkadot-sdk/pull/4236#discussion_r1574605191))

This PR:
- sanitizes all `UniversalLocation`s with `GlobalConsensus` (when
possible) - addressing
[comment](https://github.com/paritytech/polkadot-sdk/pull/4025#discussion_r1557361473)
- adds `DefaultConfig` for `pallet-xcm-benchmarks` for `system`

Working towards migrating the `parity-bridges-common` repo inside
`polkadot-sdk`. This PR upgrades some dependencies in order to align
them with the versions used in `parity-bridges-common`

Related to
https://github.com/paritytech/parity-bridges-common/issues/2538

Currently the xcm-executor returns an `Unimplemented` error if it
receives any HRMP-related instruction.
What I propose here, which is what we are currently doing in our forked
executor at polimec, is to introduce a trait implemented by the executor
which will handle those instructions.

This way, if parachains want to keep the default behavior, they just use
`()` and it will return unimplemented, but they can also implement their
own logic to establish HRMP channels with other chains in an automated
fashion, without requiring to go through governance.

Our implementation is mentioned in the [polkadot HRMP
docs](https://arc.net/l/quote/hduiivbu

), and it was suggested to us to
submit a PR to add these changes to polkadot-sdk.

---------

Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: command-bot <>

Step in https://github.com/paritytech/polkadot-sdk/issues/171

This PR removes `as [disambiguation_path]` syntax from `derive_impl`
usage across the polkadot-sdk as introduced in
https://github.com/paritytech/polkadot-sdk/pull/3505

## Problem
During the bumping of the `polkadot-fellows` repository to
`[email protected]`, I encountered a situation where the benchmarks
`teleport_assets` and `reserve_transfer_assets` in AssetHubKusama
started to fail. This issue arose due to a decreased ED balance for
AssetHubs introduced
[here](https://github.com/polkadot-fellows/runtimes/pull/158/files#diff-80668ff8e793b64f36a9a3ec512df5cbca4ad448c157a5d81abda1b15f35f1daR213),
and also because of a [missing CI
pipeline](https://github.com/polkadot-fellows/runtimes/issues/197) to
check the benchmarks, which went unnoticed.

These benchmarks expect the `caller` to have enough:
1. balance to transfer (BTT)
2. balance for paying delivery (BFPD).
 
So the initial balance was calculated as `ED * 100`, which seems
reasonable:
```
const ED_MULTIPLIER: u32 = 100;
let balance = existential_deposit.saturating_mul(ED_MULTIPLIER.into());`
```
The problem arises when the price for delivery is 100 times higher than
the existential deposit. In other words, when `ED * 100` does not cover
`BTT` + `BFPD`.

I check AHR/AHW/AHK/AHP and this problem has only AssetHubKusama
```
ED: 3333333
calculated price to parent delivery:  1031666634  (from xcm logs from the benchmark)
---

3333333 * 100 - BTT(3333333) - BFPD(1031666634) = −701666667
```
which results in the error;
```
2024-02-23 09:19:42 Unable to charge fee with error Module(ModuleError { index: 31, error: [17, 0, 0, 0], message: Some("FeesNotMet") })
Error: Input("Benchmark pallet_xcm::reserve_transfer_assets failed: FeesNotMet")
     
```

## Solution

The benchmarks `teleport_assets` and `reserve_transfer_assets` were
fixed by removing `ED * 100` and replacing it with `DeliveryHelper`
logic, which calculates the (almost real) price for delivery and sets it
along with the existential deposit as the initial balance for the
account used in the benchmark.


## TODO

- [ ] patch for 1.6 -
https://github.com/paritytech/polkadot-sdk/pull/3466
- [ ] patch for 1.7 -
https://github.com/paritytech/polkadot-sdk/pull/3465


- [ ] patch for 1.8 - TODO: PR

---------

Co-authored-by: Francisco Aguirre <[email protected]>

I found out during the cleanup of this deprecation message in the
`polkadot-fellows` repository that we deprecated `CurrencyAdapter`
without making the recommended changes.


## TODO
- [ ] fix `polkadot-fellows` bump to 1.6.0
https://github.com/polkadot-fellows/runtimes/pull/159



---------

Co-authored-by: Francisco Aguirre <[email protected]>

Changes (partial https://github.com/paritytech/polkadot-sdk/issues/994):
- Set log to `0.4.20` everywhere
- Lift `log` to the workspace

Starting with a simpler one after seeing
https://github.com/paritytech/polkadot-sdk/pull/2065 from @jsdw

.
This sets the `default-features` to `false` in the root and then
overwrites that in each create to its original value. This is necessary
since otherwise the `default` features are additive and its impossible
to disable them in the crate again once they are enabled in the
workspace.

I am using a tool to do this, so its mostly a test to see that it works
as expected.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Related https://github.com/paritytech/polkadot-sdk/issues/3032

---

Using https://github.com/liamaharon/cargo-workspace-version-tools/

 

`cargo run -- sync --path ../polkadot-sdk`

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

Moved from: https://github.com/paritytech/polkadot/pull/6951

closes https://github.com/paritytech/polkadot-sdk/issues/490

- [x] update cumulus

--- 
This PR introduces transactional processing of certain xcm instructions.
For the list of instructions checkout
https://github.com/paritytech/polkadot-sdk/issues/490. The transactional
processing is implemented as an xcm-executor config item. The two
implementations in this PR are `FrameTransactionalProcessor` and `()`.
The `()` implementation does no transactional processing. Each
implementation of the `ProcessTransaction` trait has an
`IS_TRANSACTIONAL` const that tells the XCVM if transactional processing
is actually implemented. If Transactional processing is implemented,
changes to touched registers should also be rolled back to prevent
inconsistencies.


Note for reviewers:
Check out the following safety assumption:
https://github.com/paritytech/polkadot-sdk/pull/1222/files#diff-4effad7d8c1c9de19fd27e18661cbf2128c8718f3b2420a27d2f816e0749ea53R30



---------

Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: command-bot <>

Clean up all the old syntax.

---------

Co-authored-by: command-bot <>
Co-authored-by: gupnik <[email protected]>
Co-authored-by: Nikhil Gupta <[email protected]>
Co-authored-by: Maksym H <[email protected]>

# Note for reviewer

Most changes are just syntax changes necessary for the new version.
Most important files should be the ones under the `xcm` folder.

# Description 

Added XCMv4.

## Removed `Multi` prefix
The following types have been renamed:
- MultiLocation -> Location
- MultiAsset -> Asset
- MultiAssets -> Assets
- InteriorMultiLocation -> InteriorLocation
- MultiAssetFilter -> AssetFilter
- VersionedMultiAsset -> VersionedAsset
- WildMultiAsset -> WildAsset
- VersionedMultiLocation -> VersionedLocation

In order to fix a name conflict, the `Assets` in `xcm-executor` were
renamed to `HoldingAssets`, as they represent assets in holding.

## Removed `Abstract` asset id

It was not being used anywhere and this simplifies the code.

Now assets are just constructed as follows:

```rust
let asset: Asset = (AssetId(Location::new(1, Here)), 100u128).into();
```

No need for specifying `Concrete` anymore.

## Outcome is now a named fields struct

Instead of

```rust
pub enum Outcome {
  Complete(Weight),
  Incomplete(Weight, Error),
  Error(Error),
}
```

we now have

```rust
pub enum Outcome {
  Complete { used: Weight },
  Incomplete { used: Weight, error: Error },
  Error { error: Error },
}
```

## Added Reanchorable trait

Now both locations and assets implement this trait, making it easier to
reanchor both.

## New syntax for building locations and junctions

Now junctions are built using the following methods:

```rust
let location = Location {
    parents: 1,
    interior: [Parachain(1000), PalletInstance(50), GeneralIndex(1984)].into()
};
```

or

```rust
let location = Location::new(1, [Parachain(1000), PalletInstance(50), GeneralIndex(1984)]);
```

And they are matched like so:

```rust
match location.unpack() {
  (1, [Parachain(id)]) => ...
  (0, Here) => ...,
  (1, [_]) => ...,
}
```

This syntax is mandatory in v4, and has been also implemented for v2 and
v3 for easier migration.

This was needed to make all sizes smaller.

# TODO
- [x] Scaffold v4
- [x] Port github.com/paritytech/polkadot/pull/7236
- [x] Remove `Multi` prefix
- [x] Remove `Abstract` asset id

---------

Co-authored-by: command-bot <>
Co-authored-by: Keith Yeung <[email protected]>

Many clippy lints usually enforced by `-Dcomplexity` and `-Dcorrectness`
are not caught by CI as they are gated by `features`, like
`runtime-benchmarks`, while the clippy CI job runs with only the default
features for all targets.

This PR also adds a CI step to run clippy with `--all-features` to
ensure the code quality is maintained behind feature gates from now on.

To improve local development, clippy lints are downgraded to warnings,
but they still will result in an error at CI due to the `-Dwarnings`
rustflag.

---------

Co-authored-by: Liam Aharon <[email protected]>

In the move from the old `Currency` traits to the new `fungible/s`
family of traits, we already had the `FungiblesAdapter` and
`NonFungiblesAdapter` for multiple fungible and non fungible assets
respectively. However, for handling only one fungible asset, we were
missing a `FungibleAdapter`, and so used the old `CurrencyAdapter`
instead. This PR aims to fill in that gap, and provide the new adapter
for more updated examples.

I marked the old `CurrencyAdapter` as deprecated as part of this PR, and
I'll change it to the new `FungibleAdapter` in a following PR.
The two stages are separated so as to not bloat this PR with some name
fixes in tests.

---------

Co-authored-by: command-bot <>

We currently use a bit of a hack in `.cargo/config` to make sure that
clippy isn't too annoying by specifying the list of lints.

There is now a stable way to define lints for a workspace. The only down
side is that every crate seems to have to opt into this so there's a
*few* files modified in this PR.

Dependencies:

- [x] PR that upgrades CI to use rust 1.74 is merged.

---------

Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

Using taplo, fixes all our broken and inconsistent toml formatting and
adds CI to keep them tidy.

If people want we can customise the format rules as described here
https://taplo.tamasfe.dev/configuration/formatter-options.html

@ggwpez

, I suggest zepter is used only for checking features are
propagated, and leave formatting for taplo to avoid duplicate work and
conflicts.

TODO
- [x] Use `exclude = [...]` syntax in taplo file to ignore zombienet
tests instead of deleting the dir

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Step in https://github.com/paritytech/polkadot-sdk/issues/171



This PR adds `derive_impl` on all `frame_system` config impls for mock
runtimes. The overridden configs are maintained as of now to ensure
minimal changes.

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

## Problem/Motivation
The benchmark for the `ExpectPallet` XCM instruction uses a hard-coded
version `4.0.0` for the `frame_system` pallet. Unfortunately, this
doesn't work for the `polkadot-fellows/runtimes` repository, where we
use dependencies from `crates.io`, e.g.,
[frame-system::23.0.0.0](https://github.com/polkadot-fellows/runtimes/blob/dd7f86f0d50064481ed0b7c0218494a5cfad997e/relay/kusama/Cargo.toml#L83).

Closes: https://github.com/paritytech/polkadot-sdk/issues/2284 

## Solution
This PR fixes the benchmarks that require pallet information and enables
the runtime to provide the correct/custom pallet information. The
default implementation provides `frame_system::Pallet` with index `0`,
where the version is not hard-coded but read from the runtime.


## Local testing

Added log for `T::valid_pallet` to the benchmarks like:
```
let valid_pallet = T::valid_pallet();
log::info!(
	target: "frame::benchmark::pallet",
	"valid_pallet: {}::{}::{}::{}::{}",
	valid_pallet.index,
	valid_pallet.module_name,
	valid_pallet.crate_version.major,
	valid_pallet.crate_version.minor,
	valid_pallet.crate_version.patch,
);
```

Run benchmarks for `westend`:
```
cargo run --bin=polkadot --features=runtime-benchmarks -- benchmark pallet --steps=2 --repeat=1 --extrinsic=* --heap-pages=4096 --json-file=./bench.json --chain=westend-dev --template=./polkadot/xcm/pallet-xcm-benchmarks/template.hbs --pallet=pallet_xcm_benchmarks::generic --output=./polkadot/runtime/westend/src/weights/xcm
```

---

For actual `frame_system` version:
```
[package]
name = "frame-system"
version = "4.0.0-dev"
```

Log dump:
```
2023-11-13 12:56:45 Starting benchmark: pallet_xcm_benchmarks::generic::query_pallet    
2023-11-13 12:56:45 valid_pallet: 0::frame_system::4::0::0    
2023-11-13 12:56:45 valid_pallet: 0::frame_system::4::0::0    
2023-11-13 12:56:45 valid_pallet: 0::frame_system::4::0::0    
2023-11-13 12:56:45 Starting benchmark: pallet_xcm_benchmarks::generic::expect_pallet    
2023-11-13 12:56:45 valid_pallet: 0::frame_system::4::0::0    
2023-11-13 12:56:45 valid_pallet: 0::frame_system::4::0::0    
2023-11-13 12:56:45 valid_pallet: 0::frame_system::4::0::0 
```


For changed `frame_system` version:
```
[package]
name = "frame-system"
version = "5.1.3-dev"
```

Log dump:
```
2023-11-13 12:51:51 Starting benchmark: pallet_xcm_benchmarks::generic::query_pallet    
2023-11-13 12:51:51 valid_pallet: 0::frame_system::5::1::3    
2023-11-13 12:51:51 valid_pallet: 0::frame_system::5::1::3    
2023-11-13 12:51:51 valid_pallet: 0::frame_system::5::1::3    
2023-11-13 12:51:51 Starting benchmark: pallet_xcm_benchmarks::generic::expect_pallet    
2023-11-13 12:51:51 valid_pallet: 0::frame_system::5::1::3    
2023-11-13 12:51:51 valid_pallet: 0::frame_system::5::1::3    
2023-11-13 12:51:51 valid_pallet: 0::frame_system::5::1::3
```

## References

Closes: https://github.com/paritytech/polkadot-sdk/issues/2284

Missing descriptions (47):  

- [x] `cumulus/client/collator/Cargo.toml`
- [x] `cumulus/client/relay-chain-inprocess-interface/Cargo.toml`
- [x] `cumulus/client/cli/Cargo.toml`
- [x] `cumulus/client/service/Cargo.toml`
- [x] `cumulus/client/relay-chain-rpc-interface/Cargo.toml`
- [x] `cumulus/client/relay-chain-interface/Cargo.toml`
- [x] `cumulus/client/relay-chain-minimal-node/Cargo.toml`
- [x] `cumulus/parachains/pallets/parachain-info/Cargo.toml`
- [x] `cumulus/parachains/pallets/ping/Cargo.toml`
- [x] `cumulus/primitives/utility/Cargo.toml`
- [x] `cumulus/primitives/aura/Cargo.toml`
- [x] `cumulus/primitives/core/Cargo.toml`
- [x] `cumulus/primitives/parachain-inherent/Cargo.toml`
- [x] `cumulus/test/relay-sproof-builder/Cargo.toml`
- [x] `cumulus/pallets/xcmp-queue/Cargo.toml`
- [x] `cumulus/pallets/dmp-queue/Cargo.toml`
- [x] `cumulus/pallets/xcm/Cargo.toml`
- [x] `polkadot/erasure-coding/Cargo.toml`
- [x] `polkadot/statement-table/Cargo.toml`
- [x] `polkadot/primitives/Cargo.toml`
- [x] `polkadot/rpc/Cargo.toml`
- [x] `polkadot/node/service/Cargo.toml`
- [x] `polkadot/node/core/parachains-inherent/Cargo.toml`
- [x] `polkadot/node/core/approval-voting/Cargo.toml`
- [x] `polkadot/node/core/dispute-coordinator/Cargo.toml`
- [x] `polkadot/node/core/av-store/Cargo.toml`
- [x] `polkadot/node/core/chain-api/Cargo.toml`
- [x] `polkadot/node/core/prospective-parachains/Cargo.toml`
- [x] `polkadot/node/core/backing/Cargo.toml`
- [x] `polkadot/node/core/provisioner/Cargo.toml`
- [x] `polkadot/node/core/runtime-api/Cargo.toml`
- [x] `polkadot/node/core/bitfield-signing/Cargo.toml`
- [x] `polkadot/node/network/dispute-distribution/Cargo.toml`
- [x] `polkadot/node/network/bridge/Cargo.toml`
- [x] `polkadot/node/network/collator-protocol/Cargo.toml`
- [x] `polkadot/node/network/approval-distribution/Cargo.toml`
- [x] `polkadot/node/network/availability-distribution/Cargo.toml`
- [x] `polkadot/node/network/bitfield-distribution/Cargo.toml`
- [x] `polkadot/node/network/gossip-support/Cargo.toml`
- [x] `polkadot/node/network/availability-recovery/Cargo.toml`
- [x] `polkadot/node/collation-generation/Cargo.toml`
- [x] `polkadot/node/overseer/Cargo.toml`
- [x] `polkadot/runtime/parachains/Cargo.toml`
- [x] `polkadot/runtime/common/slot_range_helper/Cargo.toml`
- [x] `polkadot/runtime/metrics/Cargo.toml`
- [x] `polkadot/xcm/pallet-xcm-benchmarks/Cargo.toml`
- [x] `polkadot/utils/generate-bags/Cargo.toml`
- [x]  `substrate/bin/minimal/runtime/Cargo.toml`

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Signed-off-by: alindima <[email protected]>
Co-authored-by: ordian <[email protected]>
Co-authored-by: Tsvetomir Dimitrov <[email protected]>
Co-authored-by: Marcin S <[email protected]>
Co-authored-by: alindima <[email protected]>
Co-authored-by: Sebastian Kunert <[email protected]>
Co-authored-by: Dmitry Markin <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

Adds a config file that allows to run `zepter` without any arguments in
the workspace to address all issues.
A secondary workflow for the CI is provided as `zepter run check`. Both
the formatting and linting are now in one check for efficiancy.

The latest version also detects some more things that `featalign` was
already showing.

Error message [in the
CI](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3916205)
now looks like this:
```pre
...
crate 'test-parachains' (/Users/vados/Documents/work/polkadot-sdk/polkadot/parachain/test-parachains/Cargo.toml)
  feature 'std'
    must propagate to:
      parity-scale-codec
Found 55 issues (run with --fix to fix).
Error: Command 'lint propagate-feature' failed with exit code 1

Polkadot-SDK uses the Zepter CLI to detect abnormalities in the feature configuration.
It looks like one more more checks failed; please check the console output. You can try to automatically address them by running `zepter`.
Otherwise please ask directly in the Merge Request, GitHub Discussions or on Matrix Chat, thank you.

For more information, see:
  - https://github.com/paritytech/polkadot-sdk/issues/1831
  - https://github.com/ggwpez/zepter


```

TODO:
- [x] Check that CI fails correctly

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

The `xcm` crate was renamed to `staging-xcm` to be able to publish it to
crates.io as someone as squatted `xcm`. The problem with this rename is
that the `TypeInfo` includes the crate name which ultimately lands in
the metadata. The metadata is consumed by downstream users like
`polkadot-js` or people building on top of `polkadot-js`. These people
are using the entire `path` to find the type in the type registry. Thus,
their code would break as the type path would now be [`staging_xcm`,
`VersionedXcm`] instead of [`xcm`, `VersionedXcm`]. This pull request
fixes this by renaming the path segment `staging_xcm` to `xcm`.

This requires: https://github.com/paritytech/scale-info/pull/197



---------

Co-authored-by: Francisco Aguirre <[email protected]>

Combination of paritytech/polkadot#7005, its addon PR
paritytech/polkadot#7585 and its companion paritytech/cumulus#2433.

This PR introduces a new XcmFeesToAccount struct which implements the
`FeeManager` trait, and assigns this struct as the `FeeManager` in the
XCM config for all runtimes.

The struct simply deposits all fees handled by the XCM executor to a
specified account. In all runtimes, the specified account is configured
as the treasury account.

XCM __delivery__ fees are now being introduced (unless the root origin
is sending a message to a system parachain on behalf of the originating
chain).

# Note for reviewers

Most file changes are tests that had to be modified to account for the
new fees.
Main changes are in:
- cumulus/pallets/xcmp-queue/src/lib.rs <- To make it track the delivery
fees exponential factor
- polkadot/xcm/xcm-builder/src/fee_handling.rs <- Added. Has the
FeeManager implementation
- All runtime xcm_config files <- To add the FeeManager to the XCM
configuration

# Important note

After this change, instructions that create and send a new XCM (Query*,
Report*, ExportMessage, InitiateReserveWithdraw, InitiateTeleport,
DepositReserveAsset, TransferReserveAsset, LockAsset and RequestUnlock)
will require the corresponding origin account in the origin register to
pay for transport delivery fees, and the onward message will fail to be
sent if the origin account does not have the required amount. This
delivery fee is on top of what we already collect as tx fees in
pallet-xcm and XCM BuyExecution fees!

Wallet UIs that want to expose the new delivery fee can do so using the
formula:

```
delivery_fee_factor * (base_fee + encoded_msg_len * per_byte_fee)
```

where the delivery fee factor can be obtained from the corresponding
pallet based on which transport you are using (UMP, HRMP or bridges),
the base fee is a constant, the encoded message length from the message
itself and the per byte fee is the same as the configured per byte fee
for txs (i.e. `TransactionByteFee`).

---------

Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Giles Cope <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>

[xcm] Use `Weight::MAX` for `reserve_asset_deposited`, `receive_teleported_asset` benchmarks (#1726)

# Description

## Summary

Previously, the `pallet_xcm::do_reserve_transfer_assets` and
`pallet_xcm::do_teleport_assets` functions relied on weight estimation
for remote chain execution, which was based on guesswork derived from
the local chain. This approach led to complications for runtimes that
did not provide or support specific [XCM
configurations](https://github.com/paritytech/polkadot-sdk/blob/7cbe0c76ef8fd2aabf9f07de0156941ce3ed44b0/polkadot/xcm/xcm-executor/src/config.rs#L43-L47)
for `IsReserve` or `IsTeleporter`. Consequently, such runtimes had to
resort to implementing hard-coded weights for XCM instructions like
`reserve_asset_deposited` or `receive_teleported_asset` to support
extrinsics such as `pallet_xcm::reserve_transfer_assets` and
`pallet_xcm::teleport_assets`, which depended on remote weight
estimation.

The issue of remote weight estimation was addressed and resolved by
[Pull Request
#1645](https://github.com/paritytech/polkadot-sdk/pull/1645), which
removed the need for remote weight estimation.

## Solution

As a continuation of this improvement, the current PR proposes further
cleanup by removing unnecessary hard-coded values and rectifying
benchmark results with `Weight::MAX` that previously used
`T::BlockWeights::get().max_block` as an override for unsupported XCM
instructions like `ReserveAssetDeposited` and `ReceiveTeleportedAsset`.


## Questions

- [x] Can we remove now also `Hardcoded till the XCM pallet is fixed`
for `deposit_asset`? E.g. for AssetHubKusama
[here](https://github.com/paritytech/polkadot-sdk/blob/7cbe0c76ef8fd2aabf9f07de0156941ce3ed44b0/cumulus/parachains/runtimes/assets/asset-hub-kusama/src/weights/xcm/mod.rs#L129-L134)
- [x] Are comments like
[this](https://github.com/paritytech/polkadot-sdk/blob/7cbe0c76ef8fd2aabf9f07de0156941ce3ed44b0/polkadot/runtime/kusama/src/weights/xcm/mod.rs#L94)
`// Kusama doesn't support ReserveAssetDeposited, so this benchmark has
a default weight` still relevant? Shouldnt be removed/changed?


## TODO

- [x] `bench bot` regenerate xcm weights for all runtimes
- [x] remove hard-coded stuff from system parachain weight files
- [ ] when merged, open `polkadot-fellow/runtimes` PR

## References

Fixes #1132
Closes #1132
Old polkadot repo [PR](https://github.com/paritytech/polkadot/pull/7546)

---------

Co-authored-by: command-bot <>

* restructure dispatch macro related exports

* moved Dispatchable to lib.rs

* fix .gitignore final newline

* ".git/.scripts/commands/fmt/fmt.sh"

* fix rustdocs

* wip

---------

Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: ordian <[email protected]>

* Rename squatted crates

This commit adds the staging- prefix to squatted crates so we can go forward and publish them to crates.io.

Using the staging- prefix is a temp fix until we decide on replacement names.
https://forum.parity.io/t/renaming-squated-crates-in-substrate-polkadot-cumulus/1964/6

* Fix test after crate renames

* Update Lockfile

* Fix build profiles

Closes https://github.com/paritytech/polkadot-sdk/issues/1155



Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Manually set version to 1.0.0

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use workspace repo

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* 'Authors and Edition from workspace

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Manually fix conflicting ?

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove duplicates

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Autofix feature propagation

zepter lint propagate-feature --feature try-runtime --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="try-runtime:frame-try-runtime"
zepter lint propagate-feature --feature runtime-benchmarks --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="runtime-benchmarks:frame-benchmarking"
zepter lint propagate-feature --feature std --left-side-feature-missing=ignore --workspace --fix
zepter f f

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump zepter

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add some duplicates

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Add some duplicates"

This reverts commit c6ce6272

.

* Remove default enabled features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump Zepter

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump in correct location 🤦



Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* DNM: Add some mistakes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* DNM: Add some mistakes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "DNM: Add some mistakes"

This reverts commit d469b3f0.

* Revert "DNM: Add some mistakes"

This reverts commit d892a73a

.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add CI for monorepo

* fix frame tests

* Format features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* add note for skipping tests and disable test-linux-stable-all

* Fix tests and compile issues (#1152)

* Fix feature dependant import

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump test timeout

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove feature gate

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add resolver 2

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove old lockfile

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Format features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix check-dependency-rules

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* rm test-runtime

Signed-o...

Signed-off-by: alvicsam <[email protected]>

* Use same rustfmt.toml as Substrate

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* format format file

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Format with new config

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add Substrate Clippy config

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Print Clippy version in CI

Otherwise its difficult to reproduce locally.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Make fmt happy

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update node/core/pvf/src/error.rs

Co-authored-by: Tsvetomir Dimitrov <[email protected]>

* Update node/core/pvf/src/error.rs

Co-authored-by: Tsvetomir Dimitrov <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Tsvetomir Dimitrov <[email protected]>

* Add license to crates

This is required to publish to crates.io

* Add more licenses

* Add polkadot XCM benchmarks

* Add temp

* ".git/.scripts/commands/bench/bench.sh" xcm polkadot pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm polkadot pallet_xcm_benchmarks::generic

* Add weights to XCM on Polkadot

* Make CI fail on old files

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update template

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add reserve_asset_deposited benchmark

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::generic

* Update weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Change initiate_reserve_deposit in runtime weights

* Update weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove trusted reserves from runtimes

* Fix pallet-xcm-benchmarks mock

* Fix test

* Change pallet xcm weigher in kusama

* Fix

* Remove merge conflict artifact

* Remove initiate_reserve_withdraw from generic benchmarks

* Add missing implementation to XCM benchmark

* Fix failing karura test

* Remove dbg!

Co-authored-by: Keith Yeung <[email protected]>

* Fix fmt

* Revert "Fix fmt"

This reverts commit 676f2d8db07d7427750c79f95494d4988d06fda5.

* Fix fmt

* Remove duplicated template code

* Add back part of the template

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible

* Don't skip reserve asset deposited benchmark

* Remove call to non-generated benchmark yet

* Underscore unused parameter

* Skip not supported benchmarks and hardcode value

* Remove ReserveAssetDeposited benchmark

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible

* Add back ReserveAssetDeposited

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm polkadot pallet_xcm_benchmarks::fungible

* Use default benchmark for ReserveAssetDeposited

* Add missing parameter

* Revert reserve asset deposited benchmark

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm kusama pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench-vm/bench-vm.sh" xcm westend pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::fungible

* Add 'real' benchmarks

* Add TrustedReserve to actual XcmConfig

* Add TrustedReserve to actual XcmConfig (fix)

* Whitelist from benchmarking XCM storage keys read each block (#6871)

* Whitelist from benchmarking XCM storage keys read each block

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime westend pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime rococo pallet_xcm

* Remove XcmPallet SupportedVersion from the benchmark whitelist

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime kusama pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime westend pallet_xcm

* ".git/.scripts/commands/bench/bench.sh" runtime rococo pallet_xcm

* WIP

* Add necessary traits, remove unnecessary whitelisted keys

* Fix tests

* Remove unused file

* Remove unused import

---------

Co-authored-by: command-bot <>

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm kusama pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm rococo pallet_xcm_benchmarks::fungible

* ".git/.scripts/commands/bench/bench.sh" xcm westend pallet_xcm_benchmarks::fungible

* Fix spellchecker issues

* Remove unused migration code

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>

* replace Index for Nonce

* remove extra Nonce

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

Moves `Block` to `frame_system` instead of `construct_runtime` and removes `Header` and `BlockNumber` (#7431)

* Companion for substrate

* Minor update

* Formatting

* Fixes for cumulus

* Fixes tests in polkadot-runtime-parachains

* Minor update

* Removes unused import

* Fixes tests in polkadot-runtime-common

* Minor fix

* Update roadmap/implementers-guide/src/runtime/configuration.md

Co-authored-by: ordian <[email protected]>

* ".git/.scripts/commands/fmt/fmt.sh"

* update lockfile for {"substrate"}

---------

Co-authored-by: ordian <[email protected]>
Co-authored-by: command-bot <>

* GenesisBuild<T,I> deprecated. BuildGenesisConfig added

* fmt

* fixes

* more fixes

* more fixes

* fixes

* update lockfile for {"substrate"}

* fix

---------

Co-authored-by: parity-processbot <>

* update genesis config

* fix wrong RuntimeGenesisConfig name

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>