This PR includes backport of the regular version bumps and `prdocs`
reordering from the `stable2412` branch back ro master

---------

Co-authored-by: ParityReleases <release-team@parity.io>
Co-authored-by: command-bot <>

Move `RelayChainDataFetcher` to its own module and rename it to
`RelayChainDataCache`. Also move the `core_selector` function to the
`slot_based` module.

Related issue: https://github.com/paritytech/polkadot-sdk/issues/6495

Relates to: https://github.com/paritytech/polkadot-sdk/issues/6918

---------

Co-authored-by: command-bot <>

Adds a new zombienet-sdk test which verifies that elastic scaling works
correctly both with the MVP and the new RFC 103 implementation which
sends the core selector as a UMP signal.

Also enables the V2 receipts node feature for testnet genesis config.

Part of https://github.com/paritytech/polkadot-sdk/issues/5049

---------

Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>

# Description

Opening this PR to add our bootnodes for the IBP. These nodes are
located in Santiago Chile, we own and manage the underlying hardware. If
you need any more information please let me know.


## Integration

```

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain asset-hub-westend \
  --reserved-only \
  --reserved-nodes "/dns/asset-hub-westend-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWDUPyF2q8b6fVFEuwxBbRV3coAy1kzuCPU3D9TRiLnUfE"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain bridge-hub-westend \
  --reserved-only \
  --reserved-nodes "/dns/bridge-hub-westend-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWJEfDZxrEKehoPbW2Mfg6rypttMXCMgMiybmapKqcByc1"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain collectives-westend \
  --reserved-only \
  --reserved-nodes "/dns/collectives-westend-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWFH7UZnWESzuRSgrLvNSfALjtpr9PmG7QGyRNCizWEHcd"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain people-westend \
  --reserved-only \
  --reserved-nodes "/dns/people-westend-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWJzL4R3kq9Ms88gsV6bS9zGT8DHySdqwau5SHNqTzToNM"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain coretime-westend \
  --reserved-only \
  --reserved-nodes "/dns/coretime-westend-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWCFNzjaiq45ZpW2qStmQdG5w7ZHrmi3RWUeG8cV2pPc2Y"


docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain asset-hub-kusama \
  --reserved-only \
  --reserved-nodes "/dns/asset-hub-kusama-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWNCg821LyWDVrAJ2mG6ScDeeBFuDPiJtLYc9jCGNCyMoq"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain bridge-hub-kusama \
  --reserved-only \
  --reserved-nodes "/dns/bridge-hub-kusama-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWBE1ZhrYqMC3ECFK6qbufS9kgKuF57XpvvZU6LKsPUSnF"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain coretime-kusama \
  --reserved-only \
  --reserved-nodes "/dns/coretime-kusama-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWMPc6jEjzFLRCK7QgbcNh3gvxCzGvDKhU4F66QWf2kZmq"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain people-kusama \
  --reserved-only \
  --reserved-nodes "/dns/people-kusama-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWN32MmhPgZN8e1Dmc8DzEUKsfC2hga3Lqekko4VWvrbhq"


docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain bridge-hub-polkadot \
  --reserved-only \
  --reserved-nodes "/dns/bridge-hub-polkadot-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWSBpo6fYU8CUr4fwA14CKSDUSj5jSgZzQDBNL1B8Dnmaw"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain collectives-polkadot \
  --reserved-only \
  --reserved-nodes "/dns/collectives-polkadot-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWNscpobBzjPEdjbbjjKRYh9j1whYJvagRJwb9UH68zCPC"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain people-polkadot \
  --reserved-only \
  --reserved-nodes "/dns/people-polkadot-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWDf2aLDKHQyLkDzdEGs6exNzWWw62s2EK9g1wrujJzRZt"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain coretime-polkadot \
  --reserved-only \
  --reserved-nodes "/dns/coretime-polkadot-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWFG9WQQTf3MX3YQypZjJtoJM5zCQgJcqYdxxTStsbhZGU"

docker run --platform=linux/amd64 --rm parity/polkadot-parachain \
  --base-path /tmp/polkadot-data \
  --no-hardware-benchmarks --no-mdns \
  --chain asset-hub-polkadot \
  --reserved-only \
  --reserved-nodes "/dns/asset-hub-polkadot-01.bootnode.stkd.io/tcp/30633/wss/p2p/12D3KooWJUhizuk3crSvpyKLGycHBtnP93rwjksVueveU6x6k6RY"

```

## Review Notes

None

Co-authored-by: Bastian Köcher <git@kchr.de>

#6450 introduced metadata checks. Supported are metadata v14 and higher.

However, of course old chain-specs have a genesis code blob that might
be on older version. This needs to be tolerated. We should just skip the
checks in that case.

Fixes #6921

---------

Co-authored-by: command-bot <>

# Description

Redundant dep that made its way in #6450 . 

. It can be
brought up when using `cargo udeps`. Added a github action that runs
`cargo udeps` on the repo too.

## Integration

N/A

## Review Notes

N/A

---------

Signed-off-by: Iulian Barbu <iulian.barbu@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

We check if there is a pending validation code in storage. If there is,
add the go-ahead signal in the relay chain storage proof.

Not super elegant, but should get the job done for development.

---------

Co-authored-by: command-bot <>

The `SlotBasedBlockImport` job is to collect the storage proofs of all
blocks getting imported. These storage proofs alongside the block are
being forwarded to the collation task. Right now they are just being
thrown away. More logic will follow later. Basically this will be
required to include multiple blocks into one `PoV` which will then be
done by the collation task.

---------

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>

Fix the broker pallet auto-renew benchmarks which have been broken since
#4424, yielding `Weightless` due to some prices being set too low, as
reported in #6474.

Upon further investigation it turned out that the auto-renew
contribution to `rotate_sale` was always failing but the error was
mapped. This is also fixed at the cost of a bit of setup overhead.

Fixes #6474

TODO:
- [x] Re-run weights

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: command-bot <>

# Description

Get runtime's metadata, parse it and verify pallets list for a pallet
named `ParachainSystem` (for now), and block number to be the same for
both node and runtime. Ideally we'll add other pallets checks too, at
least a small set of pallets we think right away as mandatory for
parachain compatibility.
Closes: #5565 

## Integration

Runtime devs must be made aware that to be fully compatible with Omni
Node, certain naming conventions should be respected when defining
pallets (e.g we verify parachain-system pallet existence by searching
for a pallet with `name` `ParachainSystem` in runtime's metadata). Not
finding such a pallet will not influence the functionality yet, but by
doing these checks we could provide useful feedback for runtimes that
are clearly not implementing what's required for full parachain
compatibility with Omni Node.

## Review Notes

- [x] parachain system check
- [x] check frame_system's metadata to ensure the block number in there
is the same as the one in the node side
- [x] add tests for the previous checking logic
- [x] update omni node polkadot-sdk docs to make these conventions
visible.
- [ ] add more pallets checks?

---------

Signed-off-by: Iulian Barbu <iulian.barbu@parity.io>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

`InitiateTransfer`, the new instruction introduced in XCMv5, allows
preserving the origin after a cross-chain transfer via the usage of the
`AliasOrigin` instruction. The receiving chain needs to be configured to
allow such this instruction to have its intended effect and not just
throw an error.

In this PR, I add the alias rules specified in the [RFC for origin
preservation](https://github.com/polkadot-fellows/RFCs/blob/main/text/0122-alias-origin-on-asset-transfers.md)
to westend chains so we can test these scenarios in the testnet.

The new scenarios include:
- Sending a cross-chain transfer from one system chain to another and
doing a Transact on the same message (1 hop)
- Sending a reserve asset transfer from one chain to another going
through asset hub and doing Transact on the same message (2 hops)

The updated chains are:
- Relay: added `AliasChildLocation`
- Collectives: added `AliasChildLocation` and
`AliasOriginRootUsingFilter<AssetHubLocation, Everything>`
- People: added `AliasChildLocation` and
`AliasOriginRootUsingFilter<AssetHubLocation, Everything>`
- Coretime: added `AliasChildLocation` and
`AliasOriginRootUsingFilter<AssetHubLocation, Everything>`

AssetHub already has `AliasChildLocation` and doesn't need the other
config item.
BridgeHub is not intended to be used by end users so I didn't add any
config item.
Only added `AliasChildOrigin` to the relay since we intend for it to be
used less.

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: command-bot <>

# Description

This PR changes a few things:
* `--dev` flag will not conflict with `--chain` anymore, but if
`--chain` is not given will set `--chain=dev`.
* `--dev-block-time` is optional and it defaults to 3000ms if not set
after setting `--dev`.
* to start OmniNode with manual seal it is enough to pass just `--dev`.
* `--dev-block-time` can still be used to start a node with manual seal,
but it will not set it up as `--dev` does (it will not set a bunch of
flags which are enabled by default when `--dev` is set: e.g. `--tmp`,
`--alice` and `--force-authoring`.

Closes: #6537

## Integration

Relevant for node/runtime developers that use OmniNode lib, including
`polkadot-omni-node` binary, although the recommended way for runtime
development is to use `chopsticks`.

## Review Notes

* Decided to focus only on OmniNode & templates docs in relation to it,
and leave the `parachain-template-node` as is (meaning `--dev` isn't
usable and testing a runtime with the `parachain-template-node` still
needs a relay chain here). I am doing this because I think we want
either way to phase out `parachain-template-node` and adding manual seal
support for it is wasted effort. We might add support though if the
demand is for `parachain-template-node`.
* Decided to not infer the block time based on AURA config yet because
there is still the option of setting a specific block time by using
`--dev-block-time`. Also, would want first to align & merge on runtime
metadata checks we added in Omni Node here:
https://github.com/paritytech/polkadot-sdk/pull/6450 before starting to
infer AURA config slot duration via the same way.

- [x] update the docs to mention `--dev` now.
- [x] mention about chopsticks in the context of runtime development

---------

Signed-off-by: Iulian Barbu <iulian.barbu@parity.io>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

Close: #5858

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Closes: https://github.com/paritytech/polkadot-sdk/issues/5551

## Description

With [permissionless lanes
PR#4949](https://github.com/paritytech/polkadot-sdk/pull/4949), the
congestion mechanism based on sending
`Transact(report_bridge_status(is_congested))` from
`pallet-xcm-bridge-hub` to `pallet-xcm-bridge-hub-router` was replaced
with a congestion mechanism that relied on monitoring XCMP queues.
However, this approach could cause issues, such as suspending the entire
XCMP queue instead of isolating the affected bridge. This PR reverts
back to using `report_bridge_status` as before.

## TODO
- [x] benchmarks
- [x] prdoc

## Follow-up

https://github.com/paritytech/polkadot-sdk/pull/6231

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: command-bot <>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

# Description

Sending XCM messages to other chains requires paying a "transport fee".
This can be paid either:
- from `origin` local account if `jit_withdraw = true`,
- taken from Holding register otherwise.

This currently works for following hops/scenarios:
1. On destination no transport fee needed (only sending costs, not
receiving),
2. Local/originating chain: just set JIT=true and fee will be paid from
signed account,
3. Intermediary hops - only if intermediary is acting as reserve between
two untrusted chains (aka only for `DepositReserveAsset` instruction) -
this was fixed in https://github.com/paritytech/polkadot-sdk/pull/3142

But now we're seeing more complex asset transfers that are mixing
reserve transfers with teleports depending on the involved chains.

# Example

E.g. transferring DOT between Relay and parachain, but through AH (using
AH instead of the Relay chain as parachain's DOT reserve).

In the `Parachain --1--> AssetHub --2--> Relay` scenario, DOT has to be
reserve-withdrawn in leg `1`, then teleported in leg `2`.
On the intermediary hop (AssetHub), `InitiateTeleport` fails to send
onward message because of missing transport fees. We also can't rely on
`jit_withdraw` because the original origin is lost on the way, and even
if it weren't we can't rely on the user having funded accounts on each
hop along the way.

# Solution/Changes

- Charge the transport fee in the executor from the transferred assets
(if available),
- Only charge from transferred assets if JIT_WITHDRAW was not set,
- Only charge from transferred assets if unless using XCMv5 `PayFees`
where we do not have this problem.

# Testing

Added regression tests in emulated transfers.

Fixes https://github.com/paritytech/polkadot-sdk/issues/4832
Fixes https://github.com/paritytech/polkadot-sdk/issues/6637

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>

Last feature we wanted for V5, changing `SetAssetClaimer` to be just one
of many possible "hints" that you can specify at the beginning of your
program to change its behaviour.

This makes it easier to add new hints in the future and have barriers
accept them.

---------

Co-authored-by: GitHub Action <action@github.com>

Closes: https://github.com/paritytech/polkadot-sdk/issues/6585

Removing the `require_weight_at_most` parameter in V5 Transact had only
one problem. Converting a message from V5 to V4 to send to chains that
didn't upgrade yet. The conversion would not know what weight to give to
the Transact, since V4 and below require it.

To fix this, I added back the weight in the form of an `Option<Weight>`
called `fallback_max_weight`. This can be set to `None` if you don't
intend to deal with a chain that hasn't upgraded yet. If you set it to
`Some(_)`, the behaviour is the same. The plan is to totally remove this
in V6 since there will be a good conversion path from V6 to V5.

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

Bump Asset-Hub westend spec version

---------

Co-authored-by: GitHub Action <action@github.com>

* Add a bunch of differential tests to ensure that responses from
eth-rpc matches the one from `geth`
- These
[tests](https://github.com/paritytech/polkadot-sdk/blob/pg/fix-geth-diff/substrate/frame/revive/rpc/examples/js/src/geth-diff.test.ts)
are not run in CI for now but can be run locally with
```bash
cd revive/rpc/examples/js
bun test
```

* EVM RPC server will not fail gas_estimation if no gas is specified, I
updated pallet-revive to add an extra `skip_transfer` boolean check to
replicate this behavior in our pallet

* `eth_transact` and `bare_eth_transact` api have been updated to use
`GenericTransaction` directly as this is what is used by
`eth_estimateGas` and `eth_call`

## TODO

- [ ]  Add tests the new `skip_transfer` flag

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

# Description

Made as a follow-up of
https://github.com/polkadot-fellows/runtimes/pull/499

## Integration

N/A

## Review Notes

N/A

---------

Co-authored-by: Dónal Murray <donal.murray@parity.io>

The `query_weight_to_asset_fee` function was trying to convert versions
by using `try_as`, this function [doesn't convert from a versioned to a
concrete
type](https://github.com/paritytech/polkadot-sdk/blob/0156ca8f/polkadot/xcm/src/lib.rs#L131).
This would cause all calls with a lower version to fail.

The correct function to use is the good old
[try_into](https://github.com/paritytech/polkadot-sdk/blob/0156ca8f

/polkadot/xcm/src/lib.rs#L184).
Now those calls work :)

---------

Co-authored-by: command-bot <>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: GitHub Action <action@github.com>

Before this was done for every imported transaction. When a lot of
transactions got imported, the import notification channel was filled.
The underlying problem was that the `status` call is read locking the
`validated_pool` which will be write locked by the internal submitting
logic. Thus, the submitting and status reading was interferring which
each other.

---------

Co-authored-by: GitHub Action <action@github.com>

Step in https://github.com/paritytech/polkadot-sdk/issues/3268

This PR adds the ability for these pallets to specify their source of
the block number. This is useful when these pallets are migrated from
the relay chain to a parachain and vice versa.

This change is backwards compatible:
1. If the `BlockNumberProvider` continues to use the system pallet's
block number
2. When a pallet deployed on the relay chain is moved to a parachain,
but still uses the relay chain's block number

However, we would need migrations if the deployed pallets are upgraded
on an existing parachain, and the `BlockNumberProvider` uses the relay
chain block number.

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Move spawning of the slot-based collator into the `run` function. Also
the tasks are being spawned as blocking task and not just as normal
tasks.

---------

Co-authored-by: GitHub Action <action@github.com>

This PR includes:  
- Refactored integrity tests to support standalone deployment of
`pallet-bridge-messages`.
- Refactored the `open_and_close_bridge_works` test case to support
multiple scenarios, such as:
  1. A local chain opening a bridge.  
  2. Sibling parachains opening a bridge.  
  3. The relay chain opening a bridge.  
- Previously, we added instance support for `pallet-bridge-relayer` but
overlooked updating the `DeliveryConfirmationPaymentsAdapter`.

---------

Co-authored-by: GitHub Action <action@github.com>

Aims to implement Stage 3 of Validator Disbling as outlined here:
https://github.com/paritytech/polkadot-sdk/issues/4359

Features:
- [x] New Disabling Strategy (Staking level)
- [x] Re-enabling logic (Session level)
- [x] More generic disabling decision output
- [x] New Disabling Events

Testing & Security:
- [x] Unit tests
- [x] Mock tests
- [x] Try-runtime checks
- [x] Try-runtime tested on westend snap
- [x] Try-runtime CI tests
- [ ] Re-enabling Zombienet Test (?)
- [ ] SRLabs Audit

Closes #4745 
Closes #2418

---------

Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>

This PR update the pallet to use the EVM 18 decimal balance in contracts
call and host functions instead of the native balance.

It also updates the js example to add the piggy-bank solidity contract
that expose the problem

---------

Co-authored-by: GitHub Action <action@github.com>

Follow up to https://github.com/paritytech/polkadot-sdk/pull/3685
Partially fixes https://github.com/paritytech/polkadot-sdk/issues/6403

The main PR introduced bare support for the new extension version byte
as well as extension weights and benchmarking.

This PR:
- Removes the redundant extension version byte from the signed v4
extrinsic, previously unused and defaulted to 0.
- Adds the extension version byte to the inherited implication passed to
`General` transactions.
- Whitelists the `pallet_authorship::Author`, `frame_system::Digest` and
`pallet_transaction_payment::NextFeeMultiplier` storage items as they
are read multiple times by extensions for each transaction, but are hot
in memory and currently overestimate the weight.
- Whitelists the benchmark caller for `CheckEra` and `CheckGenesis` as
the reads are performed for every transaction and overestimate the
weight.
- Updates the umbrella frame weight template to work with the system
extension changes.
- Plans on re-running the benchmarks at least for the `frame_system`
extensions.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: gui <gui.thiolliere@gmail.com>

Added `ExecuteWithOrigin` instruction according to the old XCM RFC 38:
https://github.com/polkadot-fellows/xcm-format/pull/38.

This instruction allows you to descend or clear while going back again.

## TODO
- [x] Implementation
- [x] Unit tests
- [x] Integration tests
- [x] Benchmarks
- [x] PRDoc

## Future work

Modify `WithComputedOrigin` barrier to allow, for example, fees to be
paid with a descendant origin using this instruction.

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Andrii <ndk@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Joseph Zhao <65984904+programskillforverification@users.noreply.github.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: command-bot <>

- Fix bare_eth_transact so that it estimate more precisely the
transaction fee
- Add some context to the build.rs to make it easier to troubleshoot
errors
- Add TransactionBuilder for the RPC tests.
- Improve error message, proxy rpc error from the node and handle
reverted error message
- Add logs in ReceiptInfo

---------

Co-authored-by: GitHub Action <action@github.com>

Based on https://github.com/paritytech/polkadot-sdk/pull/3331
Related to https://github.com/paritytech/polkadot-sdk/issues/3268

Implements migrations with customizable block number to relay height
number translation function.

Adds block to relay height migration code for rococo and westend.

---------

Co-authored-by: DavidK <davidk@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Closes #2082

change send xcm to use `xcm::executor::FeeManager` to determine if the
sender should be charged.

I had to change the `FeeManager` of the penpal config to ensure the same
test behaviour as before. For the other tests, I'm using the
`FeeManager` from the `xcm::executor::FeeManager` as this one is used to
check if the fee can be waived on the charge fees method.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: GitHub Action <action@github.com>

# Description

This seems to be an old artifact of the long closed
https://github.com/paritytech/substrate/issues/6827 that I noticed when
working on related code earlier.

## Integration

`NetworkStarter` was removed, simply remove its usage:
```diff
-let (network, system_rpc_tx, tx_handler_controller, start_network, sync_service) =
+let (network, system_rpc_tx, tx_handler_controller, sync_service) =
    build_network(BuildNetworkParams {
...
-start_network.start_network();
```

## Review Notes

Changes are trivial, the only reason for this to not be accepted is if
it is desired to not start network automatically for whatever reason, in
which case the description of network starter needs to change.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

This PR introduces usage of `build_struct_json_patch` macro in all
runtimes (also guides) within the code base.

It also fixes macro to support _field init shorthand_, and _Struct
Update_ syntax which were missing in original implementation.

Follow up of #5700 and #5813

# Description

This is a continuation of
https://github.com/paritytech/polkadot-sdk/pull/5666 that finally fixes
https://github.com/paritytech/polkadot-sdk/issues/5333.

This should allow developers to create custom syncing strategies or even
the whole syncing engine if they so desire. It also moved syncing engine
creation and addition of corresponding protocol outside
`build_network_advanced` method, which is something Bastian expressed as
desired in
https://github.com/paritytech/polkadot-sdk/issues/5#issuecomment-1700816458

Here I replaced strategy-specific types and methods in `SyncingStrategy`
trait with generic ones. Specifically `SyncingAction` is now used by all
strategies instead of strategy-specific types with conversions.
`StrategyKey` was an enum with a fixed set of options and now replaced
with an opaque type that strategies create privately and send to upper
layers as an opaque type. Requests and responses are now handled in a
generic way regardless of the strategy, which reduced and simplified
strategy API.

`PolkadotSyncingStrategy` now lives in its dedicated module (had to edit
.gitignore for this) like other strategies.

`build_network_advanced` takes generic `SyncingService` as an argument
alongside with a few other low-level types (that can probably be
extracted in the future as well) without any notion of specifics of the
way syncing is actually done. All the protocol and tasks are created
outside and not a part of the network anymore. It still adds a bunch of
protocols like for light client and some others that should eventually
be restructured making `build_network_advanced` just building generic
network and not application-specific protocols handling.

## Integration

Just like https://github.com/paritytech/polkadot-sdk/pull/5666
introduced `build_polkadot_syncing_strategy`, this PR introduces
`build_default_block_downloader`, but for convenience and to avoid
typical boilerplate a simpler high-level function
`build_default_syncing_engine` is added that will take care of creating
typical block downloader, syncing strategy and syncing engine, which is
what most users will be using going forward. `build_network` towards the
end of the PR was renamed to `build_network_advanced` and
`build_network`'s API was reverted to
pre-https://github.com/paritytech/polkadot-sdk/pull/5666, so most users
will not see much of a difference during upgrade unless they opt-in to
use new API.

## Review Notes

For `StrategyKey` I was thinking about using something like private type
and then storing `TypeId` inside instead of a static string in it, let
me know if that would preferred.

The biggest change happened to requests that different strategies make
and how their responses are handled. The most annoying thing here is
that block response decoding, in contrast to all other responses, is
dependent on request. This meant request had to be sent throughout the
system. While originally `Response` was `Vec<u8>`, I didn't want to
re-encode/decode request and response just to fit into that API, so I
ended up with `Box<dyn Any + Send>`. This allows responses to be truly
generic and each strategy will know how to downcast it back to the
concrete type when handling the response.

Import queue refactoring was needed to move `SyncingEngine` construction
out of `build_network` that awkwardly implemented for `SyncingService`,
but due to `&mut self` wasn't usable on `Arc<SyncingService>` for no
good reason. `Arc<SyncingService>` itself is of course useless, but
refactoring to replace it with just `SyncingService` was unfortunately
rejected in https://github.com/paritytech/polkadot-sdk/pull/5454

As usual I recommend to review this PR as a series of commits instead of
as the final diff, it'll make more sense that way.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [x] I have made corresponding changes to the documentation (if
applicable)

# Context

This PR aims to introduce XCMv5, for now it's in progress and will be
updated over time.
This branch will serve as a milestone branch for merging in all features
we want to add to XCM, roughly outlined
[here](https://github.com/polkadot-fellows/xcm-format/issues/60).
More features could be added.

## TODO
- [x] Migrate foreign assets from v3 to v4
- [x] Setup v5 skeleton
- [x] Remove XCMv2
- [x] https://github.com/paritytech/polkadot-sdk/pull/5390
- [x] https://github.com/paritytech/polkadot-sdk/pull/5585
- [x] https://github.com/paritytech/polkadot-sdk/pull/5420
- [x] https://github.com/paritytech/polkadot-sdk/pull/5876
- [x] https://github.com/paritytech/polkadot-sdk/pull/5971
- [x] https://github.com/paritytech/polkadot-sdk/pull/6148
- [x] https://github.com/paritytech/polkadot-sdk/pull/6228

Fixes #3434 
Fixes https://github.com/paritytech/polkadot-sdk/issues/4190
Fixes https://github.com/paritytech/polkadot-sdk/issues/5209
Fixes https://github.com/paritytech/polkadot-sdk/issues/5241
Fixes https://github.com/paritytech/polkadot-sdk/issues/4284

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Andrii <ndk@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Joseph Zhao <65984904+programskillforverification@users.noreply.github.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Serban Iorga <serban@parity.io>