Partially implements
https://github.com/paritytech/polkadot-sdk/issues/5048

- adds a core selection runtime API to cumulus and a generic way of
configuring it for a parachain
- modifies the slot based collator to utilise the claim queue and the
generic core selection

What's left to be implemented (in a follow-up PR):
- add the UMP signal for core selection into the parachain-system pallet

View the RFC for more context:
https://github.com/polkadot-fellows/RFCs/pull/103

---------

Co-authored-by: command-bot <>

Use `sp_core::Bytes` as `payload` to encode the values correctly as
`hex` string.

It is a first step for switching to the `frame-omni-bencher` for CI.

This PR includes several changes related to generating chain specs plus:

- [x] pallet `assigned_slots` fix missing `#[serde(skip)]` for phantom
- [x] pallet `paras_inherent` benchmark fix - cherry-picked from
https://github.com/paritytech/polkadot-sdk/pull/5688
- [x] migrates `get_preset` to the relevant runtimes
- [x] fixes Rococo genesis presets - does not work
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7317249
- [x] fixes Rococo benchmarks for CI 
- [x] migrate westend genesis
- [x] remove wococo stuff

Closes: https://github.com/paritytech/polkadot-sdk/issues/5680

## Follow-ups
- Fix for frame-omni-bencher
https://github.com/paritytech/polkadot-sdk/pull/5655
- Enable new short-benchmarking CI -
https://github.com/paritytech/polkadot-sdk/pull/5706
- Remove gitlab pipelines for short benchmarking
- refactor all Cumulus runtimes to use `get_preset` -
https://github.com/paritytech/polkadot-sdk/issues/5704
- https://github.com/paritytech/polkadot-sdk/issues/5705
- https://github.com/paritytech/polkadot-sdk/issues/5700
- [ ] Backport to the stable

---------

Co-authored-by: command-bot <>
Co-authored-by: ordian <noreply@reusable.software>

When using the relay chain though a `Arc<dyn RelayChainInterface>` there
is no way to call arbitrary runtime apis. Both implementations of that
trait allow this, so it feels natural to expose this functionality in
the trait.

This PR adds a `call_runtime_api` method to RelayChainInterface trait,
and a separate function also named `call_runtime_api` which allows the
caller to specify the input and output types, as opposed to having to
encode them. This generic function cannot be part of the trait because a
`dyn Trait` object cannot have generic methods.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

# Description

Fix https://github.com/paritytech/polkadot-sdk/pull/5074 which missed
the runtime migration to initialize channels of the bridge.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

# Description

When we start a node with connections to external RPC servers (as a
minimal node), we lack metrics around how many individual calls we're
doing to the remote RPC servers and their duration. This PR adds metrics
that measure durations of each RPC call made by the minimal nodes, and
implicitly how many calls there are.

Closes #5409 
Closes #5689

## Integration

Node operators should be able to track minimal node metrics and decide
appropriate actions according to how the metrics are interpreted/felt.
The added metrics can be observed by curl'ing the prometheus metrics
endpoint for the ~relaychain~ parachain (it was changed based on the
review). The metrics are represented by
~`polkadot_parachain_relay_chain_rpc_interface`~
`relay_chain_rpc_interface` namespace (I realized lining up
`parachain_relay_chain` in the same metric might be confusing :).
Excerpt from the curl:

```
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="0.001"} 15
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="0.004"} 23
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="0.016"} 23
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="0.064"} 23
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="0.256"} 24
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="1.024"} 24
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="4.096"} 24
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="16.384"} 24
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="65.536"} 24
relay_chain_rpc_interface_bucket{method="chain_getBlockHash",chain="rococo_local_testnet",le="+Inf"} 24
relay_chain_rpc_interface_sum{method="chain_getBlockHash",chain="rococo_local_testnet"} 0.11719075
relay_chain_rpc_interface_count{method="chain_getBlockHash",chain="rococo_local_testnet"} 24
```

## Review Notes

The way we measure durations/hits is based on `HistogramVec` struct
which allows us to collect timings for each RPC client method called
from the minimal node., It can be extended to measure the RPCs against
other dimensions too (status codes, response sizes, etc). The timing
measuring is done at the level of the `relay-chain-rpc-interface`, in
the `RelayChainRpcClient` struct's method 'request_tracing'. A single
entry point for all RPC requests done through the
relay-chain-rpc-interface. The requests durations will fall under
exponential buckets described by start `0.001`, factor `4` and count
`9`.

---------

Signed-off-by: Iulian Barbu <iulian.barbu@parity.io>

When running XCM emulated tests and seeing the logs with `RUST_LOG=xcm`
or `RUST_LOG=xcm=trace`, it's sometimes a bit hard to figure out the
chain where the logs are coming from.

I added a log whenever `execute_with` is called, to know the chain which
makes the following logs. Looks like so:

<img width="1499" alt="Screenshot 2024-09-13 at 20 14 13"
src="https://github.com/user-attachments/assets/a31d7aa4-11d1-4d3e-9a65-86f38347c880">

There are already log targets for when UMP, DMP and HRMP messages are
being processed. To see them, you have to use the log targets `ump`,
`dmp`, and `hrmp` respectively. So `RUST_LOG=xcm,ump,dmp,hrmp` would let
you see every log.
I prefixed the targets with `xcm::` so you can get all the relevant logs
just by filtering by `xcm`. You can always use the whole target to see
just the messages being processed.

These logs showed the message as an array of bytes, I made them show a
hexadecimal string instead since that's easier to copy in case you want
to decode it or use it in another tool. They look like this now:

<img width="1499" alt="Screenshot 2024-09-13 at 20 17 15"
src="https://github.com/user-attachments/assets/5abf4a97-1ea7-4832-b3b0-d54c54905d1a">

The HRMP and UMP ones are very similar.

# Description

Follow-up to https://github.com/paritytech/polkadot-sdk/pull/5469 and
mostly covering https://github.com/paritytech/polkadot-sdk/issues/5333.

The primary change here is that syncing strategy is no longer created
inside of syncing engine, instead syncing strategy is an argument of
syncing engine, more specifically it is an argument to `build_network`
that most downstream users will use. This also extracts addition of
request-response protocols outside of network construction, making sure
they are physically not present when they don't need to be (imagine
syncing strategy that uses none of Substrate's protocols in its
implementation for example).

This technically allows to completely replace syncing strategy with
whatever strategy chain might need.

There will be at least one follow-up PR that will simplify
`SyncingStrategy` trait and other public interfaces to remove mentions
of block/state/warp sync requests, replacing them with generic APIs,
such that strategies where warp sync is not applicable don't have to
provide dummy method implementations, etc.

## Integration

Downstream projects will have to write a bit of boilerplate calling
`build_polkadot_syncing_strategy` function to create previously default
syncing strategy.

## Review Notes

Please review PR through individual commits rather than the final diff,
it will be easier that way. The changes are mostly just moving code
around one step at a time.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [x] I have made corresponding changes to the documentation (if
applicable)

- This will ensure that a correct product name
(Polkadot/Cumulus/Substrate) is referenced in license headers.
- Closes https://github.com/paritytech/license-scanner/issues/49

Setup auto incremented asset id to `50_000_000` for trust backed assets.

In order to align with Polkadot/Kusama Asset Hub -
https://github.com/polkadot-fellows/runtimes/pull/414
The next closes existing assets IDs in Rococo is `69_696_969`, in
Westend is `88_228_866`.

### Migration
**Stakeholders**: all clients providing asset creation functionality on
Westend/Rococo Asset Hub

This change does not break the API but introduces a new constraint. It
implements an auto-incremented ID strategy for Trust-Backed Assets (50
pallet instance indexes on both networks), starting at ID 50,000,000.
Each new asset must be created with an ID that is one greater than the
last asset created. The next ID can be fetched from the `NextAssetId`
storage item of the assets pallet. An empty `NextAssetId` storage item
indicates no constraint on the next asset ID and can serve as a feature
flag for this release.

# Description

Adding support for send polkadot native assets(PNA) to Ethereum network
through snowbridge. Asset with location in view of AH Including:

- Relay token `(1,Here)`
- Native asset `(0,[PalletInstance(instance),GenereIndex(index)])`
managed by Assets Pallet
- Native asset of Parachain `(1,[Parachain(paraId)])` managed by Foreign
Assets Pallet

The original PR in https://github.com/Snowfork/polkadot-sdk/pull/128
which has been internally reviewed by Snowbridge team.

# Notes

- This feature depends on the companion solidity change in
https://github.com/Snowfork/snowbridge/pull/1155. Currently register PNA
is only allowed from
[sudo](https://github.com/Snowfork/polkadot-sdk/blob/46cb3528

/bridges/snowbridge/pallets/system/src/lib.rs#L621),
so it's actually not enabled. Will require another runtime upgrade to
make the call permissionless together with upgrading the Gateway
contract.

- To make things easy multi-hop transfer(i.e. sending PNA from Ethereum
through AH to Destination chain) is not support ed in this PR. For this
case user can switch to 2-phases transfer instead.

---------

Co-authored-by: Clara van Staden <claravanstaden64@gmail.com>
Co-authored-by: Alistair Singh <alistair.singh7@gmail.com>
Co-authored-by: Vincent Geddes <117534+vgeddes@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

Add emulated test cases for the coretime chain.

This tests the calls sent across the `CoretimeInterface` and ensures the
weights are sufficient.

Proxies are possible in the runtimes for Kusama and Polkadot but this
functionality was not previously available on testnets.

Closes #5453.

Proxies can now be used on `coretime-rococo`, `coretime-westend`,
`people-rococo` and `people-westend` in the same way as they can be on
Kusama and Polkadot chains.

The exact same proxies are configured as the production runtimes for the
respective system parachains.

This PR is a pure refactoring by consolidating all the RPC related
parameters into a dedicated `RpcParams` struct, allowing us at subcoin
to build a custom run cmd without the need to duplicate code.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

Update `RuntimeVerison` type and use `system_version` to derive extrinsics root `StateVersion` instead of `V0` (#4257)

This PR 
- Renames `RuntimeVersion::state_version` to `system_version`
- Uses `Runtime::system_version` to derive extrinsics root
`StateVersion` instead of default `StateVersion::V0`

This PR should not be breaking any existing chains so long as they use
same `RuntimeVersion::state_version` for `Runtime::system_version`

Using `RuntimeVersion::system_version = 2` will make the extrinsics root
to use `StateVersion::V1` instead of `V0`

RFC for this change - https://github.com/polkadot-fellows/RFCs/pull/42

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>

When inspecting the logs we often encounter the following message:

`Benchmarked storage weight smaller than consumed storage weight.
benchmarked: {benchmarked_weight} consumed: {consumed_weight} unspent:
{unspent}`

However, it is very hard to guess which call is causing the issue.

With the changes proposed in this PR, information about the call is
provided so that we can easily identify the source of the problem
without further delay, and this way work more efficiently in solving the
issue.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Fixes: https://github.com/paritytech/polkadot-sdk/issues/5122.

This PR extends the existing single core `benchmark_cpu` to also build a
score of the entire processor by spawning `EXPECTED_NUM_CORES(8)`
threads and averaging their throughput.

This is better than simply checking the number of cores, because also
covers multi-tenant environments where the OS sees a high number of
available CPUs, but because it has to share it with the rest of his
neighbours its total throughput does not satisfy the minimum
requirements.


## TODO
- [x] Obtain reference values on the reference hardware.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

The dry-run shows in `forwarded_xcms` all the messages in the queues at
the time of calling the API.
Each time the API is called, the result could be different.
You could get messages even if you dry-run something that doesn't send a
message, like a `System::remark`.

This PR fixes this by clearing the message queues before doing the
dry-run, so the only messages left are the ones the users of the API
actually care about.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

Introduce a dynamic proposal deposit mechanism influenced by the total
number of active proposals, with the option to set the deposit to none.

The potential cost (e.g., balance hold) for proposal submission and
storage is determined by the implementation of the `Consideration`
trait. The footprint is defined as `proposal_count`, representing the
total number of active proposals in the system, excluding the one
currently being proposed. This cost may vary based on the proposal
count. The pallet also offers various types to define a cost strategy
based on the number of proposals.

Two new calls are introduced:
- kill(origin, proposal_hash): the cancellation of a proposal,
accompanied by the burning of the associated cost/consideration ticket.
- release_proposal_cost(origin, proposal_hash): the release of the cost
for a non-active proposal.

Additionally change: 
- benchmarks have been upgraded to benchmarks::v2 for collective pallet;
- `ensure_successful` function added to the `Consideration` under
`runtime-benchmarks` feature.

---------

Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: GitHub Action <action@github.com>

# Description

Adds retry logic that makes the RPC relay chain interface more reliable
for the cases of a collator connecting to external RPC servers.

Closes #5514 
Closes #4278

Final solution still debated on #5514 , what this PR addresses might
change (e.g. #4278 might require a more advanced approach).

## Integration

Users that start collators should barely observe differences based on
this logic, since the retry logic applies only in case the collators
fail to connect to the RPC servers. In practice I assume the RPC servers
are already live before starting collators, and the issue isn't visible.

## Review Notes

The added retry logic is for retrying the connection to the RPC servers
(which can be multiple). It is at the level of the
cumulus/client/relay-chain-rpc-interface module, but more specifically
relevant to the RPC clients logic (`ClientManager`). The retry logic is
not configurable, it tries to connect to the RPC client for 5 times,
with an exponential backoff in between each iteration starting with 1
second wait time and ending with 16 seconds. The same logic is applied
in case an existing connection to an RPC is dropped. There is a
`ReconnectingWebsocketWorker` who ensures there is connectivity to at
least on RPC node, and the retry logic makes this stronger by insisting
on trying connections to the RPC servers list for 5 times.

## Testing

- This was tested manually by starting zombienet natively based on
[006-rpc_collator_builds_blocks.toml](https://github.com/paritytech/polkadot-sdk/blob/master/cumulus/zombienet/tests/0006-rpc_collator_builds_blocks.toml)
and observing collators don't fail anymore:

```bash
zombienet -l text --dir zbn-run -f --provider native spawn polkadot-sdk/cumulus/zombienet/tests/0006-rpc_collator_builds_blocks.toml
```

- Added a unit test that exercises the retry logic for a client
connection to a server that comes online in 10 seconds. The retry logic
can wait for as long as 30 seconds, but thought that it is too much for
a unit test. Just being conscious of CI time if it runs this test, but I
am happy to see suggestions around it too. I am not that sure either it
runs in CI, haven't figured it out entirely yet. The test can be
considered an integration test too, but it exercises crate internal
implementation, not the public API.

Collators example logs after the change:
```
2024-08-29 14:28:11.730  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=0 index=2 url="ws://127.0.0.1:37427/"
2024-08-29 14:28:12.737  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=1 index=0 url="ws://127.0.0.1:43617/"
2024-08-29 14:28:12.739  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=1 index=1 url="ws://127.0.0.1:37965/"
2024-08-29 14:28:12.755  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=1 index=2 url="ws://127.0.0.1:37427/"
2024-08-29 14:28:14.758  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=2 index=0 url="ws://127.0.0.1:43617/"
2024-08-29 14:28:14.759  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=2 index=1 url="ws://127.0.0.1:37965/"
2024-08-29 14:28:14.760  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=2 index=2 url="ws://127.0.0.1:37427/"
2024-08-29 14:28:18.766  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=3 index=0 url="ws://127.0.0.1:43617/"
2024-08-29 14:28:18.768  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=3 index=1 url="ws://127.0.0.1:37965/"
2024-08-29 14:28:18.768  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=3 index=2 url="ws://127.0.0.1:37427/"
2024-08-29 14:28:26.770  INFO tokio-runtime-worker reconnecting-websocket-client: [Parachain] Trying to connect to next external relaychain node. current_iteration=4 index=0 url="ws://127.0.0.1:43617/"
```

---------

Signed-off-by: Iulian Barbu <iulian.barbu@parity.io>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

Allow free Snowbridge consensus updates, if the header interval is
larger than the configured value (set to 32, so once a epoch).

This PR also moves the Rococo Snowbridge pallet config into its own
module.

Original PR: https://github.com/Snowfork/polkadot-sdk/pull/159

---------

Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>

closes https://github.com/paritytech/polkadot-sdk/issues/5044

This PR switches the runtime to the new receipts format (vstaging
primitives). I've implemented `From` to convert from new primitives to
`v7` primitives and used them in the node runtime api client
implementation. Until we implement the support in the node, it will
continue e to use the v7 primitives but the runtime apis already use the
new primitives.


An expected downside of RFC103 is decoding V2 receipts shows garbage
values if the input is V1:

_![ima_9ce77de](https://github.com/user-attachments/assets/71d80e78-e238-4518-8cd1-548ae0d74b70)_

TODO:
- [x] fix tests
- [x] A few more tests for the new primitives
- [x] PRDoc

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Relates to:
https://github.com/paritytech/parity-bridges-common/issues/2451
Closes: https://github.com/paritytech/parity-bridges-common/issues/2500

## Summary

Now, the bridging pallet supports only static lanes, which means lanes
that are hard-coded in the runtime files. This PR fixes that and adds
support for dynamic, also known as permissionless, lanes. This means
that allowed origins (relay chain, sibling parachains) can open and
close bridges (through BridgeHubs) with another bridged (substrate-like)
consensus using just `xcm::Transact` and `OriginKind::Xcm`.

_This PR is based on the migrated code from the Bridges V2
[branch](https://github.com/paritytech/polkadot-sdk/pull/4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2)._

## Explanation

Please read
[bridges/modules/xcm-bridge-hub/src/lib.rs](https://github.com/paritytech/polkadot-sdk/blob/149b0ac2

/bridges/modules/xcm-bridge-hub/src/lib.rs#L17-L136)
to understand how managing bridges works. The basic concepts around
`BridgeId` and `LaneId` are also explained there.


## TODO

- [x] search and fix for comment: `// TODO:(bridges-v2) - most of that
stuff was introduced with free header execution:
https://github.com/paritytech/polkadot-sdk/pull/4102` - more info in the
comment
[bellow](https://github.com/paritytech/polkadot-sdk/pull/4427#issuecomment-2126625043)
- [x] TODO: there's only one impl of `EnsureOrigin<Success = Location>`

## TODO - not blocking review

**benchmarking:**
- [x] regenerate all relevant weights for BH/AH runtimes
- [ ] regenerate default weights for bridging pallets e.g.
`modules/messages/src/weights.rs`
- [ ] add benchmarks for `xcm-bridge-hub` pallet
https://github.com/paritytech/polkadot-sdk/issues/5550

**testing:**
- [ ] add xcm-emulator tests for Rococo/Penpal to Westend/Penpal with
full opening channel and sending/receiving `xcm::Transact`

**migrations:**
- [x] add migrations for BridgeHubRococo/Westend
https://github.com/paritytech/parity-bridges-common/issues/2794 (to be
reusable for P/K bridge)
  - [x] check also storage migration, if needed for pallets
  - [ ] migration for XCM type (optional)
  - [x] migration for static lanes to the dynamic (reuse for fellows)

**investigation:**
- [ ] revisit
https://github.com/paritytech/parity-bridges-common/issues/2380
- [ ] check congestion around `LocalXcmChannelManager` and
`OutboundLanesCongestedSignals` impls -
https://github.com/paritytech/polkadot-sdk/issues/5551
  - to be reusable for polkadot-fellows
- return `report_bridge_status` was remove, so we need to `XcmpQueue`
alternative?

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>

This improves `sc-service` API by not requiring the whole
`&Configuration`, using specific configuration options instead.
`RpcConfiguration` was also extracted from `Configuration` to group all
RPC options together.

We don't use Substrate's CLI and would rather not use `Configuration`
either, but some key public functions require it even though they
ignored most of the fields anyway.

`RpcConfiguration` is very helpful not just for consolidation of the
fields, but also to finally make RPC optional for our use case, while
Substrate still runs RPC server on localhost even if listening address
is explicitly set to `None`, which is annoying (and I suspect there is a
reason for it, so didn't want to change the default just yet).

While this is a breaking change, most developers will not notice it if
they use higher-level APIs.

Fixes https://github.com/paritytech/polkadot-sdk/issues/2897

---------

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

# Context

Fees can already be paid in other assets locally thanks to the Trader
implementations we have.
This doesn't work when sending messages because delivery fees go through
a different mechanism altogether.
The idea is to fix this leveraging the `AssetExchanger` config item
that's able to turn the asset the user wants to pay fees in into the
asset the router expects for delivery fees.

# Main addition

An adapter was needed to use `pallet-asset-conversion` for exchanging
assets in XCM.
This was created in
https://github.com/paritytech/polkadot-sdk/pull/5130.

The XCM executor was modified to use `AssetExchanger` (when available)
to swap assets to pay for delivery fees.

## Limitations

We can only pay for delivery fees in different assets in intermediate
hops. We can't pay in different assets locally. The first hop will
always need the native token of the chain (or whatever is specified in
the `XcmRouter`).
This is a byproduct of using the `BuyExecution` instruction to know
which asset should be used for delivery fee payment.
Since this instruction is not present when executing an XCM locally, we
are left with this limitation.
To illustrate this limitation, I'll show two scenarios. All chains
involved have pools.

### Scenario 1

Parachain A --> Parachain B

Here, parachain A can use any asset in a pool with its native asset to
pay for local execution fees.
However, as of now we can't use those for local delivery fees.
This means transfers from A to B need some amount of A's native token to
pay for delivery fees.

### Scenario 2

Parachain A --> Parachain C --> Parachain B

Here, Parachain C's remote delivery fees can be paid with any asset in a
pool with its native asset.
This allows a reserve asset transfer between A and B with C as the
reserve to only need A's native token at the starting hop.
After that, it could all be pool assets.

## Future work

The fact that delivery fees go through a totally different mechanism
results in a lot of bugs and pain points.
Unfortunately, this is not so easy to solve in a backwards compatible
manner.
Delivery fees will be integrated into the language in future XCM
versions, following
https://github.com/polkadot-fellows/xcm-format/pull/53.

Old PR: https://github.com/paritytech/polkadot-sdk/pull/4375.

Gensis config presets moved from `polkadot-parachain` binary into
`asset-hub-rococo` runtime.

relates to: #3944

---------

Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

Related to https://github.com/paritytech/polkadot-sdk/issues/4787

The main changes in this PR are the following:
- making the NodeSpec logic generic on the Block type
- adding an omni-node variant with u64 block number

Apart from this, the PR also moves some of the logic in `service.rs` to
the `common` subfolder

The omni-node variant with u64 block number is not used yet. We have to
either expose the option in the CLI or to read the block number from the
chain spec somehow. Will do it in a future PR.

Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

✄
-----------------------------------------------------------------------------

Thank you for your Pull Request! 

 Please make sure it follows the
contribution guidelines outlined in [this

document](https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md)
and fill out the
sections below. Once you're ready to submit your PR for review, please
delete this section and leave only the text under
the "Description" heading.

# Description

*A concise description of what your PR is doing, and what potential
issue it is solving. Use [Github semantic

linking](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)
to link the PR to an issue that must be closed once this is merged.*

## Integration

*In depth notes about how this PR should be integrated by downstream
projects. This part is mandatory, and should be
reviewed by reviewers, if the PR does NOT have the `R0-Silent` label. In
case of a `R0-Silent`, it can be ignored.*

## Review Notes

*In depth notes about the **implementation** details of your PR. This
should be the main guide for reviewers to
understand your approach and effectively review it. If too long, use

[`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details)*.

*Imagine that someone who is depending on the old code wants to
integrate your new code and the only information that
they get is this section. It helps to include example usage and default
value here, with a `diff` code-block to show
possibly integration.*

*Include your leftover TODOs, if any, here.*

# Checklist

* [ ] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)
* [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

You can remove the "Checklist" section once all have been checked. Thank
you for your contribution!

✄
-----------------------------------------------------------------------------

Co-authored-by: Dónal Murray <donal.murray@parity.io>

rpc server: listen to `ipv6 socket` if available and `--experimental-rpc-endpoint` CLI option (#4792)

Close https://github.com/paritytech/polkadot-sdk/issues/3488,
https://github.com/paritytech/polkadot-sdk/issues/4331

This changes/adds the following:

1. The default setting is that substrate starts a rpc server that
listens to localhost both Ipv4 and Ipv6 on the same port. Ipv6 is
allowed to fail because some platforms may not support it
2. A new RPC CLI option `--experimental-rpc-endpoint` which allow to
configure arbitrary listen addresses including the port, if this is
enabled no other interfaces are enabled.
3. If the local addr is not found for any of the sockets the server is
not started throws an error.
4. Remove the deny_unsafe from the RPC implementations instead this is
an extension to allow different polices for different interfaces/sockets
such one may enable unsafe on local interface and safe on only the
external interface.

So for instance in this PR it's now possible to start up three RPC
endpoints as follows:
```
$ polkadot --experimental-rpc-endpoint "listen-addr=127.0.0.1:9944,rpc-methods=unsafe" --experimental-rpc-endpoint "listen-addr=0.0.0.0:9945,rpc-methods=safe,rate-limit=100" --experimental-rpc-endpoint "listen-addr=[::1]:9944,optional=true"
```

#### Needs to be addressed

~1. Support binding to a random port if it's fails with the default
stuff for backward compatible reasons~
~2. How to sync that the rpc CLI params and that the rpc-listen-addr
align, hard to maintain...~
~3. Add similar warning prints for exposing unsafe methods on external
interfaces..~
~4. Inline todos + the hacky String conversion from rpc params.~

#### Cons with this PR

Manual strings parsing impl more error-prone than relying on clap....

//cc @jsdw @BulatSaif @PierreBesson @bkchr



---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

Related to: https://github.com/paritytech/polkadot-sdk/issues/5210

Follow-up for https://github.com/paritytech/polkadot-sdk/pull/5288, as
per this comment:
https://github.com/paritytech/polkadot-sdk/pull/5288#discussion_r1711157476

I hope I understood this correctly.

This PR introduces a feature that allows to optionally enable using the
full PoV size.

Technically, we're ready to enable it by default, but as corresponding
runtime changes have not been propagated to the system parachain
runtimes yet, doing so could put them at risk. On the other hand, there
are teams that could benefit from it right now, and it makes no sense
for them to wait for the fellowship release and everything.

---------

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

Calls were written to be removed after June 2024. This PR removes them.

This PR will break users using those calls. The call won't be decodable
by the runtime, so it should fail early with no consequences. The
functionality must be same as before, users will just need to use the
calls in `System`.

This PR adds possibility to set BUILD_OPTIONS to the "Srtool Build" step
in the release pipeline while building runtimes.

Colses: https://github.com/paritytech/release-engineering/issues/213

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: EgorPopelyaev <EgorPopelyaev@users.noreply.github.com>

The `pallet-collator-selection` is not correctly using the weight for
the
[new_session](https://github.com/blockdeep/pallet-collator-staking/blob/main/src/benchmarking.rs#L350-L353)
function.

The first parameter is the removed candidates, and the second one the
original number of candidates before the removal, but both values are
swapped.

(Please, do not merge until SA, reverted and restored of
https://github.com/paritytech/polkadot-sdk/pull/4944)

Original PR with more context:
https://github.com/paritytech/parity-bridges-common/pull/2211
Relates to:
https://github.com/paritytech/parity-bridges-common/issues/2210

## TODO

- [x] fresh weighs for `pallet_bridge_messages`
- [x] add `try_state` for `pallet_bridge_messages` which checks for
unpruned messages - relates to the
[comment](https://github.com/paritytech/parity-bridges-common/pull/2211#issuecomment-1643224831)
- [x] ~prepare migration, that prunes leftovers, which would be pruned
eventually from `on_idle` the
[comment](https://github.com/paritytech/parity-bridges-common/pull/2211#issuecomment-1643224831)~
can be done also by `set_storage` / `kill_storage` or with
`OnRuntimeUpgrade` implementatino when `do_try_state_for_outbound_lanes`
detects problem.

## Open question

- [ ] Do we really need `oldest_unpruned_nonce` afterwards?
- after the runtime upgrade and when `do_try_state_for_outbound_lanes`
pass, we won't need any migrations here
    - we won't even need `do_try_state_for_outbound_lanes`
- please check comments bellow:
https://github.com/paritytech/polkadot-sdk/pull/4944#discussion_r1666737961

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>

I'm not sure if this is exactly what
https://github.com/paritytech/polkadot-sdk/issues/3537 meant, but I
think it should be fine to wait for relay chain before initializing
parachain node fully, which removed the need for background task and
extra hacks throughout the stack just to know where warp sync should
start.

Previously there were both `WarpSyncParams` and `WarpSyncConfig`, but
there was no longer any point in having two data structures, so I
simplified it to just `WarpSyncConfig`.

Fixes https://github.com/paritytech/polkadot-sdk/issues/3537

Add the Polkadot Coretime chain-spec to the directory with the other
system chain-specs.

This is the chain-spec used at genesis and for which the genesis head
data was generated.

It is also included in the assets for fellowship [release
v1.3.0](https://github.com/polkadot-fellows/runtimes/releases/tag/v1.3.0)

Related to https://github.com/paritytech/polkadot-sdk/issues/5210

There was no need for it to be `&mut self` since block import can happen
concurrently for different blocks and in many cases it was `&mut Arc<dyn
BlockImport>` anyway :man_shrugging:

Similar in nature to
https://github.com/paritytech/polkadot-sdk/pull/4844