Co-authored-by: command-bot <>

Partially implements
https://github.com/paritytech/polkadot-sdk/issues/5048

- adds a core selection runtime API to cumulus and a generic way of
configuring it for a parachain
- modifies the slot based collator to utilise the claim queue and the
generic core selection

What's left to be implemented (in a follow-up PR):
- add the UMP signal for core selection into the parachain-system pallet

View the RFC for more context:
https://github.com/polkadot-fellows/RFCs/pull/103

---------

Co-authored-by: command-bot <>

## Description

Details and rationale explained here:
https://github.com/paritytech/polkadot-sdk/issues/5657#issuecomment-2363076080

Fixes: https://github.com/paritytech/polkadot-sdk/issues/5657

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

The genesis state is currently partially provided via
`OverlayedChanges`, but these changes are reset by the runtime after the
first repetition, causing the second repitition to use an invalid
genesis state.

Changes:
- Provide the genesis state as a `Storage` without any
`OverlayedChanges` to make it work correctly with repetitions.
- Add `--genesis-builder-preset` option to use different genesis preset
names.
- Improve error messages.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>

Use `sp_core::Bytes` as `payload` to encode the values correctly as
`hex` string.

It is a first step for switching to the `frame-omni-bencher` for CI.

This PR includes several changes related to generating chain specs plus:

- [x] pallet `assigned_slots` fix missing `#[serde(skip)]` for phantom
- [x] pallet `paras_inherent` benchmark fix - cherry-picked from
https://github.com/paritytech/polkadot-sdk/pull/5688
- [x] migrates `get_preset` to the relevant runtimes
- [x] fixes Rococo genesis presets - does not work
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7317249
- [x] fixes Rococo benchmarks for CI 
- [x] migrate westend genesis
- [x] remove wococo stuff

Closes: https://github.com/paritytech/polkadot-sdk/issues/5680

## Follow-ups
- Fix for frame-omni-bencher
https://github.com/paritytech/polkadot-sdk/pull/5655
- Enable new short-benchmarking CI -
https://github.com/paritytech/polkadot-sdk/pull/5706
- Remove gitlab pipelines for short benchmarking
- refactor all Cumulus runtimes to use `get_preset` -
https://github.com/paritytech/polkadot-sdk/issues/5704
- https://github.com/paritytech/polkadot-sdk/issues/5705
- https://github.com/paritytech/polkadot-sdk/issues/5700
- [ ] Backport to the stable

---------

Co-authored-by: command-bot <>
Co-authored-by: ordian <noreply@reusable.software>

This PR is cherry-picked from
https://github.com/paritytech/polkadot-sdk/pull/5663 so that I can
maintain a smaller polkadot-sdk diff downstream sooner than later.

cc @lexnv @dmitry-markin



---------

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>

When using the relay chain though a `Arc<dyn RelayChainInterface>` there
is no way to call arbitrary runtime apis. Both implementations of that
trait allow this, so it feels natural to expose this functionality in
the trait.

This PR adds a `call_runtime_api` method to RelayChainInterface trait,
and a separate function also named `call_runtime_api` which allows the
caller to specify the input and output types, as opposed to having to
encode them. This generic function cannot be part of the trait because a
`dyn Trait` object cannot have generic methods.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

# Description

Fix https://github.com/paritytech/polkadot-sdk/pull/5074 which missed
the runtime migration to initialize channels of the bridge.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

# Description

When we start a node with connections to external RPC servers (as a
minimal node), we lack metrics around how many individual calls we're
doing to the remote RPC servers and their duration. This PR adds metrics
that measure durations of each RPC call made by the minimal nodes, and
implicitly how many calls there are.

Closes #5409 
Closes #5689

## Integration

Node operators should be able to track minimal node metrics and decide
appropriate actions according to how the metrics are interpreted/felt.
The added metrics can be observed by curl'ing the prometheus metrics
endpoint for the ~relaychain~ parachain (it was changed based on the
review). The metrics are represented by
~`polkadot_parachain_relay_chain_rpc_interface`~
`relay_chain_rpc_interface` namespace (I realized lining up
`parachain_relay_chain` in the same metric might be confusing :).
Excerpt from the curl:

```
relay_chain_rpc_interface_bucket{method="cha...

# Description

Adjust the PoV response size to the default values used in the
substrate.
Fixes https://github.com/paritytech/polkadot-sdk/issues/5503

## Integration

The changes shouldn't impact downstream projects since we are only
increasing the limit.

## Review Notes

You can't see it from the changes, but it affects all protocols that use
the `POV_RESPONSE_SIZE` constant.
- Protocol::ChunkFetchingV1
- Protocol::ChunkFetchingV2
- Protocol::CollationFetchingV1
- Protocol::CollationFetchingV2
- Protocol::PoVFetchingV1
- Protocol::AvailableDataFetchingV1

## Increasing timeouts


https://github.com/paritytech/polkadot-sdk/blob/fae15379

/polkadot/node/network/protocol/src/request_response/mod.rs#L126-L129

I assume the current PoV request timeout is set to 1.2s to handle 5
consecutive requests during a 6s block. This setting does not relate to
the PoV response size. I see no reason to change the current timeouts
after adjusting the response size.

However, we should consider networking speed limitations if we want to
increase the maximum PoV size to 10 MB. With the number of parallel
requests set to 10, validators will need the following networking
speeds:
- 5 MB PoV: at least 42 MB/s, ideally 50 MB/s.  
- 10 MB PoV: at least 84 MB/s, ideally 100 MB/s.

The current required speed of 50 MB/s aligns with the 62.5 MB/s
specified [in the reference hardware
requirements](https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#reference-hardware).
Increasing the PoV size to 10 MB may require a higher networking speed.

---------

Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

Co-authored-by: command-bot <>

When running XCM emulated tests and seeing the logs with `RUST_LOG=xcm`
or `RUST_LOG=xcm=trace`, it's sometimes a bit hard to figure out the
chain where the logs are coming from.

I added a log whenever `execute_with` is called, to know the chain which
makes the following logs. Looks like so:

<img width="1499" alt="Screenshot 2024-09-13 at 20 14 13"
src="https://github.com/user-attachments/assets/a31d7aa4-11d1-4d3e-9a65-86f38347c880">

There are already log targets for when UMP, DMP and HRMP messages are
being processed. To see them, you have to use the log targets `ump`,
`dmp`, and `hrmp` respectively. So `RUST_LOG=xcm,ump,dmp,hrmp` would let
you see every log.
I prefixed the targets with `xcm::` so you can get all the relevant logs
just by filtering by `xcm`. You can always use the whole target to see
just the messages being processed.

These logs showed the message as an array of bytes, I made them show a
hexadecimal string instea...

This will make sure that when uploading new code that the declared
static memory fits within a defined limit. We apply different limits to
code and data. Reason is that code will consume much more memory per
byte once decoded during lazy execution.

This PR:

1) Remove the MaxCodeLen from the `Config` to we maintain tight control
over it.
2) Defines a single `STATIC_MEMORY_BYTES` knob that limits the maximum
decoded size.
3) Enforces them only on upload but not on execution so we can raise
them later.
4) Adapt the worst case calculation in `integrity_check`.
5) Bumps the max stack depth from 5 to 10 as this will still fit within
our memory envelope.
6) The memory limit per contract is now a cool 1MB that can be spent on
data or code.
7) Bump PolkaVM for good measure
8) The blob is limited to 256kb which is just a sanity check to not even
try parsing very big inputs.

---------

Co-authored-by: Cyrill Leutwiler <cyrill@parity.io>

## Issue
https://github.com/paritytech/polkadot-sdk/issues/4858

## Description
This PR removes `libp2p::request_response::OutboundFailure` from
`substrate/client/network/sync/src/engine.rs`. This way, the dependency
with the library `libp2p` is removed from `sc-network-sync`.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

Instead of error out if the provided output buffer is smaller than what
we want to write, we can just write what fits into the output buffer
instead. We already write back the actual bytes written to the in-out
pointer, so contracts can check it anyways.

This in turn introduces the benefit of allowing contracts to implicitly
request only a portion of the returned data from calls and incantations.
Which is especially beneficial for YUL as the `call` family opcodes have
a return data size argument and this change removes the need to work
around it in contract code.

---------

Signed-off-by: xermicus <cyrill@parity.io>

When `SpendOrigin` doesn't return any `succesful_origin`, it doesn't
mean that `RejectOrigin` will do the same. Thus, this pr fixes a
potential wrong benchmarked weight for when `SpendOrigin` is set to e.g.
`NeverOrigin`.

# Description

Follow-up to https://github.com/paritytech/polkadot-sdk/pull/5469 and
mostly covering https://github.com/paritytech/polkadot-sdk/issues/5333.

The primary change here is that syncing strategy is no longer created
inside of syncing engine, instead syncing strategy is an argument of
syncing engine, more specifically it is an argument to `build_network`
that most downstream users will use. This also extracts addition of
request-response protocols outside of network construction, making sure
they are physically not present when they don't need to be (imagine
syncing strategy that uses none of Substrate's protocols in its
implementation for example).

This technically allows to completely replace syncing strategy with
whatever strategy chain might need.

There will be at least one follow-up PR that will simplify
`SyncingStrategy` trait and other public interfaces to remove mentions
of block/state/warp sync requests, replacing them with generic APIs,
such that strategies where warp sync is not applicable don't have to
provide dummy method implementations, etc.

## Integration

Downstream projects will have to write a bit of boilerplate calling
`build_polkadot_syncing_strategy` function to create previously default
syncing strategy.

## Review Notes

Please review PR through individual commits rather than the final diff,
it will be easier that way. The changes are mostly just moving code
around one step at a time.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [x] I have made corresponding changes to the documentation (if
applicable)

the range should always contain at least 2 values for the benchmark to
work
closes #5680

Setup auto incremented asset id to `50_000_000` for trust backed assets.

In order to align with Polkadot/Kusama Asset Hub -
https://github.com/polkadot-fellows/runtimes/pull/414
The next closes existing assets IDs in Rococo is `69_696_969`, in
Westend is `88_228_866`.

### Migration
**Stakeholders**: all clients providing asset creation functionality on
Westend/Rococo Asset Hub

This change does not break the API but introduces a new constraint. It
implements an auto-incremented ID strategy for Trust-Backed Assets (50
pallet instance indexes on both networks), starting at ID 50,000,000.
Each new asset must be created with an ID that is one greater than the
last asset created. The next ID can be fetched from the `NextAssetId`
storage item of the assets pallet. An empty `NextAssetId` storage item
indicates no constraint on the next asset ID and can serve as a feature
flag for this release.

Introduces a trait for verifying existence proofs in the runtime. The
trait is implemented for the 16 patricia merkle tree and the binary
tree.

---------

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: command-bot <>

### Issue

It was impossible to benchmark the pallet_treasury when `SpendOrigin =
frame_support::traits::NeverEnsureOrigin<u64>;` was specified.

### Done

- [x] Use `weight = 0` for all extrinsics that are un-callable with no
`SpendOrigin`.
- [x] Fix benchmarks for extrinsics requiring a Spend even if
`SpendOrigin = frame_support::traits::NeverEnsureOrigin<u64>;`

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

# Description

Adding support for send polkadot native assets(PNA) to Ethereum network
through snowbridge. Asset with location in view of AH Including:

- Relay token `(1,Here)`
- Native asset `(0,[PalletInstance(instance),GenereIndex(index)])`
managed by Assets Pallet
- Native asset of Parachain `(1,[Parachain(paraId)])` managed by Foreign
Assets Pallet

The original PR in https://github.com/Snowfork/polkadot-sdk/pull/128
which has been internally reviewed by Snowbridge team.

# Notes

- This feature depends on the companion solidity change in
https://github.com/Snowfork/snowbridge/pull/1155. Currently register PNA
is only allowed from
[sudo](https://github.com/Snowfork/polkadot-sdk/blob/46cb3528

/bridges/snowbridge/pallets/system/src/lib.rs#L621),
so it's actually not enabled. Will require another runtime upgrade to
make the call permissionless together with upgrading the Gateway
contract.

- To make things easy multi-hop transfer(i.e. sending PNA from Ethereum
through AH to Destination chain) is not support ed in this PR. For this
case user can switch to 2-phases transfer instead.

---------

Co-authored-by: Clara van Staden <claravanstaden64@gmail.com>
Co-authored-by: Alistair Singh <alistair.singh7@gmail.com>
Co-authored-by: Vincent Geddes <117534+vgeddes@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

Discovered a bug in the migrations pallet while debugging
https://github.com/paritytech/try-runtime-cli/pull/90.
It only occurs when a single MBM is configured - hence it did not happen
when Ajuna Network tried it...

Changes:
- Check len of the tuple before accessing its nth_id
- Make nth_id return `None` on unary tuples and n>0

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>

The salt argument should be optional to allow create1 equivalent calls.

---------

Signed-off-by: xermicus <cyrill@parity.io>

This adds an API method `balance_of`, corresponding to the
[BALANCE](https://www.evm.codes/#31?fork=cancun) EVM opcode.

In `Ext`, `balance` and `balance_of` are internally routed through the
same new `account_balance` method: `balance` is technically the same as
`balance_of` with the caller address. This avoids duplicating all the
tests and avoids a small inefficiency (in theory, `balance` directly
call `balance_of` however this introduces a round trip of converting the
target address to a H160 and back).

---------

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: command-bot <>

fix https://github.com/paritytech/polkadot-sdk/issues/5683

Proxies are possible in the runtimes for Kusama and Polkadot but this
functionality was not previously available on testnets.

Closes #5453.

Proxies can now be used on `coretime-rococo`, `coretime-westend`,
`people-rococo` and `people-westend` in the same way as they can be on
Kusama and Polkadot chains.

The exact same proxies are configured as the production runtimes for the
respective system parachains.

This is part of the work to further optimize the approval subsystems, if
you want to understand the full context start with reading
https://github.com/paritytech/polkadot-sdk/pull/4849#issue-2364261568,

# Description
This PR contain changes to make possible the run of single
approval-voting instance on a worker thread, so that it can be
instantiated by the approval-voting-parallel subsystem.

This does not contain any functional changes it just decouples the
subsystem from the subsystem Context and introduces more specific trait
dependencies for each function instead of all of them requiring a
context.

This change can be merged  independent of the followup PRs.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

### Description: 
* Adds `DeprecationStatusIR` enum to sp_metadata_ir.
Deprecation info for simple items.
* Adds `DeprecationInfoIR` enum to sp_metadata_ir.
It is a deprecation info for an enums/errors/calls. Contains
`DeprecationStatusIR`.
Denotes full/partial deprecation of the type or its variants/calls
* Adds `deprecation_info` field to 
       - `RuntimeApiMetadataIR`
       - `RuntimeApiMethodMetadataIR`
       - `StorageEntryMetadataIR`
       - `PalletConstantMetadataIR`
       - `PalletCallMetadataIR`
       - `PalletMetadataIR`
       - `PalletEventMetadataIR`
       - `PalletErrorMetadataIR`

### Testing done: 
- Unit tests to check whether or not correct `note`/`since` texts are
getting propagated to the metadata structs.
- UI test to check for error message in case of incorrect attribute
usage.
There's also some test updates to make sure that deprecation attributes
are getting propagated to the relevant structs.

see: #4098, Solution: A

### Examples of produced deprecation info metadata
They can be found in:
 - Tests for `frame-support`
 - hackmd link https://hackmd.io/@Zett98/Bys0YgbcR

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

Close #5677

I made a nit when I moved this code:
https://github.com/paritytech/polkadot-sdk/blob/v1.14.0-rc1/substrate/client/service/src/lib.rs#L379-#L385
in https://github.com/paritytech/polkadot-sdk/pull/4792

Thus:
 - (ip.is_loopback(), RpcMethods::Auto) -> allow unsafe
 - (!ip.is_loopback(), RpcMethods::Auto) -> deny unsafe

---------

Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>

Fixes #5577 

I decided to bubble up the error from where we actually try to load the
contract info. This helps to make sure that we don't miss some entry
point by accident. The draw back is that we have to live with some
additional `.expect`.

@pgherveou

 With this logic the proxy and its runtime part should be
completely unaware whether something is a contract call or a balance
transfer. They should just route everything into pallet_revive.

---------

Co-authored-by: Cyrill Leutwiler <cyrill@parity.io>

This release introduces several new features, improvements, and fixes to
the litep2p library. Key updates include enhanced error handling,
configurable connection limits, and a new API for managing public
addresses.

For a detailed set of changes, see [litep2p
changelog](https://github.com/paritytech/litep2p/blob/master/CHANGELOG.md#070---2024-09-05).

This PR makes use of:
- connection limits to optimize network throughput
- better errors that are propagated to substrate metrics 
- public addresses API to report healthy addresses to the Identify
protocol

### Warp sync time improvement

Measuring warp sync time is a bit inaccurate since the network is not
deterministic and we might end up using faster peers (peers with more
resources to handle our requests). However, I did not see warp sync
times of 16 minutes, instead, they are roughly stabilized between 8 and
10 minutes.

For measuring warp-sync time, I've used
[sub-trige-logs](https://github.com/lexnv/sub-triage-logs/?tab=readme-ov-file#warp-time)

### Litep2p

Phase | Time
 -|-
Warp  | 426.999999919s
State | 99.000000555s
Total | 526.000000474s

### Libp2p

Phase | Time
 -|-
Warp  | 731.999999837s
State | 71.000000882s
Total | 803.000000719s

Closes: https://github.com/paritytech/polkadot-sdk/issues/4986


### Low peer count

After exposing the `litep2p::public_addresses` interface, we can report
to litep2p confirmed external addresses. This should mitigate or at
least improve: https://github.com/paritytech/polkadot-sdk/issues/4925.
Will keep the issue around to confirm this.


### Improved metrics

We are one step closer to exposing similar metrics as libp2p:
https://github.com/paritytech/polkadot-sdk/issues/4681.

cc @paritytech/networking 

### Next Steps
- [x] Use public address interface to confirm addresses to identify
protocol

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

This PR is a pure refactoring by consolidating all the RPC related
parameters into a dedicated `RpcParams` struct, allowing us at subcoin
to build a custom run cmd without the need to duplicate code.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

This is a step towards
https://github.com/paritytech/polkadot-sdk/issues/5333

The commits with code moving (but no other changes) and actual changes
are separated for easier review.

Essentially this results in `SyncingStrategy` trait replacing struct
(which is renamed to `PolkadotSyncingStrategy`, open for better name
suggestions). Technically it is already possible to replace
`PolkadotSyncingStrategy<B, Client>` with `Box<dyn SyncingStrategy<B>`
in syncing engine, but I decided to postpone such change until we
actually have an ability to customize it. It should also be possible to
swap `PolkadotSyncingStrategy` with just `ChainSync` that only supports
regular full sync from genesis (it also implements `SyncingStrategy`
trait).

While extracted trait still has a lot of non-generic stuff in it like
exposed knowledge of warp sync and `StrategyKey` with hardcoded set of
options, I believe this is a step in the right direction and will
address those in upcoming PRs.

With https://github.com/paritytech/polkadot-sdk/pull/5431 that landed
earlier warp sync configuration is more straightforward, but there are
still numerous things interleaved that will take some time to abstract
away nicely and expose in network config for developers. For now this is
an internal change even though data structures are technically public
and require major version bump.

This PR addresses an issue where state sync may fail to start if the
conditions required for its initiation are not met when a finalized
block notification is received. `pending_state_sync_attempt` is
introduced to trigger the state sync later when the conditions are
satisfied.

This issue was spotted when I worked on #5406, specifically,
`queue_blocks` was not empty when the finalized block notification was
received, and then the state sync was stalled. cc @dmitry-markin



---------

Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Bastian Köcher <git@kchr.de>

Update `RuntimeVerison` type and use `system_version` to derive extrinsics root `StateVersion` instead of `V0` (#4257)

This PR 
- Renames `RuntimeVersion::state_version` to `system_version`
- Uses `Runtime::system_version` to derive extrinsics root
`StateVersion` instead of default `StateVersion::V0`

This PR should not be breaking any existing chains so long as they use
same `RuntimeVersion::state_version` for `Runtime::system_version`

Using `RuntimeVersion::system_version = 2` will make the extrinsics root
to use `StateVersion::V1` instead of `V0`

RFC for this change - https://github.com/polkadot-fellows/RFCs/pull/42

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>

Explicitly give the types in some generated code so that the error shows
up good when user code is wrong.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Fix https://github.com/paritytech/polkadot-sdk/issues/5629