For permissionless lanes, we add `lane_state` to the `InboundLaneData`
and `OutboundLaneData` structs. However, for a period of time (until
both BHK and BHP are upgraded to the same version), we need the relayer
to function with runtimes where one has been migrated with `lane_state`
and the other has not. This PR addresses the incompatibility by
introducing wrapper structs for decoding without `lane_state`.

The AllowTopLevelPaidExecutionFrom allows ClearOrigin instructions
before the expected BuyExecution instruction, it also allows messages
without any origin altering instructions.

This commit enhances the barrier to also support messages that use
AliasOrigin, or DescendOrigin. This is sometimes desired in asset
transfer XCM programs that need to run the inbound assets instructions
using the origin chain root origin, but then want to drop privileges for
the rest of the program. Currently these programs drop privileges by
clearing the origin completely, but that also unnecessarily limits the
range of actions available to the rest of the program. Using
DescendOrigin or AliasOrigin allows the sending chain to instruct the
receiving chain what the deprivileged real origin is.

See https://github.com/polkadot-fellows/RFCs/pull/109 and
https://github.com/polkadot-fellows/RFCs/pull/122 for more details on
how DescendOrigin and AliasOrigin could be used instead of ClearOrigin.

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>

# Description

Closes [#5790](https://github.com/paritytech/polkadot-sdk/issues/5790).
Useful for starting nodes based on minimal/solochain when doing
development or for testing omni node with less happy code paths. It is
reusing the presets defined for the nodes chain specs.

## Integration

Specifically useful for development/testing if generating chain-specs
for `minimal` or `solochain` runtimes from `templates` directories.

## Review Notes

Added `genesis_config_presets` modules for both minimal/solochain. I
reused the presets defined in each node `chain_spec` module
correspondingly.

### PRDOC

Not sure who uses templates, maybe node devs and runtime devs at start
of their learning journey, but happy to get some guidance on how to
write the prdoc if needed.

### Thinking out loud

I saw concerns around sharing functionality for such genesis config
presets between the template chains. I think there might be a case for
doing that, on ...

Jaeger tracing went mostly unused and it created bigger problems like
wasting CPU or memory leaks, so remove it entirely.

Fixes: https://github.com/paritytech/polkadot-sdk/issues/4995

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

# Description

The EthereumBlobExporter consumes the `dest` parameter when the
destination is not `Here`. Subsequent exporters will receive a `None`
value for the destination instead of the original destination value,
which is incorrect.
 
Closes #5788

## Integration

Minor fix related to the exporter behaviour.

## Review Notes

Verified that tests
`exporter_validate_with_invalid_dest_does_not_alter_destination` and
`exporter_validate_with_invalid_universal_source_does_not_alter_universal_source`
fail without the fix in the exporter.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

Relates to: https://github.com/paritytech/polkadot-sdk/pull/5916
Relates to: https://github.com/polkadot-js/api/pull/5976

---------

Co-authored-by: Javier Viola <javier@parity.io>

This PR adds the `stableYYMM-rcX` or `stableYYMM-X-rcX` tags to the
docker images, so that they could be published with the new tag naming
scheme.

Closes: https://github.com/paritytech/release-engineering/issues/224

Removing the shell node variant for the polkadot-parachain as discussed
here:
https://github.com/paritytech/polkadot-sdk/pull/5586#discussion_r1752635254

Resolves https://github.com/paritytech/polkadot-sdk/issues/5898

This PR removes the requirement to set the `LaneId` in the relayer CLI
configuration where it was not really necessary.

---------

Co-authored-by: command-bot <>

- update baseline for pallet_revive
- update cmd pipeline name
- Fix compilation after renaming some of benchmarks in pallet_revive.
[Runtime Dev]. Changed the "instr" benchmark so that it should no longer
return to little weight. It is still bogus but at least benchmarking
should not work. (by @athei

 )

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: command-bot <>

Close https://github.com/paritytech/polkadot-sdk/issues/5589

This PR makes it possible for `rpc_v2::Storage::query_iter_paginated` to
be "backpressured" which is achieved by having a channel where the
result is sent back and when this channel is "full" we pause the
iteration.

The chainHead_follow has an internal channel which doesn't represent the
actual connection and that is set to a very small number (16). Recall
that the JSON-RPC server has a dedicate buffer for each connection by
default of 64.

#### Notes

- Because `archive_storage` also depends on
`rpc_v2::Storage::query_iter_paginated` I had to tweak the method to
support limits as well. The reason is that archive_storage won't get
backpressured properly because it's not an subscription. (it would much
easier if it would be a subscription in rpc v2 spec because nothing
against querying huge amount storage keys)
- `query_iter_paginated` doesn't necessarily return the storage "in
order" such as
- `query_iter_paginated(vec![("key1", hash), ("key2", value)], ...)`
could return them in arbitrary order because it's wrapped in
FuturesUnordered but I could change that if we want to process it
inorder (it's slower)
- there is technically no limit on the number of storage queries in each
`chainHead_v1_storage call` rather than the rpc max message limit which
10MB and only allowed to max 16 calls `chainHead_v1_x` concurrently
(this should be fine)

#### Benchmarks using subxt on localhost

- Iterate over 10 accounts on westend-dev -> ~2-3x faster 
- Fetch 1024 storage values (i.e, not descedant values) -> ~50x faster
- Fetch 1024 descendant values -> ~500x faster

The reason for this is because as Josep explained in the issue is that
one is only allowed query five storage items per call and clients has
make lots of calls to drive it forward..

---------

Co-authored-by: command-bot <>
Co-authored-by: James Wilson <james@jsdw.me>

Bump `zombienet` version, including fixes (`ci`) and the latest version
of `pjs` embedded.
Thx!

Resolves https://github.com/paritytech/polkadot-sdk/issues/5026

This PR adds support for starting a dev node with manual seal consensus.
This can be done by using the `--dev-block-time` argument . For example:
```
polkadot-parachain --chain asset-hub-rococo-dev --dev-block-time 5000 --tmp
```

Added `HashedDescription<AccountId,
DescribeFamily<DescribeAllTerminal>>` foreign locations to local
accounts converter to all the parachains.

---------

Co-authored-by: command-bot <>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

Quoting @bkchr (from
[here](https://github.com/paritytech/polkadot-sdk/issues/5334#issuecomment-2383815078)):

> That the hardcoded limit is used there again, is IMO not correct. The
`max_pov_size` should be controlled by the `HostConfiguration` and not
limited by some "random" constant.

This PR aims to change the hard limit to a not-so-random constant,
allowing more room for maneuvering in the future.

Only retry jobs on `runner_system_failure`.
Thx!

Fixes https://github.com/paritytech/polkadot-sdk/issues/5617

Normally, the BEEFY protocol only accepts a single MMR Root entry in a
commitment's payload. But to be extra careful, when validating
equivocation reports, let's check all the MMR roots, if there are more.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

# Description

This PR removes the
`CandidateValidationMessage::ValidateFromChainState`, which was
previously used by backing, but is no longer relevant since initial
async backing implementation
https://github.com/paritytech/polkadot/pull/5557.

Fixes https://github.com/paritytech/polkadot-sdk/issues/5643

## Integration

This change should not affect downstream projects since
`ValidateFromChainState` was already unused.

## Review Notes

- Removed all occurrences of `ValidateFromChainState`.
- Moved utility functions, previously used in candidate validation tests
and malus, exclusively to candidate validation tests as they are no
longer used in malus.
- Deleted the
`polkadot_parachain_candidate_validation_validate_from_chain_state`
metric from Prometheus.
- Removed `Spawner` from `ReplaceValidationResult` in malus’
interceptors.
- `fake_validation_error` was only used for `ValidateFromChainState`
handling, while other cases directly used
`InvalidCandidate::InvalidOutputs`. It has been replaced with
`fake_validation_error`, with a fallback to
`InvalidCandidate::InvalidOutputs`.
- Updated overseer’s minimal example to replace `ValidateFromChainState`
with `ValidateFromExhaustive`.

Bumps the known_good_semver group with 1 update:
[syn](https://github.com/dtolnay/syn).

Updates `syn` from 2.0.77 to 2.0.79
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/syn/releases">syn's
releases</a>.</em></p>
<blockquote>
<h2>2.0.79</h2>
<ul>
<li>Fix infinite loop on parsing chained ranges (<a
href="https://redirect.github.com/dtolnay/syn/issues/1741">#1741</a>)</li>
<li>Fix panic in parsing <code>use</code> items containing absolute
paths (<a
href="https://redirect.github.com/dtolnay/syn/issues/1742">#1742</a>)</li>
</ul>
<h2>2.0.78</h2>
<ul>
<li>Fix infinite loop on chained comparison (<a
href="https://redirect.github.com/dtolnay/syn/issues/1739">#1739</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/dtolnay/syn/commit/732e6e39406538aebe34ed5f5803113a48c28602"><code>732e6e3</code></a>
Release 2.0.79</li>
<li><a
h...

# Description

close #5641

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

This PR backports regular version bumps and prdocs reordering from the
`stable2409` release branch to `master`

---------

Co-authored-by: Morgan Adamiec <morgan@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>

cc @AndreiEres

This is a refactor and improvement from:
https://github.com/paritytech/polkadot-sdk/pull/3881

- `sp_runtime::proving_trie` now exposes a `BasicProvingTrie` for both
`base2` and `base16`.
- APIs for `base16` are more focused on single value proofs, also
aligning their APIs with the `base2` trie
- A `ProvingTrie` trait is included which wraps both the `base2` and
`base16` trie, and exposes all APIs needed for an end to end scenario.
- A `ProofToHashes` trait is exposed which can allow us to write proper
benchmarks for the merkle trie.

---------

Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

# Description

Updates the doc comment on the `import_notification_stream` to make its
behaviour clearer.

Closes [Unexpected behaviour of block
`import_notification_stream`](https://github.com/paritytech/polkadot-sdk/issues/5596).

## Integration

Doesn't apply.

## Review Notes

The old comment docs caused some confusion to myself and some members of
my team, on when this notification stream is triggered. This is
reflected in the linked
[issue](https://github.com/paritytech/polkadot-sdk/issues/5596), and as
discussed there, this PR aims to prevent this confusion in future devs
looking to make use of this functionality.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [x] I have made corresponding changes to the documentation (if
applicable)
* [x] I have added tests that prove my fix is effective or that my
feature works (if applicable)

You can remove the "Checklist" section once all have been checked. Thank
you for your contribution!

---------

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

Just a tiny config fix

Co-authored-by: Bastian Köcher <git@kchr.de>

# Description

`substrate-wasm-builder` can be a build dependency for crates which
develop FRAME runtimes. I had a tough time seeing errors happening in
such crates (e.g. runtimes from the `templates` directory) in my IDE. I
use a combination of rust-analyzer + nvim-lsp + nvim-lspconfig +
rustacean.vim and all of this stack is not able to correctly parse
errors emitted during the `build` phase.

As a matter of fact there is also a cargo issue tracking specifically
this issue where cargo doesn't propagate the `--message-format` type to
the build phase: [here](https://github.com/rust-lang/cargo/issues/14246)
initially and then
[here](https://github.com/rust-lang/cargo/issues/8283). It feels like a
solution for this use case isn't very close, so if it comes to runtimes
development (both as an SDK user and developer), enabling wasm builder
to emit diagnostics messages friendly to IDEs would be useful for
regular workflows where IDEs are used for finding errors instead of
manually running `cargo` commands.

## Integration

It can be an issue if Substrate/FRAME SDKs users and developers rely on
the runtimes' crates build phase output in certain ways. Emitting
compilation messages as json will pollute the regular compilation output
so people that would manually run `cargo build` or `cargo check` on
their crates will have a tougher time extracting the non JSON output.

## Review Notes

Rust IDEs based on rust-analyzer rely on cargo check/clippy to extract
diagnostic information. The information is generated by passing flags
like `--messages-format=json` to the `cargo` commands. The messages are
extracted by rust-analyzer and published to LSP clients that will
populate UIs accordingly.

We need to build against the wasm target by using `message-format=json`
too so that IDEs can show the errors for crates that have a build
dependency on `substrate-wasm-builder`.

---------

Signed-off-by: Iulian Barbu <iulian.barbu@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

This PR removes a workaround which had a reference comment to a rust
compiler issue. The issue has been fixed and we should be able to remove
that trait bound.

- install frame-omni-bencher from the path
- fix bench_features config

Reported in:
#5844
#5848

Jaeger spans were not usable for debugging, see
https://github.com/paritytech/polkadot-sdk/issues/4995, but we pay a
price in CPU cost, subsystem-benchmarks show this brings a reduction of
about 10-15% in CPU usage per subsystem, so remove it.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

cc https://github.com/paritytech/ci_cd/issues/1035

cc https://github.com/paritytech/ci_cd/issues/1023

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Maksym H <1177472+mordamax@users.noreply.github.com>
Co-authored-by: gui <gui.thiolliere@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>

Bump `zombienet` version to prevent report fails at teardown phase.

Changes in PR:
- disabled cargo-hfuzz until [the
issue](https://github.com/paritytech/polkadot-sdk/issues/5812) is fixed.
- enabled condition to skip jobs when no rust files are changed

This is the implementation of the approach described here:
https://github.com/paritytech/polkadot-sdk/issues/1617#issuecomment-2150321612
&
https://github.com/paritytech/polkadot-sdk/issues/1617#issuecomment-2154357547
&
https://github.com/paritytech/polkadot-sdk/issues/1617#issuecomment-2154721395.

## Description of changes

The end goal is to have an architecture where we have single
subsystem(`approval-voting-parallel`) and multiple worker types that
would full-fill the work that currently is fulfilled by the
`approval-distribution` and `approval-voting` subsystems. The main loop
of the new subsystem would do just the distribution of work to the
workers.

The new subsystem will have:
- N approval-distribution workers: This would do the work that is
currently being done by the approval-distribution subsystem and in
addition to that will also perform the crypto-checks that an assignment
is valid and that a vote is correctly signed. Work is assigned via ...

Closes #5714

A change that I missed to add in
https://github.com/paritytech/polkadot-sdk/pull/5525 .

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

This PR introduces 2 new syscalls: `return_data_size` and
`return_data_copy`, resembling the semantics of the EVM `RETURNDATASIZE`
and `RETURNDATACOPY` opcodes.

The ownership of `ExecReturnValue` (the return data) has moved to the
`Frame`. This allows implementing the new contract API functionality in
ext with no additional copies. Returned data is passed via contract
memory, memory is (will be) metered, hence the amount of returned data
can not be statically known, so we should avoid storing copies of the
returned data if we can. By moving the ownership of the exectuables
return value into the `Frame` struct we achieve this.

A zero-copy implementation of those APIs would be technically possible
without that internal change by making the callsite in the runtime
responsible for moving the returned data into the frame after any call.
However, resetting the stored output needs to be handled in ext, since
plain transfers will _not_ affect the stored return data (and we don't
want to handle this special call case inside the `runtime` API). This
has drawbacks:
- It can not be tested easily in the mock.
- It introduces an inconsistency where resetting the stored output is
handled in ext, but the runtime API is responsible to store it back
correctly after any calls made. Instead, with ownership of the data in
`Frame`, both can be handled in a single place. Handling both in `fn
run()` is more natural and leaves less room for runtime API bugs.

The returned output is reset each time _before_ running any executable
in a nested stack. This change should not incur any overhead to the
overall memory usage as _only_ the returned data from the last executed
frame will be kept around at any time.

---------

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: PG Herveou <pgherveou@gmail.com>

This change adds the required validation for stronger UX guarantees when
using `InitiateReserveWithdraw` or `InitiateTeleport` XCM instructions.
Execution of the instructions will fail if the local chain is not
configured to trust the "destination" or "reserve" chain as a
reserve/trusted-teleporter for the provided "assets".

With this change, misuse of `InitiateReserveWithdraw`/`InitiateTeleport`
fails on origin with no overall side-effects, rather than failing on
destination (with side-effects to origin's assets issuance).

The commit also makes the same validations for pallet-xcm transfers, and
adds regression tests.

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>