# Description

The EthereumBlobExporter consumes the `dest` parameter when the
destination is not `Here`. Subsequent exporters will receive a `None`
value for the destination instead of the original destination value,
which is incorrect.
 
Closes #5788

## Integration

Minor fix related to the exporter behaviour.

## Review Notes

Verified that tests
`exporter_validate_with_invalid_dest_does_not_alter_destination` and
`exporter_validate_with_invalid_universal_source_does_not_alter_universal_source`
fail without the fix in the exporter.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

Relates to: https://github.com/paritytech/polkadot-sdk/pull/5916
Relates to: https://github.com/polkadot-js/api/pull/5976

---------

Co-authored-by: Javier Viola <javier@parity.io>

This PR adds the `stableYYMM-rcX` or `stableYYMM-X-rcX` tags to the
docker images, so that they could be published with the new tag naming
scheme.

Closes: https://github.com/paritytech/release-engineering/issues/224

Removing the shell node variant for the polkadot-parachain as discussed
here:
https://github.com/paritytech/polkadot-sdk/pull/5586#discussion_r1752635254

Resolves https://github.com/paritytech/polkadot-sdk/issues/5898

This PR removes the requirement to set the `LaneId` in the relayer CLI
configuration where it was not really necessary.

---------

Co-authored-by: command-bot <>

- update baseline for pallet_revive
- update cmd pipeline name
- Fix compilation after renaming some of benchmarks in pallet_revive.
[Runtime Dev]. Changed the "instr" benchmark so that it should no longer
return to little weight. It is still bogus but at least benchmarking
should not work. (by @athei

 )

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: command-bot <>

Close https://github.com/paritytech/polkadot-sdk/issues/5589

This PR makes it possible for `rpc_v2::Storage::query_iter_paginated` to
be "backpressured" which is achieved by having a channel where the
result is sent back and when this channel is "full" we pause the
iteration.

The chainHead_follow has an internal channel which doesn't represent the
actual connection and that is set to a very small number (16). Recall
that the JSON-RPC server has a dedicate buffer for each connection by
default of 64.

#### Notes

- Because `archive_storage` also depends on
`rpc_v2::Storage::query_iter_paginated` I had to tweak the method to
support limits as well. The reason is that archive_storage won't get
backpressured properly because it's not an subscription. (it would much
easier if it would be a subscription in rpc v2 spec because nothing
against querying huge amount storage keys)
- `query_iter_paginated` doesn't necessarily return the storage "in
order" such as
- ...

Bump `zombienet` version, including fixes (`ci`) and the latest version
of `pjs` embedded.
Thx!

Resolves https://github.com/paritytech/polkadot-sdk/issues/5026

This PR adds support for starting a dev node with manual seal consensus.
This can be done by using the `--dev-block-time` argument . For example:
```
polkadot-parachain --chain asset-hub-rococo-dev --dev-block-time 5000 --tmp
```

Added `HashedDescription<AccountId,
DescribeFamily<DescribeAllTerminal>>` foreign locations to local
accounts converter to all the parachains.

---------

Co-authored-by: command-bot <>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

Quoting @bkchr (from
[here](https://github.com/paritytech/polkadot-sdk/issues/5334#issuecomment-2383815078)):

> That the hardcoded limit is used there again, is IMO not correct. The
`max_pov_size` should be controlled by the `HostConfiguration` and not
limited by some "random" constant.

This PR aims to change the hard limit to a not-so-random constant,
allowing more room for maneuvering in the future.

Only retry jobs on `runner_system_failure`.
Thx!

Fixes https://github.com/paritytech/polkadot-sdk/issues/5617

Normally, the BEEFY protocol only accepts a single MMR Root entry in a
commitment's payload. But to be extra careful, when validating
equivocation reports, let's check all the MMR roots, if there are more.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

# Description

This PR removes the
`CandidateValidationMessage::ValidateFromChainState`, which was
previously used by backing, but is no longer relevant since initial
async backing implementation
https://github.com/paritytech/polkadot/pull/5557.

Fixes https://github.com/paritytech/polkadot-sdk/issues/5643

## Integration

This change should not affect downstream projects since
`ValidateFromChainState` was already unused.

## Review Notes

- Removed all occurrences of `ValidateFromChainState`.
- Moved utility functions, previously used in candidate validation tests
and malus, exclusively to candidate validation tests as they are no
longer used in malus.
- Deleted the
`polkadot_parachain_candidate_validation_validate_from_chain_state`
metric from Prometheus.
- Removed `Spawner` from `ReplaceValidationResult` in malus’
interceptors.
- `fake_validation_error` was only used for `ValidateFromChainState`
handling, while other cases directly used
`InvalidCandidate::Invalid...

Bumps the known_good_semver group with 1 update:
[syn](https://github.com/dtolnay/syn).

Updates `syn` from 2.0.77 to 2.0.79
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/syn/releases">syn's
releases</a>.</em></p>
<blockquote>
<h2>2.0.79</h2>
<ul>
<li>Fix infinite loop on parsing chained ranges (<a
href="https://redirect.github.com/dtolnay/syn/issues/1741">#1741</a>)</li>
<li>Fix panic in parsing <code>use</code> items containing absolute
paths (<a
href="https://redirect.github.com/dtolnay/syn/issues/1742">#1742</a>)</li>
</ul>
<h2>2.0.78</h2>
<ul>
<li>Fix infinite loop on chained comparison (<a
href="https://redirect.github.com/dtolnay/syn/issues/1739">#1739</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/dtolnay/syn/commit/732e6e39406538aebe34ed5f5803113a48c28602"><code>732e6e3</code></a>
Release 2.0.79</li>
<li><a
h...

# Description

close #5641

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

This PR backports regular version bumps and prdocs reordering from the
`stable2409` release branch to `master`

---------

Co-authored-by: Morgan Adamiec <morgan@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>

cc @AndreiEres

This is a refactor and improvement from:
https://github.com/paritytech/polkadot-sdk/pull/3881

- `sp_runtime::proving_trie` now exposes a `BasicProvingTrie` for both
`base2` and `base16`.
- APIs for `base16` are more focused on single value proofs, also
aligning their APIs with the `base2` trie
- A `ProvingTrie` trait is included which wraps both the `base2` and
`base16` trie, and exposes all APIs needed for an end to end scenario.
- A `ProofToHashes` trait is exposed which can allow us to write proper
benchmarks for the merkle trie.

---------

Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

# Description

Updates the doc comment on the `import_notification_stream` to make its
behaviour clearer.

Closes [Unexpected behaviour of block
`import_notification_stream`](https://github.com/paritytech/polkadot-sdk/issues/5596).

## Integration

Doesn't apply.

## Review Notes

The old comment docs caused some confusion to myself and some members of
my team, on when this notification stream is triggered. This is
reflected in the linked
[issue](https://github.com/paritytech/polkadot-sdk/issues/5596), and as
discussed there, this PR aims to prevent this confusion in future devs
looking to make use of this functionality.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [x] I have made corresponding changes to the documentation (if
applicable)
* [x] I have added tests that prove my fix is effective or that my
feature works (if applicable)

You can remove the "Checklist" section once all have been checked. Thank
you for your contribution!

---------

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

Just a tiny config fix

Co-authored-by: Bastian Köcher <git@kchr.de>

# Description

`substrate-wasm-builder` can be a build dependency for crates which
develop FRAME runtimes. I had a tough time seeing errors happening in
such crates (e.g. runtimes from the `templates` directory) in my IDE. I
use a combination of rust-analyzer + nvim-lsp + nvim-lspconfig +
rustacean.vim and all of this stack is not able to correctly parse
errors emitted during the `build` phase.

As a matter of fact there is also a cargo issue tracking specifically
this issue where cargo doesn't propagate the `--message-format` type to
the build phase: [here](https://github.com/rust-lang/cargo/issues/14246)
initially and then
[here](https://github.com/rust-lang/cargo/issues/8283). It feels like a
solution for this use case isn't very close, so if it comes to runtimes
development (both as an SDK user and developer), enabling wasm builder
to emit diagnostics messages friendly to IDEs would be useful for
regular workflows where IDEs are used...

This PR removes a workaround which had a reference comment to a rust
compiler issue. The issue has been fixed and we should be able to remove
that trait bound.

- install frame-omni-bencher from the path
- fix bench_features config

Reported in:
#5844
#5848

Jaeger spans were not usable for debugging, see
https://github.com/paritytech/polkadot-sdk/issues/4995, but we pay a
price in CPU cost, subsystem-benchmarks show this brings a reduction of
about 10-15% in CPU usage per subsystem, so remove it.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

cc https://github.com/paritytech/ci_cd/issues/1035

cc https://github.com/paritytech/ci_cd/issues/1023

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Maksym H <1177472+mordamax@users.noreply.github.com>
Co-authored-by: gui <gui.thiolliere@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>

Bump `zombienet` version to prevent report fails at teardown phase.

Changes in PR:
- disabled cargo-hfuzz until [the
issue](https://github.com/paritytech/polkadot-sdk/issues/5812) is fixed.
- enabled condition to skip jobs when no rust files are changed

This is the implementation of the approach described here:
https://github.com/paritytech/polkadot-sdk/issues/1617#issuecomment-2150321612
&
https://github.com/paritytech/polkadot-sdk/issues/1617#issuecomment-2154357547
&
https://github.com/paritytech/polkadot-sdk/issues/1617#issuecomment-2154721395.

## Description of changes

The end goal is to have an architecture where we have single
subsystem(`approval-voting-parallel`) and multiple worker types that
would full-fill the work that currently is fulfilled by the
`approval-distribution` and `approval-voting` subsystems. The main loop
of the new subsystem would do just the distribution of work to the
workers.

The new subsystem will have:
- N approval-distribution workers: This would do the work that is
currently being done by the approval-distribution subsystem and in
addition to that will also perform the crypto-checks that an assignment
is valid and that a vote is correctly signed. Work is assigned via the
following formula: `worker_index = msg.validator % WORKER_COUNT`, this
guarantees that all assignments and approvals from the same validator
reach the same worker.
- 1 approval-voting worker: This would receive an already valid message
and do everything the approval-voting currently does, except the
crypto-checking that has been moved already to the approval-distribution
worker.

On the hot path of processing messages **no** synchronisation and
waiting is needed between approval-distribution and approval-voting
workers.

<img width="1431" alt="Screenshot 2024-06-07 at 11 28 08"
src="https://github.com/paritytech/polkadot-sdk/assets/49718502/a196199b-b705-4140-87d4-c6900ba8595e">



## Guidelines for reading

The full implementation is broken in 5 PRs and all of them are
self-contained and improve things incrementally even without the
parallelisation being implemented/enabled, the reason this approach was
taken instead of a big-bang PR, is to make things easier to review and
reduced the risk of breaking this critical subsystems.

After reading the full description of this PR, the changes should be
read in the following order:
1. https://github.com/paritytech/polkadot-sdk/pull/4848, some other
micro-optimizations for networks with a high number of validators. This
change gives us a speed up by itself without any other changes.
2. https://github.com/paritytech/polkadot-sdk/pull/4845 , this contains
only interface changes to decouple the subsystem from the `Context` and
be able to run multiple instances of the subsystem on different threads.
**No functional changes**
3. https://github.com/paritytech/polkadot-sdk/pull/4928, moving of the
crypto checks from approval-voting in approval-distribution, so that the
approval-distribution has no reason to wait after approval-voting
anymore. This change gives us a speed up by itself without any other
changes.
4. https://github.com/paritytech/polkadot-sdk/pull/4846, interface
changes to make approval-voting runnable on a separate thread. **No
functional changes**
5. This PR, where we instantiate an `approval-voting-parallel` subsystem
that runs on different workers the logic currently in
`approval-distribution` and `approval-voting`.
6. The next step after this changes get merged and deploy would be to
bring all the files from approval-distribution, approval-voting,
approval-voting-parallel into a single rust crate, to make it easier to
maintain and understand the structure.

## Results
Running subsystem-benchmarks with 1000 validators 100 fully ocuppied
cores and triggering all assignments and approvals for all tranches

#### Approval does not lags behind. 
 Master
```
Chain selection approved  after 72500 ms hash=0x0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a
```
With this PoC
```
Chain selection approved  after 3500 ms hash=0x0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a0a
```

#### Gathering enough assignments
 
Enough assignments are gathered in less than 500ms, so that gives un a
guarantee that un-necessary work does not get triggered, on master on
the same benchmark because the subsystems fall behind on work, that
number goes above 32 seconds on master.
 
<img width="2240" alt="Screenshot 2024-06-20 at 15 48 22"
src="https://github.com/paritytech/polkadot-sdk/assets/49718502/d2f2b29c-5ff6-44b4-a245-5b37ab8e58bc">


#### Cpu usage:
Master
```
CPU usage, seconds                     total   per block
approval-distribution                96.9436      9.6944
approval-voting                     117.4676     11.7468
test-environment                     44.0092      4.4009
```
With this PoC
```
CPU usage, seconds                     total   per block
approval-distribution                 0.0014      0.0001 --- unused
approval-voting                       0.0437      0.0044.  --- unused
approval-voting-parallel              5.9560      0.5956
approval-voting-parallel-0           22.9073      2.2907
approval-voting-parallel-1           23.0417      2.3042
approval-voting-parallel-2           22.0445      2.2045
approval-voting-parallel-3           22.7234      2.2723
approval-voting-parallel-4           21.9788      2.1979
approval-voting-parallel-5           23.0601      2.3060
approval-voting-parallel-6           22.4805      2.2481
approval-voting-parallel-7           21.8330      2.1833
approval-voting-parallel-db          37.1954      3.7195.  --- the approval-voting thread.
```

# Enablement strategy

Because just some trivial plumbing is needed in approval-distribution
and approval-voting to be able to run things in parallel and because
this subsystems plays a critical part in the system this PR proposes
that we keep both ways of running the approval work, as separated
subsystems and just a single subsystem(`approval-voting-parallel`) which
has multiple workers for the distribution work and one worker for the
approval-voting work and switch between them with a comandline flag.

The benefits for this is twofold.
1. With the same polkadot binary we can easily switch just a few
validators to use the parallel approach and gradually make this the
default way of running, if now issues arise.
2. In the worst case scenario were it becomes the default way of running
things, but we discover there are critical issues with it we have the
path to quickly disable it by asking validators to adjust their command
line flags.


# Next steps
- [x] Make sure through various testing we are not missing anything 
- [x] Polish the implementations to make them production ready
- [x] Add Unittest Tests for approval-voting-parallel.
- [x] Define and implement the strategy for rolling this change, so that
the blast radius is minimal(single validator) in case there are problems
with the implementation.
- [x]  Versi long running tests.
- [x] Add relevant metrics.

@ordian @eskimor @sandreim @AndreiEres

, let me know what you think.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

Closes #5714

A change that I missed to add in
https://github.com/paritytech/polkadot-sdk/pull/5525 .

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

This PR introduces 2 new syscalls: `return_data_size` and
`return_data_copy`, resembling the semantics of the EVM `RETURNDATASIZE`
and `RETURNDATACOPY` opcodes.

The ownership of `ExecReturnValue` (the return data) has moved to the
`Frame`. This allows implementing the new contract API functionality in
ext with no additional copies. Returned data is passed via contract
memory, memory is (will be) metered, hence the amount of returned data
can not be statically known, so we should avoid storing copies of the
returned data if we can. By moving the ownership of the exectuables
return value into the `Frame` struct we achieve this.

A zero-copy implementation of those APIs would be technically possible
without that internal change by making the callsite in the runtime
responsible for moving the returned data into the frame after any call.
However, resetting the stored output needs to be handled in ext, since
plain transfers will _not_ affect the stored return data (and we don't
want to handle this special call case inside the `runtime` API). This
has drawbacks:
- It can not be tested easily in the mock.
- It introduces an inconsistency where resetting the stored output is
handled in ext, but the runtime API is responsible to store it back
correctly after any calls made. Instead, with ownership of the data in
`Frame`, both can be handled in a single place. Handling both in `fn
run()` is more natural and leaves less room for runtime API bugs.

The returned output is reset each time _before_ running any executable
in a nested stack. This change should not incur any overhead to the
overall memory usage as _only_ the returned data from the last executed
frame will be kept around at any time.

---------

Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: PG Herveou <pgherveou@gmail.com>

This change adds the required validation for stronger UX guarantees when
using `InitiateReserveWithdraw` or `InitiateTeleport` XCM instructions.
Execution of the instructions will fail if the local chain is not
configured to trust the "destination" or "reserve" chain as a
reserve/trusted-teleporter for the provided "assets".

With this change, misuse of `InitiateReserveWithdraw`/`InitiateTeleport`
fails on origin with no overall side-effects, rather than failing on
destination (with side-effects to origin's assets issuance).

The commit also makes the same validations for pallet-xcm transfers, and
adds regression tests.

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>

# MBM try-runtime support

This MR adds support to the try-runtime trait such that the
try-runtime-CLI will be able to support MBM testing
[here](https://github.com/paritytech/try-runtime-cli/pull/90). It mainly
adds two feature-gated hooks to the `SteppedMigration` hook to
facilitate testing. These hooks are named `pre_upgrade` and
`post_upgrade` and have the same signature and implications as for
single-block migrations.

## Integration

To make use of this in your Multi-Block-Migration, just implement the
two new hooks and test pre- and post-conditions in them:

```rust
#[cfg(feature = "try-runtime")]
fn pre_upgrade() -> Result<Vec<u8>, frame_support::sp_runtime::TryRuntimeError> {
	// ...
}

#[cfg(feature = "try-runtime")]
fn post_upgrade(prev: Vec<u8>) -> Result<(), frame_support::sp_runtime::TryRuntimeError> {
    // ...
}
```

You may return an error or panic in these functions to indicate failure.
This will then show up in the try-runtime-CLI and can be used in CI for
testing.

Changes:
- Adds `try-runtime` gated methods `pre_upgrade` and `post_upgrade` on
`SteppedMigration`
- Adds `try-runtime` gated methods `nth_pre_upgrade` and
`nth_post_upgrade` on `SteppedMigrations`
- Modifies `pallet_migrations` implementation to run pre_upgrade and
post_upgrade steps at the appropriate times, and panic in the event of
migration failure.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: claravanstaden <claravanstaden64@gmail.com>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>
Co-authored-by: georgepisaltu <george.pisaltu@parity.io>

Minor cleanup.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

This PR primarily fixes the issue with
`zombienet-bridges-0001-asset-transfer-works` (see:
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7404903).

The PR looks large, but most of the changes involve splitting `LaneId`
into `LegacyLaneId` and `HashedLaneId`. All pallets now use `LaneId` as
a generic parameter.

The actual bridging pallets are now backward compatible and work with
actual **substrate-relay v1.6.10**, which does not even known anything
about permissionless lanes or the new pallet changes.



## Important

- [x] added migration for `pallet_bridge_relayers` and
`RewardsAccountParams` change order of params, which generates different
accounts

## Deployment follow ups
- [ ] fix monitoring for
`at_{}_relay_{}_reward_for_msgs_from_{}_on_lane_{}`
- [ ] check sovereign reward accounts - because of changed
`RewardsAccountParams`
- [ ] deploy another messages instances for permissionless lanes - on
BHs or AHs?
- [ ] return back `open_and_close_bridge_works` for another
`pallet-bridge-messages` instance

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Fix `parachain-template-test` (bump `zombienet` version).
Thx!