This PR exposes the *unstable* metadata V16. The metadata is exposed
under the unstable u32::MAX number.

Developers can start experimenting with the new features of the metadata
v16. *Please note that this metadata is under development and expect
breaking changes until stabilization.*

The `ExtrinsicMetadata` trait receives a breaking change. Its associated
type `VERSION` is rename to `VERSIONS` and now supports a constant
static list of metadata versions.
The versions implemented for `UncheckedExtrinsic` are v4 (legacy
version) and v5 (new version).

For metadata collection, it is assumed that all `TransactionExtensions`
are under version 0.

Builds on top of: https://github.com/paritytech/polkadot-sdk/pull/5274

Closes: https://github.com/paritytech/polkadot-sdk/issues/5980
Closes: https://github.com/paritytech/polkadot-sdk/issues/5347
Closes: https://github.com/paritytech/polkadot-sdk/issues/5285

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: GitHub Action <action@github.com>

Request responses are initialized with a main protocol name, and
optional protocol names as a fallback.

Running litep2p in kusama as a validator has surfaced a `debug_asserts`
coming from the sync component:

https://github.com/paritytech/polkadot-sdk/blob/3906c578c96d97a8a099a4bdac4685acbe375a7c/substrate/client/network/sync/src/strategy/chain_sync.rs#L640-L646

The issue is that we initiate a request-response over the main protocol
name `/genesis/sync/2` but receive a response over the legacy procotol
`ksm/sync/2`. This behavior is correct because litep2p propagates to the
higher levels the protocol that responded.

In contrast, libp2p provides the main protocol name regardless of
negotiating a legacy protocol.

Because of this, higher level components assume that only the main
protocol name will respond.
To not break this assumption, this PR alings litep2p shim layer with the
libp2p behavior.


Closes: https://github.com/paritytech/polkadot-sdk/issues/6581



---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>

The backport bot opens PR for `A4-needs-backport` only for stable2407
stable2409, but we have already stable2412.

The question is, when should we append a new `stable*` branch here?
Should it be done when a new `stable*` branch is created? Can we
automate this process somehow?

The `query_weight_to_asset_fee` function was trying to convert versions
by using `try_as`, this function [doesn't convert from a versioned to a
concrete
type](https://github.com/paritytech/polkadot-sdk/blob/0156ca8f/polkadot/xcm/src/lib.rs#L131).
This would cause all calls with a lower version to fail.

The correct function to use is the good old
[try_into](https://github.com/paritytech/polkadot-sdk/blob/0156ca8f

/polkadot/xcm/src/lib.rs#L184).
Now those calls work :)

---------

Co-authored-by: command-bot <>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: GitHub Action <action@github.com>

See https://github.com/polkadot-fellows/runtimes/issues/248

 : using
`TypeWithDefault` having derived `TypeInfo` for `Nonce` causes a
breaking change in metadata for nonce type because it's no longer `u64`.

Adding a default implementation of `TypeInfo` for `TypeWithDefault` to
restore the original type info in metadata.

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

`FungibleAdapter` will now check if the _refund amount_ is zero before
calling deposit & emitting an event.

Fixes https://github.com/paritytech/polkadot-sdk/issues/6469.

---------

Co-authored-by: GitHub Action <action@github.com>

This PR addresses two small fixes:  

1. Fixed a typo ("as as") found on the way.  
2. Resolved a bug in the `local/remote exporters` used for bridging.
Previously, they consumed `dest` and `msg` without returning them when
inner routers/exporters failed with `NotApplicable`. This PR ensures
compliance with the
[`SendXcm`](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/src/v5/traits.rs#L449-L450)
and
[`ExportXcm`](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/xcm-executor/src/traits/export.rs#L44-L45)
traits.

---------

Co-authored-by: GitHub Action <action@github.com>

This pipeline should replace a manual action done on the `cleamroom`
server to publish the `polkadot` deb package to our apt repo with the
pipeline triggered from the new paritytech-release org.
Right now, this is done manually by running the
[add-packages.sh](https://github.com/paritytech/cleanroom/blob/master/ansible/roles/parity-repos/files/add-packages.sh)
script on the `cleanroom` machine.
What is done under the hood:
- Pipeline downloads `polakdot` deb package from S3, that was prebuilt
in the [Build release rc
pipeline](https://github.com/paritytech/polkadot-sdk/blob/master/.github/workflows/release-build-rc.yml)
- Prepares and syncs local apt repository
- Adds and signs deb package to it using `reprepro`
- Uploads new deb package to the distributed repo

Closes: https://github.com/paritytech/release-engineering/issues/239

The semver-check is failing with the following
[error](https://github.com/paritytech/polkadot-sdk/actions/runs/11908981132/job/33185572284):

```bash
error[E0658]: use of unstable library feature 'error_in_core'
  --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/frame-decode-0.5.0/src/decoding/extrinsic_decoder.rs:56:6
   |
56 | impl core::error::Error for ExtrinsicDecodeError {}
   |      ^^^^^^^^^^^^^^^^^^
   |
   = note: see issue #103765 <https://github.com/rust-lang/rust/issues/103765> for more information
   = help: add `#![feature(error_in_core)]` to the crate attributes to enable
   = note: this compiler was built on 2024-05-31; consider upgrading it if it is out of date
```

This is related to the toolchain nightly version 1.80. In rust, 1.81 the
`core::error::Error` is stable.

After updating the rust-toolchain, parity-publish crate must be updated
as well.
The `cargo-semver-checks` dependency of `parity-publish` crate is
updated from 0.34 to 0.38.
This update enables rustdoc v36 that fixes the following
[issue](https://github.com/paritytech/polkadot-sdk/actions/runs/11912689841/job/33196936011

):


```bash
 validating prdocs...
checking file changes...
checking semver changes...
(1/18) building frame-support-HEAD...
(2/18) building frame-support-28.0.0...
Error: rustdoc format v36 for file /__w/polkadot-sdk/polkadot-sdk/target/doc/frame_support.new is not supported
```

This PR is pending on a release of parity-publish to version 0.9.0
(fixes already on origin/master)

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Bumps [rustls](https://github.com/rustls/rustls) from 0.23.14 to
0.23.18.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/rustls/rustls/commit/33af2c38b0f1e4abf44d59d5b74ccf12f5cf5e56"><code>33af2c3</code></a>
Prepare 0.23.18</li>
<li><a
href="https://github.com/rustls/rustls/commit/ffe646d1ff07d3e1d9f009daadd94228d462425d"><code>ffe646d</code></a>
Add reproducer for bug 2227</li>
<li><a
href="https://github.com/rustls/rustls/commit/69b6f7473a7ae096a9cf9e2d6eb3cd8b22743597"><code>69b6f74</code></a>
Record and restore the processed cursor in first_handshake_message</li>
<li><a
href="https://github.com/rustls/rustls/commit/4ef3532cf2fceda9fdd19947f871adf7020d0b49"><code>4ef3532</code></a>
Upgrade to mio 1</li>
<li><a
href="https://github.com/rustls/rustls/commit/092a16427e81e58ecd6bcfdacc0b49f02bad8db3"><code>092a164</code></a>
Manage dependencies via the workspace</li>
<li><a
href="https://github.com/rustls/rustls/commit/a01bd6bcb536c6cdd490942b0eae2903abcfcab3"><code>a01bd6b</code></a>
rustls-bench: fix warnings with no features</li>
<li><a
href="https://github.com/rustls/rustls/commit/7d74de2c1bdf265531036f50abd3fc04df8c148a"><code>7d74de2</code></a>
tests: linearize new test code helper</li>
<li><a
href="https://github.com/rustls/rustls/commit/499d797b267e0b548fef5225b044b99fc215a8ff"><code>499d797</code></a>
fix: do not send session_ticket(35) extension for TLS 1.3</li>
<li><a
href="https://github.com/rustls/rustls/commit/faca28904efcb3b5a4a5f05be8e03374bf5086df"><code>faca289</code></a>
chore(deps): lock file maintenance</li>
<li><a
href="https://github.com/rustls/rustls/commit/d12f42385c2da74e1ec16826af68d17a13101152"><code>d12f423</code></a>
fix(deps): update rust crate asn1 to 0.20</li>
<li>Additional commits viewable in <a
href="https://github.com/rustls/rustls/compare/v/0.23.14...v/0.23.18">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=rustls&package-manager=cargo&previous-version=0.23.14&new-version=0.23.18)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/paritytech/polkadot-sdk/network/alerts

).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Fix #6575.

cc: @iulianbarbu

Co-authored-by: Iulian Barbu <14218860+iulianbarbu@users.noreply.github.com>

# Description

The PR ensures that the id_provider variable is cloned instead of taken,
which can help prevent issues related id provider being reset to the
default.

In [a test in
moonbeam](https://github.com/moonbeam-foundation/moonbeam/blob/c6d07d703dfcdd94cc311fa83b553071b7d433ff/test/suites/dev/moonbase/test-subscription/test-subscription.ts#L20-L31)
we found that the id_provider is being reset somehow and changed to the
default one. Changing .take() to .clone() would fix the issue.


# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process


) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)
* [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

---------

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

Closes #6415 

# Description

Remove unused message `ReportCollator` and test related to this message
on the collator protocol validator side.

cc: @tdimitrov



---------

Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: command-bot <>

PR addresses
https://github.com/paritytech/polkadot-sdk/pull/6265#issuecomment-2497506857

cc https://github.com/paritytech/ci_cd/issues/1084

Closes https://github.com/paritytech/ci_cd/issues/1067

Fixes https://github.com/paritytech/polkadot-sdk/issues/6209

This PR adds the support for cfg attributes in the runtime macro.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Before this was done for every imported transaction. When a lot of
transactions got imported, the import notification channel was filled.
The underlying problem was that the `status` call is read locking the
`validated_pool` which will be write locked by the internal submitting
logic. Thus, the submitting and status reading was interferring which
each other.

---------

Co-authored-by: GitHub Action <action@github.com>

Co-authored-by: Robert <robert@gonimo.com>
Co-authored-by: ordian <write@reusable.software>

Step in https://github.com/paritytech/polkadot-sdk/issues/3268

This PR adds the ability for these pallets to specify their source of
the block number. This is useful when these pallets are migrated from
the relay chain to a parachain and vice versa.

This change is backwards compatible:
1. If the `BlockNumberProvider` continues to use the system pallet's
block number
2. When a pallet deployed on the relay chain is moved to a parachain,
but still uses the relay chain's block number

However, we would need migrations if the deployed pallets are upgraded
on an existing parachain, and the `BlockNumberProvider` uses the relay
chain block number.

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Add support for all eth tx types
Note that js libs will continue to use the Legacy type since we don't
include base_fee_per_gas yet in the block.
We can think about setting these values after we revisit how we encode
the gas into weight & deposit_limit in a follow up PR

---------

Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: GitHub Action <action@github.com>

# Description

Reused as before the `properties` variable when defining a development
chain spec for parachain-template-node.

## Integration

N/A

## Review Notes

One line change, pretty self explanatory (it got lost within the history
of changes over the parachain-template-node/chain_spec.rs file). To be
honest, not really sure how useful it is, but I had the choice of
removing the `properties` var or reuse it as before, and I went with the
latter.

Signed-off-by: Iulian Barbu <iulian.barbu@parity.io>

[Fix|NominationPools] Only allow apply slash to be executed if the slash amount is atleast ED (#6540)

This change prevents `pools::apply_slash` from being executed when the
pending slash amount of the member is lower than the ED.

The issue came to light with the failing [benchmark
test](https://github.com/polkadot-fellows/runtimes/actions/runs/11879471717/job/33101445269?pr=490#step:11:765

)
in Kusama. The problem arises from the inexact conversion between points
and balance. Specifically, when points are converted to balance and then
back to points, rounding can introduce a small discrepancy between the
input and the resulting value. This issue surfaced in Kusama due to its
ED being different from Westend and Polkadot (1 UNIT/300), making the
rounding issue noticeable.

This fix is also significant because applying a slash is feeless and
permissionless. Allowing super small slash amounts to be applied without
a fee is undesirable. With this change, such small slashes will still be
applied but only when member funds are withdrawn.

---------

Co-authored-by: Dónal Murray <donal.murray@parity.io>

# Description

This PR adds the required changes to release `polkadot`,
`polkadot-parachain` and `polkadot-omni-node` binaries built on Apple
Sillicon macos.

## Integration

This addresses requests from the community for such binaries: #802, and
they should be part of the Github release page.

## Review Notes

Test on paritytech-stg solely focused on macos binaries:
https://github.com/paritytech-stg/polkadot-sdk/actions/runs/11824692766/job/32946793308

,
except the steps related to `pgpkms` (which need AWS credentials,
missing from paritytech-stg). The binary names don't have a `darwin-arm`
identifier, and conflict with the existing x86_64-linux binaries. I
haven't tested building everything on `paritytech-stg` because the
x86_64-linux builds run on `unbutu-latest-m` which isn't enabled on
`pairtytech-stg` (and I haven't asked CI team to enable one), so testing
how to go around naming conflicts should be covered next.

### TODO

- [x] Test the workflow start to end (especially the last bits related
to uploading the binaries on S3 and ensuring the previous binaries and
the new ones coexist harmoniously on S3/action artifacts storage without
naming conflicts) @EgorPopelyaev
- [x] Publish the arm binaries on the Github release page - to clarify
what's needed @iulianbarbu . Current practice is to manually publish the
binaries built via `release-build-rc.yml` workflow, taken from S3. Would
be great to have the binaries there in the first place before working on
automating this, but I would also do it in a follow up PR.

### Follow ups

- [ ] unify the binaries building under
`release-30_publish_release_draft.yml` maybe?
- [ ] automate binary artifacts upload to S3 in
`release-30_publish_release_draft.yml`

---------

Signed-off-by: Iulian Barbu <iulian.barbu@parity.io>
Co-authored-by: EgorPopelyaev <egor@parity.io>

Move spawning of the slot-based collator into the `run` function. Also
the tasks are being spawned as blocking task and not just as normal
tasks.

---------

Co-authored-by: GitHub Action <action@github.com>

The `GossipEngine::poll_next` implementation polls both the
`notification_service` and the `sync_event_stream`.

If both polls produce valid data to be processed
(`Poll::Ready(Some(..))`), then the sync event is ignored when we
receive `NotificationEvent::NotificationStreamOpened` and the role
cannot be deduced.

This PR ensures both events are processed gracefully. While at it, I
have added a warning to the sync engine related to
`notification_service` producing `Poll::Ready(None)`.

This effectively ensures that `SyncEvents` propagate to the network
potentially fixing any state mismatch.


For more context: https://github.com/paritytech/polkadot-sdk/issues/6507

cc @paritytech/sdk-node

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

The eth RPC tests fail sometimes because they run into a connect timeout
because the node takes a long time to start. This bumps the connect
timeout from 30 to 120 seconds. Locally they take around 40s for me.

As a drive by I also remove a apparently duplicated nextest config.

---------

Co-authored-by: ordian <write@reusable.software>

https://github.com/paritytech/polkadot-sdk/pull/3970 updated the
treasury pallet to support relay chain block number provider. However,
it added a constraint to the BlockNumberProvider to have the same block
number type as frame_system:

```rust
type BlockNumberProvider: BlockNumberProvider<BlockNumber = BlockNumberFor<Self>>;
```

This PR removes that constraint as suggested by @gui1117

Part of:

- #6202.

---------

Signed-off-by: Xavier Lau <x@acg.box>
Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>
Co-authored-by: Guillaume Thiolliere <guillaume.thiolliere@parity.io>

This PR includes:  
- Refactored integrity tests to support standalone deployment of
`pallet-bridge-messages`.
- Refactored the `open_and_close_bridge_works` test case to support
multiple scenarios, such as:
  1. A local chain opening a bridge.  
  2. Sibling parachains opening a bridge.  
  3. The relay chain opening a bridge.  
- Previously, we added instance support for `pallet-bridge-relayer` but
overlooked updating the `DeliveryConfirmationPaymentsAdapter`.

---------

Co-authored-by: GitHub Action <action@github.com>

The default trie cache size before was set to `64MiB`, which is quite
low to achieve real speed ups. `1GiB` should be a reasonable number as
the requirements for validators/collators/full nodes are much higher
when it comes to minimum memory requirements. Also the cache will not
use `1GiB` from the start and fills over time. The setting can be
changed by setting `--trie-cache-size BYTE_SIZE`.

---------

Co-authored-by: GitHub Action <action@github.com>

# Description

https://github.com/paritytech/polkadot-sdk/pull/4613 introduced events
for `pallet_conviction_voting::{vote, remove_vote, remove_other_vote}`.
However:
1. it did not include `unlock`
2. the pallet's unit tests were missing an update

## Integration

N/A

## Review Notes

This is as https://github.com/paritytech/polkadot-sdk/pull/6261 was, so
it is a trivial change.

This PR is the second part of the pure state sync refactoring,
encapsulating `StateSyncMetadata` as a separate entity. Now it's pretty
straightforward what changes are needed for the persistent state sync as
observed in the struct `StateSync`:

- `state`: redirect directly to the DB layer instead of being
accumulated in the memory.
- `metadata`: handle the state sync metadata on disk whenever the state
is forwarded to the DB, resume an ongoing state sync on a restart, etc.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

This is part 2 of
https://github.com/paritytech/polkadot-sdk/issues/5406#issuecomment-2325064863,
properly handling the block gap generated during fast sync.

Although #5406 remains unresolved due to the known issues in #5663, I
decided to open up this PR earlier than later to speed up the overall
progress. I've tested the fast sync locally with this PR, and it appears
to be functioning well. (I was doing a fast sync from a discontinued
archive node locally, thus the issue highlighted in
https://github.com/paritytech/polkadot-sdk/pull/5663#discussion_r1752124039
was bypassed exactly.)

Once the edge cases in #5663 are addressed, we can move forward by
removing the body attribute from the LightState block request and
complete the work on #5406. The changes in this PR are incremental, so
reviewing commit by commit should provide the best clarity.

cc @dmitry-markin



---------

Co-authored-by: Bastian Köcher <git@kchr.de>

This pull request forward all the logging directives given to the node
via `RUST_LOG` or `-l` to the workers, instead of only forwarding
`RUST_LOG`.

---------

Co-authored-by: GitHub Action <action@github.com>

# Description

Migrates pallet-democracy benchmarks to benchmark v2 syntax
This is Part of https://github.com/paritytech/polkadot-sdk/issues/6202

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

Aims to implement Stage 3 of Validator Disbling as outlined here:
https://github.com/paritytech/polkadot-sdk/issues/4359

Features:
- [x] New Disabling Strategy (Staking level)
- [x] Re-enabling logic (Session level)
- [x] More generic disabling decision output
- [x] New Disabling Events

Testing & Security:
- [x] Unit tests
- [x] Mock tests
- [x] Try-runtime checks
- [x] Try-runtime tested on westend snap
- [x] Try-runtime CI tests
- [ ] Re-enabling Zombienet Test (?)
- [ ] SRLabs Audit

Closes #4745 
Closes #2418

---------

Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>

Fix prometheus metrics not shutting down if there are open connections.
I fixed the same issue in the past but it broke again after a dependecy
upgrade.

See also:

https://github.com/paritytech/polkadot-sdk/pull/1637

Enhance the `delegate_call` function to accept an `address` target
parameter instead of a `code_hash`. This allows direct identification of
the target contract using the provided address.
Additionally, introduce parameters for specifying a customizable
`ref_time` limit and `proof_size` limit, thereby improving flexibility
and control during contract interactions.

---------

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

# Description

Opening another PR because I added a test to check for my fix pushed in
#6486 and realized that for some reason I completely forgot how to code
and did not fix the underlying issue, since out-of-bounds indexing could
still happen even with the check I added. This one should fix that and,
as an added bonus, has a simple test used as an integrity check to make
sure future changes don't accidently revert this fix.

Now `sp-trie` should definitely not panic when faced with bad
`CompactProof`s. Sorry about that 



This, like #6486, is related to issue #6485

## Integration

No changes have to be done downstream, and as such the version bump
should be minor.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>