Closes #6196 
Closes #7204

Example of PR: https://github.com/paritytech/polkadot-sdk/pull/6816

Every sunday 01:00 AM it's going to start to benchmark (with /cmd bench)
all runtimes and all pallets
Then diff total will be pushed to a branch and PR open,. I assume
review-bot is going assign required reviewers per changed files

I afraid each weeks will be too much to review & merge, but we can
adjust later

Bonus: fix for pallet_multisig lib and
substrate/.maintain/frame-weight-template.hbs , which didn't let to
compile new weights

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Currently the `para_backing_state` API is used only by the prospective
parachains subsystems and returns 2 things: the constraints for
parachain blocks and the candidates pending availability.

This PR deprecates `para_backing_state` and introduces a new
`backing_constraints` API that can be used together with
`candidates_pending_availability` to get the same information provided
by `para_backing_state`.

TODO:
- [x] PRDoc

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: command-bot <>

Fixed condition which sets weights/large images

Aims to 
- close #7049 
- close https://github.com/paritytech/opstooling/issues/449
- close https://github.com/paritytech/opstooling/issues/463

What's changed:
- removed is paritytech member check as required prerequisite to run a
command
- run the cmd.py script taking it from master, if someone who run this
is not a member of paritytech, and from current branch, if is a member.
That keeps the developer experience & easy testing if paritytech members
are contributing to cmd.py
- isolate the cmd job from being able to access GH App token or PR
token- currently the fmt/bench/prdoc are being run with limited
permissions scope, just to generate output, which then is uploaded to
artifacts, and then the other job which doesn't run any files from repo,
does a commit/push more securely

This is the right value after
https://github.com/paritytech/polkadot-sdk/pull/4880, which corresponds
to an allowedAncestryLen of 2 (which is the default)

WIll fix https://github.com/paritytech/polkadot-sdk/issues/7105

This PR contains small fixes identified during work on the larger PR:
https://github.com/paritytech/polkadot-sdk/issues/6906.

---------

Co-authored-by: command-bot <>

# Description

Closes https://github.com/paritytech/polkadot-sdk/issues/7265.

## Integration

Requires changes in
`https://github.com/polkadot-js/api/packages/{rpc-augment,types-support,types}`
to be visible in Polkadot\Substrate Portal and in other libraries where
we should explicitly state RPC methods.

Accompany PR to `polkadot-js/api`:
https://github.com/polkadot-js/api/pull/6070.

## Review Notes

Please put the right label on my PR.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

# Derived Dev Accounts

Resolves https://github.com/paritytech/polkadot-sdk/issues/6040

## Description
This update introduces support for creating an arbitrary number of
developer accounts at the genesis block based on a specified derivation
path. This functionality is gated by the runtime-benchmarks feature,
ensuring it is only enabled during benchmarking scenarios.

### Key Features
- Arbitrary Dev Accounts at Genesis: Developers can now specify any
number of accounts to be generated at genesis using a hard derivation
path.

- Default Derivation Path: If no derivation path is provided (i.e., when
`Option<dev_accounts: (..., None)>` is set to `Some` at genesis), the
system will default to the path `//Sender//{}`.

- No Impact on Total Token Issuance: Developer accounts are excluded
from the total issuance of the token supply at genesis, ensuring they do
not affect the overall balance or token distribution.

polkadot address: 14SRqZTC1d8rfxL8W1tBTnfUBPU23ACFVPzp61FyGf4ftUFg

---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

This PR contains a fix for the rc-build release pipeline. The problem
was, that for the deb package build were all the artefacts downloaded
and merged together, what could lead to the issue, that the polkadot
linux binary was overwritten with the macos one.

- Fix the EVM fee cost estimation.
The estimation shown in EVM wallet was using Native instead of EVM
decimals
- Remove the precise code length estimation in dry run call.
Over-estimating is fine, since extra gas is refunded anyway.
- Ensure that the estimated fee calculated from gas_price x gas use the
encoded weight & deposit limit instead of the exact one calculated by
the dry-run. Else we can end up with a fee that is lower than the actual
fee paid by the user

---------

Co-authored-by: command-bot <>

Part of https://github.com/paritytech/polkadot-sdk/issues/6504

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

This PR enforces that outbound requests are finished within the
specified protocol timeout.

The stable2412 version running libp2p 0.52.4 contains a bug which does
not track request timeouts properly:
- https://github.com/libp2p/rust-libp2p/pull/5429

The issue has been detected while submitting libp2p -> litep2p requests
in kusama. This aims to check that pending outbound requests have not
timedout. Although the issue has been fixed in libp2p, there might be
other cases where this may happen. For example:
- https://github.com/libp2p/rust-libp2p/pull/5417

For more context see:
https://github.com/paritytech/polkadot-sdk/issues/7076#issuecomment-2596085096


1. Ideally, the force-timeout mechanism in this PR should never be
triggered in production. However, origin/stable2412 occasionally
encounters this issue. When this happens, 2 warnings may be generated:
- one warning introduced by this PR wrt force timeout terminating the
request
- possible one warning when the libp2p decides (if at all) to provide
the response back to substrate (as mentioned by @alexggh
[here](https://github.com/paritytech/polkadot-sdk/pull/7222/files#diff-052aeaf79fef3d9a18c2cfd67006aa306b8d52e848509d9077a6a0f2eb856af7L769)
and
[here](https://github.com/paritytech/polkadot-sdk/pull/7222/files#diff-052aeaf79fef3d9a18c2cfd67006aa306b8d52e848509d9077a6a0f2eb856af7L842)

2. This implementation does not propagate to the substrate service the
`RequestFinished { error: .. }`. That event is only used internally by
substrate to increment metrics. However, we don't have the peer
information available to propagate the event properly when we
force-timeout the request. Considering this should most likely not
happen in production (origin/master) and that we'll be able to extract
information by warnings, I would say this is a good tradeoff for code
simplicity:


https://github.com/paritytech/polkadot-sdk/blob/06e3b5c6

/substrate/client/network/src/service.rs#L1543


### Testing

Added a new test to ensure the timeout is reached properly, even if
libp2p does not produce a response in due time.

I've also transitioned the tests to using `tokio::test` due to a
limitation of
[CI](https://github.com/paritytech/polkadot-sdk/actions/runs/12832055737/job/35784043867)

```
--- TRY 1 STDERR:        sc-network request_responses::tests::max_response_size_exceeded ---
thread 'request_responses::tests::max_response_size_exceeded' panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/time/interval.rs:139:26:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
```



cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

Closes #7055

@skunert @bkchr



---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

Fix bridge tests image

## Issue
[[#7107] Core Index Mismatch in Commitments and
Descriptor](https://github.com/paritytech/polkadot-sdk/issues/7107)

## Description
This PR resolves a bug where normal (non-malus) undying collators failed
to generate and submit collations, resulting in the following error:

`ERROR tokio-runtime-worker parachain::collation-generation: Failed to
construct and distribute collation: V2 core index check failed: The core
index in commitments doesn't match the one in descriptor.`

More details about the issue and reproduction steps are described in the
[related issue](https://github.com/paritytech/polkadot-sdk/issues/7107).

## Summary of Fix
- When core selectors are provided in the UMP signals, core indexes will
be chosen using them;
- The fix ensures that functionality remains unchanged for parachains
not using UMP signals;
- Added checks to stop processing if the same core is selected
repeatedly.

## TODO
- [X] Implement the fix;
- [x] Add tests;
- [x] Add PRdoc.

Related to https://github.com/paritytech/polkadot-sdk/issues/4523

Follow-up for: https://github.com/paritytech/polkadot-sdk/pull/5188

Reopening https://github.com/paritytech/polkadot-sdk/pull/6732 as a new
PR

---------

Co-authored-by: command-bot <>

This version include a fix that make test
`zombienet-polkadot-malus-0001-dispute-valid` green again.
Thx!

Part of  #6504

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>

Link-checker job is constantly failing because of these two links.

In the browser there is a redirect, apparently our lychee checker can't
handle it.

Copies all the integration tests from Rococo to Westend.

Closes: https://github.com/paritytech/polkadot-sdk/issues/6389

Currently `pallet-collator-selection` does not expose a way to query the
assigned pot account derived from the `PotId` configuration item.
Without it, it is not possible to transfer the existential deposit to
it.

This PR addresses this issue by exposing an extra constant.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

For eth-indexer, it's probably safer to use `subscribe_finalized` and
index these blocks into the DB rather than `subscribe_best`

---------

Co-authored-by: command-bot <>

## Description

The `frame-benchmarking-cli` crate has not been buildable without the
`rocksdb` feature since version 1.17.0.

**Error:**  
```rust
self.database()?.unwrap_or(Database::RocksDb),
                             ^^^^^^^ variant or associated item not found in `Database`
```

This issue is also related to the `rocksdb` feature bleeding (#3793),
where the `rocksdb` feature was always activated even when compiling
this crate with `--no-default-features`.

**Fix:**  
- Resolved the error by choosing `paritydb` as the default database when
compiled without the `rocksdb` feature.
- Fixed the issue where the `sc-cli` crate's `rocksdb` feature was
always active, even compiling `frame-benchmarking-cli` with
`--no-default-features`.

## Review Notes

Fix the crate to be built without rocksdb, not intended to solve #3793.

---------

Co-authored-by: command-bot <>

Part of https://github.com/paritytech/polkadot-sdk/issues/6504

### Added modules

- `utility`: Traits not tied to any direct operation in the runtime.

polkadot address: 14SRqZTC1d8rfxL8W1tBTnfUBPU23ACFVPzp61FyGf4ftUFg

---------

Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>

Part of https://github.com/paritytech/polkadot-sdk/issues/6504

This PR addresses a few minor issues found while working on the
polkadot-fellows PR
[https://github.com/polkadot-fellows/runtimes/pull/522](https://github.com/polkadot-fellows/runtimes/pull/522):
- Incorrect generic type for `InboundLaneData` in
`check_message_lane_weights`.
- Renaming leftovers: `assigner_on_demand` -> `on_demand`.

Log error instead of failing with an error when block processing fails

---------

Co-authored-by: command-bot <>

Saw this test flake a few times, last time
[here](https://github.com/paritytech/polkadot-sdk/actions/runs/12834432188/job/35791830215).

We first fetch all processes in the test, then query `/proc/<pid>/stat`
for every one of them. When the file was not found, we would error. Now
we tolerate not finding this file. Ran 200 times locally without error,
before would fail a few times, probably depending on process fluctuation
(which I expect to be high on CI runners).

Use docify export for parachain template hardcoded configuration and embed it in its README #6333 (#7093)

Use docify export for parachain template hardcoded configuration and
embed it in its README #6333

Docify currently has a limitation of not being able to embed a
variable/const in its code, without embedding it's definition, even if
do something in a string like

"this is a sample string ${sample_variable}"

It will embed the entire string 
"this is a sample string ${sample_variable}"
without replacing the value of sample_variable from the code

Hence, the goal was just to make it obvious in the README where the
PARACHAIN_ID value is coming from, so a note has been added at the start
for the same, so whenever somebody is running these commands, they will
be aware about the value and replace accordingly.

To make it simpler, we added a 
rust ignore block so the user can just look it up in the readme itself
and does not have to scan through the runtime directory for the value.

---------

Co-authored-by: Iulian Barbu <14218860+iulianbarbu@users.noreply.github.com>

Follow-up to #6825, which introduced this bug.

We use the `can_build_upon` method to ask the runtime if it is fine to
build another block. The runtime checks this based on the
[`ConsensusHook`](https://github.com/paritytech/polkadot-sdk/blob/c1b7c302/cumulus/pallets/aura-ext/src/consensus_hook.rs#L110-L110)
implementation, the most popular one being the `FixedConsensusHook`.

In #6825 I removed a check that would always allow us to build when we
are building on an included block. Turns out this check is still
required when:
1. The [`UnincludedSegment`
](https://github.com/paritytech/polkadot-sdk/blob/c1b7c302

/cumulus/pallets/parachain-system/src/lib.rs#L758-L758)
storage item in pallet-parachain-system is equal or larger than the
unincluded segment.
2. We are calling the `can_build_upon` runtime API where the included
block has progressed offchain to the current parent block (so last entry
in the `UnincludedSegment` storage item).

In this scenario the last entry in `UnincludedSegment` does not have a
hash assigned yet (because it was not available in `on_finalize` of the
previous block). So the unincluded segment will be reported at its
maximum length which will forbid building another block.

Ideally we would have a more elegant solution than to rely on the
node-side here. But for now the check is reintroduced and a test is
added to not break it again by accident.

---------

Co-authored-by: command-bot <>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

Add foundation for supporting call traces in pallet_revive

Follow up:
- PR #7167 Add changes to eth-rpc to introduce debug endpoint that will
use pallet-revive tracing features
- PR #6727 Add new RPC to the client and implement tracing runtime API
that can capture traces on previous blocks

---------

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

This adds a new Proxy type to Westend Runtime called ParaRegistration.
This is related to:
https://github.com/polkadot-fellows/runtimes/pull/520.

This new proxy allows:
1. Reserve paraID
2. Register Parachain
3. Leverage Utilites pallet
4. Remove proxy.

---------

Co-authored-by: command-bot <>
Co-authored-by: Dónal Murray <donal.murray@parity.io>

Deprecation warning for old command bot

Upgrading PAPI in review-bot:
https://github.com/paritytech/review-bot/issues/140

Update to PolkaVM `0.19`. This version renumbers the opcodes in order to
be in-line with the grey paper. Hopefully, for the last time. This means
that it breaks existing contracts.

---------

Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: xermicus <cyrill@parity.io>

We already use it for lots of pallet.

Keeping it feature gated by experimental means we lose the information
of which pallet was using experimental before the migration to frame
crate usage.

We can consider `polkadot-sdk-frame` crate unstable but let's not use
the feature `experimental`.

---------

Co-authored-by: command-bot <>

Migrate staking currency from `traits::LockableCurrency` to
`traits::fungible::holds`.

Resolves part of https://github.com/paritytech/polkadot-sdk/issues/226.

## Changes
### Nomination Pool
TransferStake is now incompatible with fungible migration as old pools
were not meant to have additional ED. Since they are anyways deprecated,
removed its usage from all test runtimes.

### Staking
- Config: `Currency` becomes of type `Fungible` while `OldCurrency` is
the `LockableCurrency` used before.
- Lazy migration of accounts. Any ledger update will create a new hold
with no extra reads/writes. A permissionless extrinsic
`migrate_currency()` releases the old `lock` along with some
housekeeping.
- Staking now requires ED to be left free. It also adds no consumer to
staking accounts.
- If hold cannot be applied to all stake, the un-holdable part is force
withdrawn from the ledger.

### Delegated Staking
The pallet does not add provider for agents anymore.

## Migration stats
### Polkadot
Total accounts that can be migrated: 59564
Accounts failing to migrate: 0
Accounts with stake force withdrawn greater than ED: 59
Total force withdrawal: 29591.26 DOT

### Kusama
Total accounts that can be migrated: 26311
Accounts failing to migrate: 0
Accounts with stake force withdrawn greater than ED: 48
Total force withdrawal: 1036.05 KSM


[Full logs here](https://hackmd.io/@ak0n/BklDuFra0).

## Note about locks (freeze) vs holds
With locks or freezes, staking could use total balance of an account.
But with holds, the account needs to be left with at least Existential
Deposit in free balance. This would also affect nomination pools which
till now has been able to stake all funds contributed to it. An
alternate version of this PR is
https://github.com/paritytech/polkadot-sdk/pull/5658 where staking
pallet does not add any provider, but means pools and delegated-staking
pallet has to provide for these accounts and makes the end to end logic
(of provider and consumer ref) lot less intuitive and prone to bug.

This PR now introduces requirement for stakers to maintain ED in their
free balance. This helps with removing the bug prone incrementing and
decrementing of consumers and providers.

## TODO
- [x] Test: Vesting + governance locked funds can be staked.
- [ ] can `Call::restore_ledger` be removed? @gpestana 
- [x] Ensure unclaimed withdrawals is not affected by no provider for
pool accounts.
- [x] Investigate kusama accounts with balance between 0 and ED.
- [x] Permissionless call to release lock.
- [x] Migration of consumer (dec) and provider (inc) for direct stakers.
- [x] force unstake if hold cannot be applied to all stake.
- [x] Fix try state checks (it thinks nothing is staked for unmigrated
ledgers).
- [x] Bench `migrate_currency`.
- [x] Virtual Staker migration test.
- [x] Ensure total issuance is upto date when minting rewards.

## Followup
- https://github.com/paritytech/polkadot-sdk/issues/5742

---------

Co-authored-by: command-bot <>

✄
-----------------------------------------------------------------------------

Thank you for your Pull Request! 

 Please make sure it follows the
contribution guidelines outlined in [this

document](https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md)
and fill out the
sections below. Once you're ready to submit your PR for review, please
delete this section and leave only the text under
the "Description" heading.

# Description

Hello, I fix some typos in logs and comments. Thank you very much.


## Integration

*In depth notes about how this PR should be integrated by downstream
projects. This part is mandatory, and should be
reviewed by reviewers, if the PR does NOT have the `R0-Silent` label. In
case of a `R0-Silent`, it can be ignored.*

## Review Notes

*In depth notes about the **implementation** details of your PR. This
should be the main guide for reviewers to
understand your approach and effectively review it. If too long, use

[`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details)*.

*Imagine that someone who is depending on the old code wants to
integrate your new code and the only information that
they get is this section. It helps to include example usage and default
value here, with a `diff` code-block to show
possibly integration.*

*Include your leftover TODOs, if any, here.*

# Checklist

* [ ] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)
* [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

You can remove the "Checklist" section once all have been checked. Thank
you for your contribution!

✄
-----------------------------------------------------------------------------

Signed-off-by: chloefeal <188809157+chloefeal@users.noreply.github.com>

Zombienet substrate tests PoC (using native provider).

cc: @emamihe @alvicsam

closes https://github.com/paritytech/polkadot-sdk/issues/6911