We already use it for lots of pallet.

Keeping it feature gated by experimental means we lose the information
of which pallet was using experimental before the migration to frame
crate usage.

We can consider `polkadot-sdk-frame` crate unstable but let's not use
the feature `experimental`.

---------

Co-authored-by: command-bot <>

Migrate staking currency from `traits::LockableCurrency` to
`traits::fungible::holds`.

Resolves part of https://github.com/paritytech/polkadot-sdk/issues/226.

## Changes
### Nomination Pool
TransferStake is now incompatible with fungible migration as old pools
were not meant to have additional ED. Since they are anyways deprecated,
removed its usage from all test runtimes.

### Staking
- Config: `Currency` becomes of type `Fungible` while `OldCurrency` is
the `LockableCurrency` used before.
- Lazy migration of accounts. Any ledger update will create a new hold
with no extra reads/writes. A permissionless extrinsic
`migrate_currency()` releases the old `lock` along with some
housekeeping.
- Staking now requires ED to be left free. It also adds no consumer to
staking accounts.
- If hold cannot be applied to all stake, the un-holdable part is force
withdrawn from the ledger.

### Delegated Staking
The pallet does not add provider for agents anymore.

## Migration stats
### Polkadot
Total accounts that can be migrated: 59564
Accounts failing to migrate: 0
Accounts with stake force withdrawn greater than ED: 59
Total force withdrawal: 29591.26 DOT

### Kusama
Total accounts that can be migrated: 26311
Accounts failing to migrate: 0
Accounts with stake force withdrawn greater than ED: 48
Total force withdrawal: 1036.05 KSM


[Full logs here](https://hackmd.io/@ak0n/BklDuFra0).

## Note about locks (freeze) vs holds
With locks or freezes, staking could use total balance of an account.
But with holds, the account needs to be left with at least Existential
Deposit in free balance. This would also affect nomination pools which
till now has been able to stake all funds contributed to it. An
alternate version of this PR is
https://github.com/paritytech/polkadot-sdk/pull/5658 where staking
pallet does not add any provider, but means pools and delegated-staking
pallet has to provide for these accounts and makes the end to end logic
(of provider and consumer ref) lot less intuitive and prone to bug.

This PR now introduces requirement for stakers to maintain ED in their
free balance. This helps with removing the bug prone incrementing and
decrementing of consumers and providers.

## TODO
- [x] Test: Vesting + governance locked funds can be staked.
- [ ] can `Call::restore_ledger` be removed? @gpestana 
- [x] Ensure unclaimed withdrawals is not affected by no provider for
pool accounts.
- [x] Investigate kusama accounts with balance between 0 and ED.
- [x] Permissionless call to release lock.
- [x] Migration of consumer (dec) and provider (inc) for direct stakers.
- [x] force unstake if hold cannot be applied to all stake.
- [x] Fix try state checks (it thinks nothing is staked for unmigrated
ledgers).
- [x] Bench `migrate_currency`.
- [x] Virtual Staker migration test.
- [x] Ensure total issuance is upto date when minting rewards.

## Followup
- https://github.com/paritytech/polkadot-sdk/issues/5742

---------

Co-authored-by: command-bot <>

closes https://github.com/paritytech/polkadot-sdk/issues/6911

Closes #3149 

## Description

This PR introduces `pallet-asset-rewards`, which allows accounts to be
rewarded for freezing `fungible` tokens. The motivation for creating
this pallet is to allow incentivising LPs.

See the pallet docs for more info about the pallet.

## Runtime changes

The pallet has been added to
- `asset-hub-rococo`
- `asset-hub-westend`

The `NativeAndAssets` `fungibles` Union did not contain `PoolAssets`, so
it has been renamed `NativeAndNonPoolAssets`

A new `fungibles` Union `NativeAndAllAssets` was created to encompass
all assets and the native token.

## TODO
- [x] Emulation tests
- [x] Fill in Freeze logic (blocked
https://github.com/paritytech/polkadot-sdk/issues/3342) and re-run
benchmarks

---------

Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: muharem <ismailov.m.h@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Closes https://github.com/paritytech/polkadot-sdk/issues/6891.

cc @athei @xermicus @pgherveou

Remove all pallet::events except for the `ContractEmitted` event that is
emitted by contracts

---------

Co-authored-by: command-bot <>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Remove the `debug_buffer` feature

---------

Co-authored-by: command-bot <>
Co-authored-by: Cyrill Leutwiler <cyrill@parity.io>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

This PR provides the partial results of the `GetRecord` kademlia query.

This significantly improves the authority discovery records, from ~37
minutes to ~2/3 minutes.
In contrast, libp2p discovers authority records in around ~10 minutes. 

The authority discovery was slow because litep2p provided the records
only after the Kademlia query was completed. A normal Kademlia query
completes in around 40 seconds to a few minutes.
In this PR, partial records are provided as soon as they are discovered
from the network.

### Testing Done

Started a node in Kusama with `--validator` and litep2p backend.
The node discovered 996/1000 authority records in ~ 1 minute 45 seconds.

![Screenshot 2025-01-09 at 12 26
08](https://github.com/user-attachments/assets/b618bf7c-2bba-43a0-a021-4047e854c075)


### Before & After

In this image, on the left side is libp2p, in the middle litep2p without
this PR, on the right litep2p with this PR

![Screenshot 2025-01-07 at 17 57
56](https://github.com/user-attachments/assets/a8d467f7-8dc7-461c-bcff-163b94d01ae8)



Closes: https://github.com/paritytech/polkadot-sdk/issues/7077

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Bump asset-hub westend spec version

---------

Co-authored-by: command-bot <>

Closes #6846 .

---------

Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: xermicus <cyrill@parity.io>

Closes https://github.com/paritytech/polkadot-sdk/issues/6767.

The return type of the host function `caller_is_root` was denoted as
`u32` in `pallet_revive_uapi`. This PR fixes the return type to `bool`.

As a drive-by, the PR re-exports `pallet_revive::exec::Origin` to extend
what can be tested externally.

---------

Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

This PR rejects inbound requests from banned peers (reputation is below
the banned threshold).

This mirrors the request-response implementation from the libp2p side.
I won't expect this to get triggered too often, but we'll monitor this
metric.

While at it, have registered a new inbound failure metric to have
visibility into this.

Discovered during the investigation of:
https://github.com/paritytech/polkadot-sdk/issues/7076#issuecomment-2589613046

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Introduce `frame_system::Pallet::run_to_block`,
`frame_system::Pallet::run_to_block_with`, and
`frame_system::RunToBlockHooks` to establish a generic `run_to_block`
mechanism for mock tests, minimizing redundant implementations across
various pallets.

Closes #299.

---

Polkadot address: 156HGo9setPcU2qhFMVWLkcmtCEGySLwNqa3DaEiYSWtte4Y

---------

Signed-off-by: Xavier Lau <x@acg.box>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Co-authored-by: Guillaume Thiolliere <guillaume.thiolliere@parity.io>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Fix error message in `DispatchInfo` where post-dispatch and pre-dispatch
weight was reversed.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

Add an option to persist EVM transaction hash to a SQL db.
This should make it possible to run a full archive ETH RPC node
(assuming the substrate node is also a full archive node)

Some queries such as eth_getTransactionByHash,
eth_getBlockTransactionCountByHash, and other need to work with a
transaction hash indexes, which are not stored in Substrate and need to
be stored by the eth-rpc proxy.

The refactoring break down the Client into a `BlockInfoProvider` and
`ReceiptProvider`
- BlockInfoProvider does not need any persistence data, as we can fetch
all block info from the source substrate chain
- ReceiptProvider comes in two flavor, 
  - An in memory cache implementation - This is the one we had so far.
- A DB implementation - This one persist rows with the block_hash, the
transaction_index and the transaction_hash, so that we can later fetch
the block and extrinsic for that receipt and reconstruct the ReceiptInfo
object.

This PR also adds ...

This PR adds the `(litep2p)` suffix to the agent version (user agent) of
the identify protocol.

The change is needed to gain visibility into network backends and
determine exactly the number of validators that are running litep2p.
Using tools like subp2p-explorer, we can determine if the validators are
running litep2p nodes.

This reflects on the identify protocol:

```
info=Identify {
  protocol_version: Some("/substrate/1.0"),
  agent_version: Some("polkadot-parachain/v1.17.0-967989c5

 (kusama-node-name-01) (litep2p)")
  ...
}
```

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Higher-priority transactions can now replace lower-priority transactions
even when the internal _tx_mem_pool_ is full.

**Notes for reviewers:**
- The _tx_mem_pool_ now maintains information about transaction
priority. Although _tx_mem_pool_ itself is stateless, transaction
priority is updated after submission to the view. An alternative
approach could involve validating transactions at the `at` block, but
this is computationally expensive. To avoid additional validation
overhead, I opted to use the priority obtained from runtime during
submission to the view. This is the rationale behind introducing the
`SubmitOutcome` struct, which synchronously communicates transaction
priority from the view to the pool. This results in a very brief window
during which the transaction priority remains unknown - those
transaction are not taken into consideration while dropping takes place.
In the future, if needed, we could update transaction priorit...

As per #3326, removes pallet::getter macro usage from pallet-grandpa.
The syntax `StorageItem::<T, I>::get()` should be used instead.

cc @muraca

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

# Description

This PR modifies the hard-coded size of extrinsics cache within
[`PoolRotator`](https://github.com/paritytech/polkadot-sdk/blob/cdf107de/substrate/client/transaction-pool/src/graph/rotator.rs#L36-L45)
to be inline with pool limits.

The problem was, that due to small size (comparing to number of txs in
single block) of hard coded size:

https://github.com/paritytech/polkadot-sdk/blob/cdf107de/substrate/client/transaction-pool/src/graph/rotator.rs#L34
excessive number of unnecessary verification were performed in
`prune_tags`:

https://github.com/paritytech/polkadot-sdk/blob/cdf107de

/substrate/client/transaction-pool/src/graph/pool.rs#L369-L370

This was resulting in quite long durations of `prune_tags` execution
time (which was ok for 6s, but becomes noticable for 2s blocks):
```
Pruning at HashAndNumber { number: 83, ... }. Resubmitting transactions: 6142, reverification took: 237.818955ms    
Pruning at HashAndNumber { number: 84, ... }. Resubmitting transactions: 5985, reverification took: 222.118218ms    
Pruning at HashAndNumber { number: 85, ... }. Resubmitting transactions: 5981, reverification took: 215.546847ms
```

The fix reduces the overhead:
```
Pruning at HashAndNumber { number: 92, ... }. Resubmitting transactions: 6325, reverification took: 14.728354ms    
Pruning at HashAndNumber { number: 93, ... }. Resubmitting transactions: 7030, reverification took: 23.973607ms    
Pruning at HashAndNumber { number: 94, ... }. Resubmitting transactions: 4465, reverification took: 9.532472ms    
```

## Review Notes
I decided to leave the hardocded `EXPECTED_SIZE` for the legacy
transaction pool. Removing verification of transactions during
re-submission may negatively impact the behavior of the legacy
(single-state) pool. As in long-term we probably want to deprecate old
pool, I did not invest time to assess the impact of rotator change in
behavior of the legacy pool.

---------

Co-authored-by: command-bot <>
Co-authored-by: Iulian Barbu <14218860+iulianbarbu@users.noreply.github.com>

Update the current approach to attach the `ref_time`, `pov` and
`deposit` parameters to an Ethereum transaction.
Previously we will pass these 3 parameters along with the signed
payload, and check that the fees resulting from `gas x gas_price` match
the actual fees paid by the user for the extrinsic.

This approach unfortunately can be attacked. A malicious actor could
force such a transaction to fail by injecting low values for some of
these extra parameters as they are not part of the signed payload.

The new approach encodes these 3 extra parameters in the lower digits of
the transaction gas, approximating the the log2 of the actual values to
encode each components on 2 digits

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: command-bot <>

Closes: https://github.com/paritytech/polkadot-sdk/issues/7033

# Description

- Used 10 notifications and requests within the benchmarks. After moving
the network workers' initialization out of the benchmarks, it is
acceptable to use this small number without losing precision.
- Removed the 128MB payload that consumed most of the execution time.

# Description

Migrating salary pallet to use umbrella crate. It is a follow-up from
https://github.com/paritytech/polkadot-sdk/pull/7025
Why did I create this new branch? 
I did this, so that the unnecessary cargo fmt changes from the previous
branch are discarded and hence opened this new PR.



## Review Notes

This PR migrates pallet-salary to use the umbrella crate.

Added change: Explanation requested for why `TestExternalities` was
replaced by `TestState` as testing_prelude already includes it
`pub use sp_io::TestExternalities as TestState;`


I have also modified the defensive! macro to be compatible with umbrella
crate as it was being used in the salary pallet

Co-authored-by: Dónal Murray <donalm@seadanda.dev>

Co-authored-by: Dónal Murray <donalm@seadanda.dev>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

PR for #7054 

Replaced frame_support with $crate from @gui1117 's suggestion to fix
the dependency issue

---------

Co-authored-by: command-bot <>

# Description

Implements NetworkRequest::request for litep2p that we need for
networking benchmarks


## Review Notes

Duplicates implementation for NetworkService

https://github.com/paritytech/polkadot-sdk/blob/5bf9dd2a/substrate/client/network/src/service.rs#L1186-L1205

---------

Co-authored-by: command-bot <>

# Description

Migrate pallet-node-authorization to use umbrella crate. Part of #6504 

## Review Notes

* This PR migrates pallet-node-authorization to use the umbrella crate.
* Some imports like below have not been added to any prelude as they
have very limited usage across the various pallets.
```rust
use sp_core::OpaquePeerId as PeerId;
```
* Added a commonly used runtime trait for testing in the
`testing_prelude` in `substrate/frame/src/lib.rs`:
```rust
pub use sp_runtime::traits::BadOrigin;
```
* `weights.rs` uses the `weights_prelude` like:
```rust
use frame::weights_prelude::*;
```
* `tests.rs` and `mock.rs` use the `testing_prelude`:
```rust
use frame::testing_prelude::*;
```
* `lib.rs` uses the main `prelude` like:
```rust
use frame::prelude::*;
```
* For testing: Checked that local build works and tests run
successfully.

# Description

This PR removes usage of deprecated `sp-std` from Substrate. (following
PR of #5010)

## Integration

This PR doesn't remove re-exported `sp_std` from any crates yet, so
downstream projects using re-exported `sp_std` will not be affected.

## Review Notes

The existing code using `sp-std` is refactored to use `alloc` and `core`
directly. The key-value maps are instantiated from a vector of tuples
directly instead of using `sp_std::map!` macro.

`sp_std::Writer` is a helper type to use `Vec<u8>` with
`core::fmt::Write` trait. This PR copied it into `sp-runtime`, because
all crates using `sp_std::Writer` (including `sp-runtime` itself,
`frame-support`, etc.) depend on `sp-runtime`.

If this PR is merged, I would write following PRs to remove remaining
usage of `sp-std` from `bridges` and `cumulus`.

---------

Co-authored-by: command-bot <>
Co-authored-by: Guillaume Thiolliere <guillaume.thiolliere@parity.io>
Co-authored-by: Bastian Köcher <inf...

Co-authored-by: Dónal Murray <donal.murray@parity.io>

Changes:
- Add call `import_member` to the core-fellowship pallet.
- Move common logic between `import` and `import_member` into
`do_import`.

## `import_member`

Can be used to induct an arbitrary collective member and is callable by
any signed origin. Pays no fees upon success.
This is useful in the case that members did not induct themselves and
are idling on their rank.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>

Co-authored-by: Dónal Murray <donal.murray@parity.io>

Implement cumulus StorageWeightReclaim as wrapping transaction extension + frame system ReclaimWeight (#6140)

(rebasing of https://github.com/paritytech/polkadot-sdk/pull/5234)

## Issues:

* Transaction extensions have weights and refund weight. So the
reclaiming of unused weight must happen last in the transaction
extension pipeline. Currently it is inside `CheckWeight`.
* cumulus storage weight reclaim transaction extension misses the proof
size of logic happening prior to itself.

## Done:

* a new storage `ExtrinsicWeightReclaimed` in frame-system. Any logic
which attempts to do some reclaim must use this storage to avoid double
reclaim.
* a new function `reclaim_weight` in frame-system pallet: info and post
info in arguments, read the already reclaimed weight, calculate the new
unused weight from info and post info. do the more accurate reclaim if
higher.
* `CheckWeight` is unchanged and still reclaim the weight in post
dispatch
* `ReclaimWeight` is a new transaction extension in frame system. For
solo chains it must be used last in the transactino extension pipeline.
It does the final most accurate reclaim
* `StorageWeightReclaim` is moved from cumulus primitives into its own
pallet (in order to define benchmark) and is changed into a wrapping
transaction extension.
It does the recording of proof size and does the reclaim using this
recording and the info and post info. So parachains don't need to use
`ReclaimWeight`. But also if they use it, there is no bug.

    ```rust
  /// The TransactionExtension to the basic transaction logic.
pub type TxExtension =
cumulus_pallet_weight_reclaim::StorageWeightReclaim<
         Runtime,
         (
                 frame_system::CheckNonZeroSender<Runtime>,
                 frame_system::CheckSpecVersion<Runtime>,
                 frame_system::CheckTxVersion<Runtime>,
                 frame_system::CheckGenesis<Runtime>,
                 frame_system::CheckEra<Runtime>,
                 frame_system::CheckNonce<Runtime>,
                 frame_system::CheckWeight<Runtime>,
pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
                 BridgeRejectObsoleteHeadersAndMessages,

(bridge_to_rococo_config::OnBridgeHubWestendRefundBridgeHubRococoMessages,),
frame_metadata_hash_extension::CheckMetadataHash<Runtime>,
         ),
  >;
  ```

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: command-bot <>

Currently `(A, B, C)` and `((A, B), C)` change the order of implications
in the transaction extension pipeline. This order is not accessible in
the metadata, because the metadata is just a vector of transaction
extension, the nested structure is not visible.

This PR make the implementation for tuple of `TransactionExtension`
better for tuple of tuple. `(A, B, C)` and `((A, B), C)` don't change
the implication for the validation A.

This is a breaking change but only when using the trait
`TransactionExtension` the code implementing the trait is not breaking
(surprising rust behavior but fine).

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

# Description

Migrate pallet-mixnet to use umbrella crate whilst adding a few types
and traits in the frame prelude that are used by other pallets as well.

## Review Notes

* This PR migrates `pallet-mixnet` to use the umbrella crate. 
* Note that some imports like `use
sp_application_crypto::RuntimeAppPublic;` and imports from
`sp_mixnet::types::` have not been migrated to the umbrella crate as
they are not used in any / many other places and are relevant only to
the `pallet-mixnet`.
* Transaction related helpers to submit transactions from `frame-system`
have been added to the main `prelude` as they have usage across various
pallets.
```Rust
	pub use frame_system::offchain::*;
```
* Exporting `arithmetic` module in the main `prelude` since this is used
a lot throughout various pallets.
* Nightly formatting has been applied using `cargo fmt`
* Benchmarking dependencies have been removed from`palet-mixnet` as
there is no benchmarking.rs present for `pallet-mixnet`. For the same
reason, `"pallet-mixnet?/runtime-benchmarks"` has been removed from
`umbrella/Cargo.toml`.

---------

Co-authored-by: Dónal Murray <donalm@seadanda.dev>

Changes:
- Automatically detect if current env is tty. If not disable the spinner
logging.
- Add `Result` type.
- Format log style.

Originally reported from:
-
https://github.com/hack-ink/polkadot-runtime-releaser/blob/4811d2b419649a73edd5bd1f748a858b846eb139/action/try-runtime/action.yml#L75-L91
-
https://github.com/hack-ink/polkadot-runtime-releaser-workshop/pull/3#issuecomment-2563883943

Closes #7010.

---

Polkadot address: 156HGo9setPcU2qhFMVWLkcmtCEGySLwNqa3DaEiYSWtte4Y

---------

Signed-off-by: Xavier Lau <x@acg.box>
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

# Description

This PR removes the outdated warning message from the
`frame-omni-bencher` CLI that states the tool is "not yet battle
tested". Fixes #7019

## Integration

No integration steps are required.

## Review Notes

The functionality of the tool remains unchanged. Removes the warning
message from the CLI output.

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>

fix https://github.com/paritytech/polkadot-sdk/issues/6970

---------

Co-authored-by: command-bot <>

Introduce `SyncEvent::InitialPeers` message sent to new subscribers to
allow them correctly tracking sync peers. This resolves a race condition
described in
https://github.com/paritytech/polkadot-sdk/issues/6573#issuecomment-2563091343.

Fixes https://github.com/paritytech/polkadot-sdk/issues/6573.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>