- remove old bench from cmd.py and left alias for backward compatibility
- reverted the frame-wight-template as the problem was that it umbrella
template wasn't picked correctly in the old benchmarks, in
frame-omni-bench it correctly identifies the dependencies and uses
correct template

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Instead of using libp2p-provided external address candidates,
susceptible to address translation issues, use litep2p-backend approach
based on confirming addresses observed by multiple peers as external.

Fixes https://github.com/paritytech/polkadot-sdk/issues/7207.

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Closes #216.

This PR allows pallets to define a `view_functions` impl like so:

```rust
#[pallet::view_functions]
impl<T: Config> Pallet<T>
where
	T::AccountId: From<SomeType1> + SomeAssociation1,
{
	/// Query value no args.
	pub fn get_value() -> Option<u32> {
		SomeValue::<T>::get()
	}

	/// Query value with args.
	pub fn get_value_with_arg(key: u32) -> Option<u32> {
		SomeMap::<T>::get(key)
	}
}
```
### `QueryId`

Each view function is uniquely identified by a `QueryId`, which for this
implementation is generated by:

```twox_128(pallet_name) ++ twox_128("fn_name(fnarg_types) -> return_ty")```

The prefix `twox_128(pallet_name)` is the same as the storage prefix for pallets and take into account multiple instances of the same pallet.

The suffix is generated from the fn type signature so is guaranteed to be unique for that pallet impl. For one of the view fns in the example above it would be `twox_128("get_value_with_arg(u32) -> Option<u32>")`. It is a known limitation that only the type names themselves are taken into account: in the case of type aliases the signature may have the same underlying types but a different id; for generics the concrete types may be different but the signatures will remain the same.

The existing Runtime `Call` dispatchables are addressed by their concatenated indices `pallet_index ++ call_index`, and the dispatching is handled by the SCALE decoding of the `RuntimeCallEnum::PalletVariant(PalletCallEnum::dispatchable_variant(payload))`. For `view_functions` the runtime/pallet generated enum structure is replaced by implementing the `DispatchQuery` trait on the outer (runtime) scope, dispatching to a pallet based on the id prefix, and the inner (pallet) scope dispatching to the specific function based on the id suffix.

Future implementations could also modify/extend this scheme and routing to pallet agnostic queries.

### Executing externally

These view functions can be executed externally via the system runtime api:

```rust
pub trait ViewFunctionsApi<QueryId, Query, QueryResult, Error> where
	QueryId: codec::Codec,
	Query: codec::Codec,
	QueryResult: codec::Codec,
	Error: codec::Codec,
{
	/// Execute a view function query.
fn execute_query(query_id: QueryId, query: Query) -> Result<QueryResult,
Error>;
}
```
### `XCQ`
Currently there is work going on by @xlc to implement [`XCQ`](https://github.com/open-web3-stack/XCQ/) which may eventually supersede this work.

It may be that we still need the fixed function local query dispatching in addition to XCQ, in the same way that we have chain specific runtime dispatchables and XCM.

I have kept this in mind and the high level query API is agnostic to the underlying query dispatch and execution. I am just providing the implementation for the `view_function` definition.

### Metadata
Currently I am utilizing the `custom` section of the frame metadata, to avoid modifying the official metadata format until this is standardized.

### vs `runtime_api`
There are similarities with `runtime_apis`, some differences being:
- queries can be defined directly on pallets, so no need for boilerplate declarations and implementations
- no versioning, the `QueryId` will change if the signature changes. 
- possibility for queries to be executed from smart contracts (see below)

### Calling from contracts
Future work would be to add `weight` annotations to the view function queries, and a host function to `pallet_contracts` to allow executing these queries from contracts.

### TODO

- [x] Consistent naming (view functions pallet impl, queries, high level api?)
- [ ] End to end tests via `runtime_api`
- [ ] UI tests
- [x] Mertadata tests
- [ ] Docs

---------

Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>
Co-authored-by: Guillaume Thiolliere <guillaume.thiolliere@parity.io>

This PR changes how we call runtime API methods with more than 6
arguments: They are no longer spilled to the stack but packed into
registers instead. Pointers are 32 bit wide so we can pack two of them
into a single 64 bit register. Since we mostly pass pointers, this
technique effectively increases the number of arguments we can pass
using the available registers.

To make this work for `instantiate` too we now pass the code hash and
the call data in the same buffer, akin to how the `create` family
opcodes work in the EVM. The code hash is fixed in size, implying the
start of the constructor call data.

---------

Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Co-authored-by: command-bot <>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

closes #5978

---------

Co-authored-by: command-bot <>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

Revert https://github.com/paritytech/polkadot-sdk/pull/7011 and replace
it with a backward-compatible solution suitable for backporting to a
release branch.

### Review notes
It's easier to review this PR per commit: the first commit is just a
revert, so it's enough to review only the second one, which is almost a
one-liner.

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Part of: https://github.com/paritytech/polkadot-sdk/issues/6906

Improve debugging by using `#[track_caller]` in system `assert_last_event` and `assert_has_event` (#7142)

Without track caller the error message of the assert points to the
`assert_last_event` function, which is not useful.
```
thread 'tests::set_metadata_works' panicked at /home/gui/Developpement/polkadot-sdk/substrate/frame/system/src/lib.rs:2034:9:
assertion `left == right` failed: expected event RuntimeEvent::Referenda(Event::MetadataSet { index: 0, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa }) is not equal to the last event RuntimeEvent::Referenda(Event::MetadataSet { index: 1, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
  left: RuntimeEvent::Referenda(Event::MetadataSet { index: 1, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
 right: RuntimeEvent::Referenda(Event::MetadataSet { index: 0, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
```

With the track caller the error message points to the caller, showing
the source of the error:
```
thread 'tests::set_metadata_works' panicked at substrate/frame/referenda/src/tests.rs:639:9:
assertion `left == right` failed: expected event RuntimeEvent::Referenda(Event::MetadataSet { index: 0, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa }) is not equal to the last event RuntimeEvent::Referenda(Event::MetadataSet { index: 1, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
  left: RuntimeEvent::Referenda(Event::MetadataSet { index: 1, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
 right: RuntimeEvent::Referenda(Event::MetadataSet { index: 0, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
```

I also improved the error message to include a warning when checking
events on block number zero.

The link to Polkachu is not working

This regression was introduced with some of the recent PRs. Regression
fixed and test added.

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

- Add option to specify database_url using DATABASE_URL environment
variable
- Add a eth-rpc-tester rust bin that can be used to test deployment
before releasing eth-rpc
- make evm_block non fallible so that it can return an Ok response for
older blocks when the runtime API is not available
- update cargo.lock to integrate changes from
https://github.com/paritytech/subxt/pull/1904

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Closes #6196 
Closes #7204

Example of PR: https://github.com/paritytech/polkadot-sdk/pull/6816

Every sunday 01:00 AM it's going to start to benchmark (with /cmd bench)
all runtimes and all pallets
Then diff total will be pushed to a branch and PR open,. I assume
review-bot is going assign required reviewers per changed files

I afraid each weeks will be too much to review & merge, but we can
adjust later

Bonus: fix for pallet_multisig lib and
substrate/.maintain/frame-weight-template.hbs , which didn't let to
compile new weights

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

# Description

Closes https://github.com/paritytech/polkadot-sdk/issues/7265.

## Integration

Requires changes in
`https://github.com/polkadot-js/api/packages/{rpc-augment,types-support,types}`
to be visible in Polkadot\Substrate Portal and in other libraries where
we should explicitly state RPC methods.

Accompany PR to `polkadot-js/api`:
https://github.com/polkadot-js/api/pull/6070.

## Review Notes

Please put the right label on my PR.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

# Derived Dev Accounts

Resolves https://github.com/paritytech/polkadot-sdk/issues/6040

## Description
This update introduces support for creating an arbitrary number of
developer accounts at the genesis block based on a specified derivation
path. This functionality is gated by the runtime-benchmarks feature,
ensuring it is only enabled during benchmarking scenarios.

### Key Features
- Arbitrary Dev Accounts at Genesis: Developers can now specify any
number of accounts to be generated at genesis using a hard derivation
path.

- Default Derivation Path: If no derivation path is provided (i.e., when
`Option<dev_accounts: (..., None)>` is set to `Some` at genesis), the
system will default to the path `//Sender//{}`.

- No Impact on Total Token Issuance: Developer accounts are excluded
from the total issuance of the token supply at genesis, ensuring they do
not affect the overall balance or token distribution.

polkadot address: 14SRqZTC1d8rfxL8W1tBTnfUBPU23ACFVPzp61FyGf4ftUFg

---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

- Fix the EVM fee cost estimation.
The estimation shown in EVM wallet was using Native instead of EVM
decimals
- Remove the precise code length estimation in dry run call.
Over-estimating is fine, since extra gas is refunded anyway.
- Ensure that the estimated fee calculated from gas_price x gas use the
encoded weight & deposit limit instead of the exact one calculated by
the dry-run. Else we can end up with a fee that is lower than the actual
fee paid by the user

---------

Co-authored-by: command-bot <>

Part of https://github.com/paritytech/polkadot-sdk/issues/6504

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

This PR enforces that outbound requests are finished within the
specified protocol timeout.

The stable2412 version running libp2p 0.52.4 contains a bug which does
not track request timeouts properly:
- https://github.com/libp2p/rust-libp2p/pull/5429

The issue has been detected while submitting libp2p -> litep2p requests
in kusama. This aims to check that pending outbound requests have not
timedout. Although the issue has been fixed in libp2p, there might be
other cases where this may happen. For example:
- https://github.com/libp2p/rust-libp2p/pull/5417

For more context see:
https://github.com/paritytech/polkadot-sdk/issues/7076#issuecomment-2596085096


1. Ideally, the force-timeout mechanism in this PR should never be
triggered in production. However, origin/stable2412 occasionally
encounters this issue. When this happens, 2 warnings may be generated:
- one warning introduced by this PR wrt force timeout terminating the
request
- possible one warning when the libp2p decides (if at all) to provide
the response back to substrate (as mentioned by @alexggh
[here](https://github.com/paritytech/polkadot-sdk/pull/7222/files#diff-052aeaf79fef3d9a18c2cfd67006aa306b8d52e848509d9077a6a0f2eb856af7L769)
and
[here](https://github.com/paritytech/polkadot-sdk/pull/7222/files#diff-052aeaf79fef3d9a18c2cfd67006aa306b8d52e848509d9077a6a0f2eb856af7L842)

2. This implementation does not propagate to the substrate service the
`RequestFinished { error: .. }`. That event is only used internally by
substrate to increment metrics. However, we don't have the peer
information available to propagate the event properly when we
force-timeout the request. Considering this should most likely not
happen in production (origin/master) and that we'll be able to extract
information by warnings, I would say this is a good tradeoff for code
simplicity:


https://github.com/paritytech/polkadot-sdk/blob/06e3b5c6

/substrate/client/network/src/service.rs#L1543


### Testing

Added a new test to ensure the timeout is reached properly, even if
libp2p does not produce a response in due time.

I've also transitioned the tests to using `tokio::test` due to a
limitation of
[CI](https://github.com/paritytech/polkadot-sdk/actions/runs/12832055737/job/35784043867)

```
--- TRY 1 STDERR:        sc-network request_responses::tests::max_response_size_exceeded ---
thread 'request_responses::tests::max_response_size_exceeded' panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/time/interval.rs:139:26:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
```



cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

Related to https://github.com/paritytech/polkadot-sdk/issues/4523

Follow-up for: https://github.com/paritytech/polkadot-sdk/pull/5188

Reopening https://github.com/paritytech/polkadot-sdk/pull/6732 as a new
PR

---------

Co-authored-by: command-bot <>

Part of  #6504

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>

Link-checker job is constantly failing because of these two links.

In the browser there is a redirect, apparently our lychee checker can't
handle it.

For eth-indexer, it's probably safer to use `subscribe_finalized` and
index these blocks into the DB rather than `subscribe_best`

---------

Co-authored-by: command-bot <>

## Description

The `frame-benchmarking-cli` crate has not been buildable without the
`rocksdb` feature since version 1.17.0.

**Error:**  
```rust
self.database()?.unwrap_or(Database::RocksDb),
                             ^^^^^^^ variant or associated item not found in `Database`
```

This issue is also related to the `rocksdb` feature bleeding (#3793),
where the `rocksdb` feature was always activated even when compiling
this crate with `--no-default-features`.

**Fix:**  
- Resolved the error by choosing `paritydb` as the default database when
compiled without the `rocksdb` feature.
- Fixed the issue where the `sc-cli` crate's `rocksdb` feature was
always active, even compiling `frame-benchmarking-cli` with
`--no-default-features`.

## Review Notes

Fix the crate to be built without rocksdb, not intended to solve #3793.

---------

Co-authored-by: command-bot <>

Part of https://github.com/paritytech/polkadot-sdk/issues/6504

### Added modules

- `utility`: Traits not tied to any direct operation in the runtime.

polkadot address: 14SRqZTC1d8rfxL8W1tBTnfUBPU23ACFVPzp61FyGf4ftUFg

---------

Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>

Part of https://github.com/paritytech/polkadot-sdk/issues/6504

Log error instead of failing with an error when block processing fails

---------

Co-authored-by: command-bot <>

Add foundation for supporting call traces in pallet_revive

Follow up:
- PR #7167 Add changes to eth-rpc to introduce debug endpoint that will
use pallet-revive tracing features
- PR #6727 Add new RPC to the client and implement tracing runtime API
that can capture traces on previous blocks

---------

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Update to PolkaVM `0.19`. This version renumbers the opcodes in order to
be in-line with the grey paper. Hopefully, for the last time. This means
that it breaks existing contracts.

---------

Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: xermicus <cyrill@parity.io>

We already use it for lots of pallet.

Keeping it feature gated by experimental means we lose the information
of which pallet was using experimental before the migration to frame
crate usage.

We can consider `polkadot-sdk-frame` crate unstable but let's not use
the feature `experimental`.

---------

Co-authored-by: command-bot <>

Migrate staking currency from `traits::LockableCurrency` to
`traits::fungible::holds`.

Resolves part of https://github.com/paritytech/polkadot-sdk/issues/226.

## Changes
### Nomination Pool
TransferStake is now incompatible with fungible migration as old pools
were not meant to have additional ED. Since they are anyways deprecated,
removed its usage from all test runtimes.

### Staking
- Config: `Currency` becomes of type `Fungible` while `OldCurrency` is
the `LockableCurrency` used before.
- Lazy migration of accounts. Any ledger update will create a new hold
with no extra reads/writes. A permissionless extrinsic
`migrate_currency()` releases the old `lock` along with some
housekeeping.
- Staking now requires ED to be left free. It also adds no consumer to
staking accounts.
- If hold cannot be applied to all stake, the un-holdable part is force
withdrawn from the ledger.

### Delegated Staking
The pallet does not add provider for agents anymore.

## Migration stats
### Polkadot
Total accounts that can be migrated: 59564
Accounts failing to migrate: 0
Accounts with stake force withdrawn greater than ED: 59
Total force withdrawal: 29591.26 DOT

### Kusama
Total accounts that can be migrated: 26311
Accounts failing to migrate: 0
Accounts with stake force withdrawn greater than ED: 48
Total force withdrawal: 1036.05 KSM


[Full logs here](https://hackmd.io/@ak0n/BklDuFra0).

## Note about locks (freeze) vs holds
With locks or freezes, staking could use total balance of an account.
But with holds, the account needs to be left with at least Existential
Deposit in free balance. This would also affect nomination pools which
till now has been able to stake all funds contributed to it. An
alternate version of this PR is
https://github.com/paritytech/polkadot-sdk/pull/5658 where staking
pallet does not add any provider, but means pools and delegated-staking
pallet has to provide for these accounts and makes the end to end logic
(of provider and consumer ref) lot less intuitive and prone to bug.

This PR now introduces requirement for stakers to maintain ED in their
free balance. This helps with removing the bug prone incrementing and
decrementing of consumers and providers.

## TODO
- [x] Test: Vesting + governance locked funds can be staked.
- [ ] can `Call::restore_ledger` be removed? @gpestana 
- [x] Ensure unclaimed withdrawals is not affected by no provider for
pool accounts.
- [x] Investigate kusama accounts with balance between 0 and ED.
- [x] Permissionless call to release lock.
- [x] Migration of consumer (dec) and provider (inc) for direct stakers.
- [x] force unstake if hold cannot be applied to all stake.
- [x] Fix try state checks (it thinks nothing is staked for unmigrated
ledgers).
- [x] Bench `migrate_currency`.
- [x] Virtual Staker migration test.
- [x] Ensure total issuance is upto date when minting rewards.

## Followup
- https://github.com/paritytech/polkadot-sdk/issues/5742

---------

Co-authored-by: command-bot <>

closes https://github.com/paritytech/polkadot-sdk/issues/6911

Closes #3149 

## Description

This PR introduces `pallet-asset-rewards`, which allows accounts to be
rewarded for freezing `fungible` tokens. The motivation for creating
this pallet is to allow incentivising LPs.

See the pallet docs for more info about the pallet.

## Runtime changes

The pallet has been added to
- `asset-hub-rococo`
- `asset-hub-westend`

The `NativeAndAssets` `fungibles` Union did not contain `PoolAssets`, so
it has been renamed `NativeAndNonPoolAssets`

A new `fungibles` Union `NativeAndAllAssets` was created to encompass
all assets and the native token.

## TODO
- [x] Emulation tests
- [x] Fill in Freeze logic (blocked
https://github.com/paritytech/polkadot-sdk/issues/3342) and re-run
benchmarks

---------

Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: muharem <ismailov.m.h@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Closes https://github.com/paritytech/polkadot-sdk/issues/6891.

cc @athei @xermicus @pgherveou

Remove all pallet::events except for the `ContractEmitted` event that is
emitted by contracts

---------

Co-authored-by: command-bot <>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Remove the `debug_buffer` feature

---------

Co-authored-by: command-bot <>
Co-authored-by: Cyrill Leutwiler <cyrill@parity.io>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

This PR provides the partial results of the `GetRecord` kademlia query.

This significantly improves the authority discovery records, from ~37
minutes to ~2/3 minutes.
In contrast, libp2p discovers authority records in around ~10 minutes. 

The authority discovery was slow because litep2p provided the records
only after the Kademlia query was completed. A normal Kademlia query
completes in around 40 seconds to a few minutes.
In this PR, partial records are provided as soon as they are discovered
from the network.

### Testing Done

Started a node in Kusama with `--validator` and litep2p backend.
The node discovered 996/1000 authority records in ~ 1 minute 45 seconds.

![Screenshot 2025-01-09 at 12 26
08](https://github.com/user-attachments/assets/b618bf7c-2bba-43a0-a021-4047e854c075)


### Before & After

In this image, on the left side is libp2p, in the middle litep2p without
this PR, on the right litep2p with this PR

![Screenshot 2025-01-07 at 17 57
56](https://github.com/user-attachments/assets/a8d467f7-8dc7-461c-bcff-163b94d01ae8)



Closes: https://github.com/paritytech/polkadot-sdk/issues/7077

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Bump asset-hub westend spec version

---------

Co-authored-by: command-bot <>

Closes #6846 .

---------

Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: xermicus <cyrill@parity.io>

Closes https://github.com/paritytech/polkadot-sdk/issues/6767.

The return type of the host function `caller_is_root` was denoted as
`u32` in `pallet_revive_uapi`. This PR fixes the return type to `bool`.

As a drive-by, the PR re-exports `pallet_revive::exec::Origin` to extend
what can be tested externally.

---------

Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com>