- remove old bench from cmd.py and left alias for backward compatibility
- reverted the frame-wight-template as the problem was that it umbrella
template wasn't picked correctly in the old benchmarks, in
frame-omni-bench it correctly identifies the dependencies and uses
correct template

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Closes #216.

This PR allows pallets to define a `view_functions` impl like so:

```rust
#[pallet::view_functions]
impl<T: Config> Pallet<T>
where
	T::AccountId: From<SomeType1> + SomeAssociation1,
{
	/// Query value no args.
	pub fn get_value() -> Option<u32> {
		SomeValue::<T>::get()
	}

	/// Query value with args.
	pub fn get_value_with_arg(key: u32) -> Option<u32> {
		SomeMap::<T>::get(key)
	}
}
```
### `QueryId`

Each view function is uniquely identified by a `QueryId`, which for this
implementation is generated by:

```twox_128(pallet_name) ++ twox_128("fn_name(fnarg_types) -> return_ty")```

The prefix `twox_128(pallet_name)` is the same as the storage prefix for pallets and take into account multiple instances of the same pallet.

The suffix is generated from the fn type signature so is guaranteed to be unique for that pallet impl. For one of the view fns in the example above it would be `twox_128("get_value_with_arg(u32) -> Option<u32>")`. It is a known limitation that only the type names themselves are taken into account: in the case of type aliases the signature may have the same underlying types but a different id; for generics the concrete types may be different but the signatures will remain the same.

The existing Runtime `Call` dispatchables are addressed by their concatenated indices `pallet_index ++ call_index`, and the dispatching is handled by the SCALE decoding of the `RuntimeCallEnum::PalletVariant(PalletCallEnum::dispatchable_variant(payload))`. For `view_functions` the runtime/pallet generated enum structure is replaced by implementing the `DispatchQuery` trait on the outer (runtime) scope, dispatching to a pallet based on the id prefix, and the inner (pallet) scope dispatching to the specific function based on the id suffix.

Future implementations could also modify/extend this scheme and routing to pallet agnostic queries.

### Executing externally

These view functions can be executed externally via the system runtime api:

```rust
pub trait ViewFunctionsApi<QueryId, Query, QueryResult, Error> where
	QueryId: codec::Codec,
	Query: codec::Codec,
	QueryResult: codec::Codec,
	Error: codec::Codec,
{
	/// Execute a view function query.
fn execute_query(query_id: QueryId, query: Query) -> Result<QueryResult,
Error>;
}
```
### `XCQ`
Currently there is work going on by @xlc to implement [`XCQ`](https://github.com/open-web3-stack/XCQ/) which may eventually supersede this work.

It may be that we still need the fixed function local query dispatching in addition to XCQ, in the same way that we have chain specific runtime dispatchables and XCM.

I have kept this in mind and the high level query API is agnostic to the underlying query dispatch and execution. I am just providing the implementation for the `view_function` definition.

### Metadata
Currently I am utilizing the `custom` section of the frame metadata, to avoid modifying the official metadata format until this is standardized.

### vs `runtime_api`
There are similarities with `runtime_apis`, some differences being:
- queries can be defined directly on pallets, so no need for boilerplate declarations and implementations
- no versioning, the `QueryId` will change if the signature changes. 
- possibility for queries to be executed from smart contracts (see below)

### Calling from contracts
Future work would be to add `weight` annotations to the view function queries, and a host function to `pallet_contracts` to allow executing these queries from contracts.

### TODO

- [x] Consistent naming (view functions pallet impl, queries, high level api?)
- [ ] End to end tests via `runtime_api`
- [ ] UI tests
- [x] Mertadata tests
- [ ] Docs

---------

Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>
Co-authored-by: Guillaume Thiolliere <guillaume.thiolliere@parity.io>

This PR changes how we call runtime API methods with more than 6
arguments: They are no longer spilled to the stack but packed into
registers instead. Pointers are 32 bit wide so we can pack two of them
into a single 64 bit register. Since we mostly pass pointers, this
technique effectively increases the number of arguments we can pass
using the available registers.

To make this work for `instantiate` too we now pass the code hash and
the call data in the same buffer, akin to how the `create` family
opcodes work in the EVM. The code hash is fixed in size, implying the
start of the constructor call data.

---------

Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Co-authored-by: command-bot <>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Part of: https://github.com/paritytech/polkadot-sdk/issues/6906

Improve debugging by using `#[track_caller]` in system `assert_last_event` and `assert_has_event` (#7142)

Without track caller the error message of the assert points to the
`assert_last_event` function, which is not useful.
```
thread 'tests::set_metadata_works' panicked at /home/gui/Developpement/polkadot-sdk/substrate/frame/system/src/lib.rs:2034:9:
assertion `left == right` failed: expected event RuntimeEvent::Referenda(Event::MetadataSet { index: 0, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa }) is not equal to the last event RuntimeEvent::Referenda(Event::MetadataSet { index: 1, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
  left: RuntimeEvent::Referenda(Event::MetadataSet { index: 1, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
 right: RuntimeEvent::Referenda(Event::MetadataSet { index: 0, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
```

With the track caller the error message points to the caller, showing
the source of the error:
```
thread 'tests::set_metadata_works' panicked at substrate/frame/referenda/src/tests.rs:639:9:
assertion `left == right` failed: expected event RuntimeEvent::Referenda(Event::MetadataSet { index: 0, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa }) is not equal to the last event RuntimeEvent::Referenda(Event::MetadataSet { index: 1, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
  left: RuntimeEvent::Referenda(Event::MetadataSet { index: 1, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
 right: RuntimeEvent::Referenda(Event::MetadataSet { index: 0, hash: 0xbb30a42c1e62f0afda5f0a4e8a562f7a13a24cea00ee81917b86b89e801314aa })
```

I also improved the error message to include a warning when checking
events on block number zero.

This regression was introduced with some of the recent PRs. Regression
fixed and test added.

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

- Add option to specify database_url using DATABASE_URL environment
variable
- Add a eth-rpc-tester rust bin that can be used to test deployment
before releasing eth-rpc
- make evm_block non fallible so that it can return an Ok response for
older blocks when the runtime API is not available
- update cargo.lock to integrate changes from
https://github.com/paritytech/subxt/pull/1904

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Closes #6196 
Closes #7204

Example of PR: https://github.com/paritytech/polkadot-sdk/pull/6816

Every sunday 01:00 AM it's going to start to benchmark (with /cmd bench)
all runtimes and all pallets
Then diff total will be pushed to a branch and PR open,. I assume
review-bot is going assign required reviewers per changed files

I afraid each weeks will be too much to review & merge, but we can
adjust later

Bonus: fix for pallet_multisig lib and
substrate/.maintain/frame-weight-template.hbs , which didn't let to
compile new weights

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

# Derived Dev Accounts

Resolves https://github.com/paritytech/polkadot-sdk/issues/6040

## Description
This update introduces support for creating an arbitrary number of
developer accounts at the genesis block based on a specified derivation
path. This functionality is gated by the runtime-benchmarks feature,
ensuring it is only enabled during benchmarking scenarios.

### Key Features
- Arbitrary Dev Accounts at Genesis: Developers can now specify any
number of accounts to be generated at genesis using a hard derivation
path.

- Default Derivation Path: If no derivation path is provided (i.e., when
`Option<dev_accounts: (..., None)>` is set to `Some` at genesis), the
system will default to the path `//Sender//{}`.

- No Impact on Total Token Issuance: Developer accounts are excluded
from the total issuance of the token supply at genesis, ensuring they do
not affect the overall balance or token distribution....

- Fix the EVM fee cost estimation.
The estimation shown in EVM wallet was using Native instead of EVM
decimals
- Remove the precise code length estimation in dry run call.
Over-estimating is fine, since extra gas is refunded anyway.
- Ensure that the estimated fee calculated from gas_price x gas use the
encoded weight & deposit limit instead of the exact one calculated by
the dry-run. Else we can end up with a fee that is lower than the actual
fee paid by the user

---------

Co-authored-by: command-bot <>

Part of https://github.com/paritytech/polkadot-sdk/issues/6504

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

Related to https://github.com/paritytech/polkadot-sdk/issues/4523

Follow-up for: https://github.com/paritytech/polkadot-sdk/pull/5188

Reopening https://github.com/paritytech/polkadot-sdk/pull/6732 as a new
PR

---------

Co-authored-by: command-bot <>

Part of  #6504

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>

Link-checker job is constantly failing because of these two links.

In the browser there is a redirect, apparently our lychee checker can't
handle it.

For eth-indexer, it's probably safer to use `subscribe_finalized` and
index these blocks into the DB rather than `subscribe_best`

---------

Co-authored-by: command-bot <>

Part of https://github.com/paritytech/polkadot-sdk/issues/6504

### Added modules

- `utility`: Traits not tied to any direct operation in the runtime.

polkadot address: 14SRqZTC1d8rfxL8W1tBTnfUBPU23ACFVPzp61FyGf4ftUFg

---------

Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>

Part of https://github.com/paritytech/polkadot-sdk/issues/6504

Log error instead of failing with an error when block processing fails

---------

Co-authored-by: command-bot <>

Add foundation for supporting call traces in pallet_revive

Follow up:
- PR #7167 Add changes to eth-rpc to introduce debug endpoint that will
use pallet-revive tracing features
- PR #6727 Add new RPC to the client and implement tracing runtime API
that can capture traces on previous blocks

---------

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Update to PolkaVM `0.19`. This version renumbers the opcodes in order to
be in-line with the grey paper. Hopefully, for the last time. This means
that it breaks existing contracts.

---------

Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: xermicus <cyrill@parity.io>

We already use it for lots of pallet.

Keeping it feature gated by experimental means we lose the information
of which pallet was using experimental before the migration to frame
crate usage.

We can consider `polkadot-sdk-frame` crate unstable but let's not use
the feature `experimental`.

---------

Co-authored-by: command-bot <>

Migrate staking currency from `traits::LockableCurrency` to
`traits::fungible::holds`.

Resolves part of https://github.com/paritytech/polkadot-sdk/issues/226.

## Changes
### Nomination Pool
TransferStake is now incompatible with fungible migration as old pools
were not meant to have additional ED. Since they are anyways deprecated,
removed its usage from all test runtimes.

### Staking
- Config: `Currency` becomes of type `Fungible` while `OldCurrency` is
the `LockableCurrency` used before.
- Lazy migration of accounts. Any ledger update will create a new hold
with no extra reads/writes. A permissionless extrinsic
`migrate_currency()` releases the old `lock` along with some
housekeeping.
- Staking now requires ED to be left free. It also adds no consumer to
staking accounts.
- If hold cannot be applied to all stake, the un-holdable part is force
withdrawn from the ledger.

### Delegated Staking
The pallet does not add provider for agents anymore.

## Migration stats
### Polkadot
Total accounts that can be migrated: 59564
Accounts failing to migrate: 0
Accounts with stake force withdrawn greater than ED: 59
Total force withdrawal: 29591.26 DOT

### Kusama
Total accounts that can be migrated: 26311
Accounts failing to migrate: 0
Accounts with stake force withdrawn greater than ED: 48
Total force withdrawal: 1036.05 KSM


[Full logs here](https://hackmd.io/@ak0n/BklDuFra0).

## Note about locks (freeze) vs holds
With locks or freezes, staking could use total balance of an account.
But with holds, the account needs to be left with at least Existential
Deposit in free balance. This would also affect nomination pools which
till now has been able to stake all funds contributed to it. An
alternate version of this PR is
https://github.com/paritytech/polkadot-sdk/pull/5658 where staking
pallet does not add any provider, but means pools and delegated-staking
pallet has to provide for these accounts and makes the end to end logic
(of provider and consumer ref) lot less intuitive and prone to bug.

This PR now introduces requirement for stakers to maintain ED in their
free balance. This helps with removing the bug prone incrementing and
decrementing of consumers and providers.

## TODO
- [x] Test: Vesting + governance locked funds can be staked.
- [ ] can `Call::restore_ledger` be removed? @gpestana 
- [x] Ensure unclaimed withdrawals is not affected by no provider for
pool accounts.
- [x] Investigate kusama accounts with balance between 0 and ED.
- [x] Permissionless call to release lock.
- [x] Migration of consumer (dec) and provider (inc) for direct stakers.
- [x] force unstake if hold cannot be applied to all stake.
- [x] Fix try state checks (it thinks nothing is staked for unmigrated
ledgers).
- [x] Bench `migrate_currency`.
- [x] Virtual Staker migration test.
- [x] Ensure total issuance is upto date when minting rewards.

## Followup
- https://github.com/paritytech/polkadot-sdk/issues/5742

---------

Co-authored-by: command-bot <>

closes https://github.com/paritytech/polkadot-sdk/issues/6911

Closes #3149 

## Description

This PR introduces `pallet-asset-rewards`, which allows accounts to be
rewarded for freezing `fungible` tokens. The motivation for creating
this pallet is to allow incentivising LPs.

See the pallet docs for more info about the pallet.

## Runtime changes

The pallet has been added to
- `asset-hub-rococo`
- `asset-hub-westend`

The `NativeAndAssets` `fungibles` Union did not contain `PoolAssets`, so
it has been renamed `NativeAndNonPoolAssets`

A new `fungibles` Union `NativeAndAllAssets` was created to encompass
all assets and the native token.

## TODO
- [x] Emulation tests
- [x] Fill in Freeze logic (blocked
https://github.com/paritytech/polkadot-sdk/issues/3342) and re-run
benchmarks

---------

Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: muharem <ismailov.m.h@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Closes https://github.com/paritytech/polkadot-sdk/issues/6891.

cc @athei @xermicus @pgherveou

Remove all pallet::events except for the `ContractEmitted` event that is
emitted by contracts

---------

Co-authored-by: command-bot <>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Remove the `debug_buffer` feature

---------

Co-authored-by: command-bot <>
Co-authored-by: Cyrill Leutwiler <cyrill@parity.io>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Bump asset-hub westend spec version

---------

Co-authored-by: command-bot <>

Closes #6846 .

---------

Signed-off-by: xermicus <cyrill@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: xermicus <cyrill@parity.io>

Closes https://github.com/paritytech/polkadot-sdk/issues/6767.

The return type of the host function `caller_is_root` was denoted as
`u32` in `pallet_revive_uapi`. This PR fixes the return type to `bool`.

As a drive-by, the PR re-exports `pallet_revive::exec::Origin` to extend
what can be tested externally.

---------

Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

Introduce `frame_system::Pallet::run_to_block`,
`frame_system::Pallet::run_to_block_with`, and
`frame_system::RunToBlockHooks` to establish a generic `run_to_block`
mechanism for mock tests, minimizing redundant implementations across
various pallets.

Closes #299.

---

Polkadot address: 156HGo9setPcU2qhFMVWLkcmtCEGySLwNqa3DaEiYSWtte4Y

---------

Signed-off-by: Xavier Lau <x@acg.box>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Co-authored-by: Guillaume Thiolliere <guillaume.thiolliere@parity.io>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Fix error message in `DispatchInfo` where post-dispatch and pre-dispatch
weight was reversed.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

Add an option to persist EVM transaction hash to a SQL db.
This should make it possible to run a full archive ETH RPC node
(assuming the substrate node is also a full archive node)

Some queries such as eth_getTransactionByHash,
eth_getBlockTransactionCountByHash, and other need to work with a
transaction hash indexes, which are not stored in Substrate and need to
be stored by the eth-rpc proxy.

The refactoring break down the Client into a `BlockInfoProvider` and
`ReceiptProvider`
- BlockInfoProvider does not need any persistence data, as we can fetch
all block info from the source substrate chain
- ReceiptProvider comes in two flavor, 
  - An in memory cache implementation - This is the one we had so far.
- A DB implementation - This one persist rows with the block_hash, the
transaction_index and the transaction_hash, so that we can later fetch
the block and extrinsic for that receipt and reconstruct the ReceiptInfo
object.

This PR also adds ...

As per #3326, removes pallet::getter macro usage from pallet-grandpa.
The syntax `StorageItem::<T, I>::get()` should be used instead.

cc @muraca

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Update the current approach to attach the `ref_time`, `pov` and
`deposit` parameters to an Ethereum transaction.
Previously we will pass these 3 parameters along with the signed
payload, and check that the fees resulting from `gas x gas_price` match
the actual fees paid by the user for the extrinsic.

This approach unfortunately can be attacked. A malicious actor could
force such a transaction to fail by injecting low values for some of
these extra parameters as they are not part of the signed payload.

The new approach encodes these 3 extra parameters in the lower digits of
the transaction gas, approximating the the log2 of the actual values to
encode each components on 2 digits

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: command-bot <>

# Description

Migrating salary pallet to use umbrella crate. It is a follow-up from
https://github.com/paritytech/polkadot-sdk/pull/7025
Why did I create this new branch? 
I did this, so that the unnecessary cargo fmt changes from the previous
branch are discarded and hence opened this new PR.



## Review Notes

This PR migrates pallet-salary to use the umbrella crate.

Added change: Explanation requested for why `TestExternalities` was
replaced by `TestState` as testing_prelude already includes it
`pub use sp_io::TestExternalities as TestState;`


I have also modified the defensive! macro to be compatible with umbrella
crate as it was being used in the salary pallet

Co-authored-by: Dónal Murray <donalm@seadanda.dev>