# Derived Dev Accounts

Resolves https://github.com/paritytech/polkadot-sdk/issues/6040

## Description
This update introduces support for creating an arbitrary number of
developer accounts at the genesis block based on a specified derivation
path. This functionality is gated by the runtime-benchmarks feature,
ensuring it is only enabled during benchmarking scenarios.

### Key Features
- Arbitrary Dev Accounts at Genesis: Developers can now specify any
number of accounts to be generated at genesis using a hard derivation
path.

- Default Derivation Path: If no derivation path is provided (i.e., when
`Option<dev_accounts: (..., None)>` is set to `Some` at genesis), the
system will default to the path `//Sender//{}`.

- No Impact on Total Token Issuance: Developer accounts are excluded
from the total issuance of the token supply at genesis, ensuring they do
not affect the overall balance or token distribution.

polkadot address: 14SRqZTC1d8rfxL8W1tBTnfUBPU23ACFVPzp61FyGf4ftUFg

---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

Co-authored-by: Dónal Murray <donalm@seadanda.dev>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

Follow up to https://github.com/paritytech/polkadot-sdk/pull/3685
Partially fixes https://github.com/paritytech/polkadot-sdk/issues/6403

The main PR introduced bare support for the new extension version byte
as well as extension weights and benchmarking.

This PR:
- Removes the redundant extension version byte from the signed v4
extrinsic, previously unused and defaulted to 0.
- Adds the extension version byte to the inherited implication passed to
`General` transactions.
- Whitelists the `pallet_authorship::Author`, `frame_system::Digest` and
`pallet_transaction_payment::NextFeeMultiplier` storage items as they
are read multiple times by extensions for each transaction, but are hot
in memory and currently overestimate the weight.
- Whitelists the benchmark caller for `CheckEra` and `CheckGenesis` as
the reads are performed for every transaction and overestimate the
weight.
- Updates the umbrella frame weight template to work with the system
extension changes.
- Plans on re-running the benchmarks at least for the `frame_system`
extensions.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: gui <gui.thiolliere@gmail.com>

# Description

Resolves #6193

This PR introduces `ConstUint` as a replacement for existing constant
getter types like `ConstU8`, `ConstU16`, etc., providing a more flexible
and unified approach.

## Integration

This update is backward compatible, so developers can choose to adopt
`ConstUint` in new implementations or continue using the existing types
as needed.

## Review Notes

`ConstUint` is a convenient alternative to `ConstU8`, `ConstU16`, and
similar types, particularly useful for configuring `DefaultConfig` in
pallets. It enables configuring the underlying integer for a specific
type without the need to update all dependent types, offering enhanced
flexibility in type management.

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)
* [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

Original PR https://github.com/paritytech/polkadot-sdk/pull/2280
reverted in https://github.com/paritytech/polkadot-sdk/pull/3665

This PR reintroduces the reverted functionality with additional changes,
related effort
[here](https://github.com/paritytech/polkadot-sdk/pull/3623).
Description is copied over from the original PR

First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415)

Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for the latter. (Ultimately, the latter will be phased out as
a type of transaction, and Bare will only be used for Inherents.)

Types of extrinsic are now therefore:
- Bare (no hardcoded signature, no Extra data; used to be known as
"Unsigned")
- Bare transactions (deprecated): Gossiped, validated with
`ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
`TransactionExtension` (deprecated).
  - Inherents: Not gossiped, validated with `ProvideInherent`.
- Extended (Extra data): Gossiped, validated via `TransactionExtension`.
  - Signed transactions (with a hardcoded signature) in extrinsic v4.
- General transactions (without a hardcoded signature) in extrinsic v5.

`TransactionExtension` differs from `SignedExtension` because:
- A signature on the underlying transaction may validly not be present.
- It may alter the origin during validation.
- `pre_dispatch` is renamed to `prepare` and need not contain the checks
present in `validate`.
- `validate` and `prepare` is passed an `Origin` rather than a
`AccountId`.
- `validate` may pass arbitrary information into `prepare` via a new
user-specifiable type `Val`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`. It is encoded *for the entire transaction* and
passed in to each extension as a new argument to `validate`. This
facilitates the ability of extensions to acts as underlying crypto.

There is a new `DispatchTransaction` trait which contains only default
function impls and is impl'ed for any `TransactionExtension` impler. It
provides several utility functions which reduce some of the tedium from
using `TransactionExtension` (indeed, none of its regular functions
should now need to be called directly).

Three transaction version discriminator ("versions") are now permissible
(RFC [here](https://github.com/polkadot-fellows/RFCs/pull/84)) in
extrinsic version 5:
- 0b00000100 or 0b00000101: Bare (used to be called "Unsigned"):
contains Signature or Extra (extension data). After bare transactions
are no longer supported, this will strictly identify an Inherents only.
Available in both extrinsic versions 4 and 5.
- 0b10000100: Old-school "Signed" Transaction: contains Signature, Extra
(extension data) and an extension version byte, introduced as part of
[RFC99](https://github.com/polkadot-fellows/RFCs/blob/main/text/0099-transaction-extension-version.md).
Still available as part of extrinsic v4.
- 0b01000101: New-school "General" Transaction: contains Extra
(extension data) and an extension version byte, as per RFC99, but no
Signature. Only available in extrinsic v5.

For the New-school General Transaction, it becomes trivial for authors
to publish extensions to the mechanism for authorizing an Origin, e.g.
through new kinds of key-signing schemes, ZK proofs, pallet state,
mutations over pre-authenticated origins or any combination of the
above.

`UncheckedExtrinsic` still maintains encode/decode backwards
compatibility with extrinsic version 4, where the first byte was encoded
as:
- 0b00000100 - Unsigned transactions
- 0b10000100 - Old-school Signed transactions, without the extension
version byte

Now, `UncheckedExtrinsic` contains a `Preamble` and the actual call. The
`Preamble` describes the type of extrinsic as follows:
```rust
/// A "header" for extrinsics leading up to the call itself. Determines the type of extrinsic and
/// holds any necessary specialized data.
#[derive(Eq, PartialEq, Clone)]
pub enum Preamble<Address, Signature, Extension> {
	/// An extrinsic without a signature or any extension. This means it's either an inherent or
	/// an old-school "Unsigned" (we don't use that terminology any more since it's confusable with
	/// the general transaction which is without a signature but does have an extension).
	///
	/// NOTE: In the future, once we remove `ValidateUnsigned`, this will only serve Inherent
	/// extrinsics and thus can be renamed to `Inherent`.
	Bare(ExtrinsicVersion),
	/// An old-school transaction extrinsic which includes a signature of some hard-coded crypto.
	/// Available only on extrinsic version 4.
	Signed(Address, Signature, ExtensionVersion, Extension),
	/// A new-school transaction extrinsic which does not include a signature by default. The
	/// origin authorization, through signatures or other means, is performed by the transaction
	/// extension in this extrinsic. Available starting with extrinsic version 5.
	General(ExtensionVersion, Extension),
}
```

## Code Migration

### NOW: Getting it to build

Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
accompanied by renaming your aggregate type in line with the new
terminology. E.g. Before:

```rust
/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
	/* snip */
	MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
```

After:

```rust
/// The extension to the basic transaction logic.
pub type TxExtension = (
	/* snip */
	AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
```

You'll also need to alter any transaction building logic to add a
`.into()` to make the conversion happen. E.g. Before:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let extra: SignedExtra = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */),
	);
	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		extra,
	)
}
```

After:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let tx_ext: TxExtension = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */).into(),
	);
	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		tx_ext,
	)
}
```

### SOON: Migrating to `TransactionExtension`

Most `SignedExtension`s can be trivially converted to become a
`TransactionExtension`. There are a few things to know.

- Instead of a single trait like `SignedExtension`, you should now
implement two traits individually: `TransactionExtensionBase` and
`TransactionExtension`.
- Weights are now a thing and must be provided via the new function `fn
weight`.

#### `TransactionExtensionBase`

This trait takes care of anything which is not dependent on types
specific to your runtime, most notably `Call`.

- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`.
- Weight must be returned by implementing the `weight` function. If your
extension is associated with a pallet, you'll probably want to do this
via the pallet's existing benchmarking infrastructure.

#### `TransactionExtension`

Generally:
- `pre_dispatch` is now `prepare` and you *should not reexecute the
`validate` functionality in there*!
- You don't get an account ID any more; you get an origin instead. If
you need to presume an account ID, then you can use the trait function
`AsSystemOriginSigner::as_system_origin_signer`.
- You get an additional ticket, similar to `Pre`, called `Val`. This
defines data which is passed from `validate` into `prepare`. This is
important since you should not be duplicating logic from `validate` to
`prepare`, you need a way of passing your working from the former into
the latter. This is it.
- This trait takes a `Call` type parameter. `Call` is the runtime call
type which used to be an associated type; you can just move it to become
a type parameter for your trait impl.
- There's no `AccountId` associated type any more. Just remove it.

Regarding `validate`:
- You get three new parameters in `validate`; all can be ignored when
migrating from `SignedExtension`.
- `validate` returns a tuple on success; the second item in the tuple is
the new ticket type `Self::Val` which gets passed in to `prepare`. If
you use any information extracted during `validate` (off-chain and
on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
pass it through with this. For the tuple's last item, just return the
`origin` argument.

Regarding `prepare`:
- This is renamed from `pre_dispatch`, but there is one change:
- FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
`validate`!!
- (This is different to `SignedExtension` which was required to run the
same checks in `pre_dispatch` as in `validate`.)

Regarding `post_dispatch`:
- Since there are no unsigned transactions handled by
`TransactionExtension`, `Pre` is always defined, so the first parameter
is `Self::Pre` rather than `Option<Self::Pre>`.

If you make use of `SignedExtension::validate_unsigned` or
`SignedExtension::pre_dispatch_unsigned`, then:
- Just use the regular versions of these functions instead.
- Have your logic execute in the case that the `origin` is `None`.
- Ensure your transaction creation logic creates a General Transaction
rather than a Bare Transaction; this means having to include all
`TransactionExtension`s' data.
- `ValidateUnsigned` can still be used (for now) if you need to be able
to construct transactions which contain none of the extension data,
however these will be phased out in stage 2 of the Transactions Horizon,
so you should consider moving to an extension-centric design.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>

closes #5942

Couldn't find any emissions of `Event::Issued` without amount check
other than in this PR.

Currently, we have;


https://github.com/paritytech/polkadot-sdk/blob/4bda956d/substrate/frame/balances/src/impl_currency.rs#L212-L220

and


https://github.com/paritytech/polkadot-sdk/blob/4bda956d/substrate/frame/balances/src/impl_currency.rs#L293-L306

# Description
## What?
Make it possible for other pallets to implement their own logic when a
slash on a balance occurs.

## Why?
In the [introduction of
holds](https://github.com/paritytech/substrate/pull/12951) @gavofyork
said:
> Since Holds are designed to be infallibly slashed, this means that any
logic using a Freeze must handle the possibility of the frozen amount
being reduced, potentially to zero. A permissionless function should be
provided in order to allow bookkeeping to be updated in this instance.

At Polimec we needed to find a way to reduce the vesting schedules of
our users after a slash was made, and after talking to @Kianenigma

 at
the Web3Summit, we realized there was no easy way to implement this with
the current traits, so we came up with this solution.



## How?
- First we abstract the `done_slash` function of holds::Balanced to it's
own trait that any pallet can implement.
- Then we add a config type in pallet-balances that accepts a callback
tuple of all the pallets that implement this trait.
- Finally implement done_slash for pallet-balances such that it calls
the config type.

## Integration
The default implementation of done_slash is still an empty function, and
the new config type of pallet-balances can be set to an empty tuple, so
nothing changes by default.

## Review Notes
- I suggest to focus on the first commit which contains the main logic
changes.
- I also have a working implementation of done_slash for pallet_vesting,
should I add it to this PR?
- If I run `cargo +nightly fmt --all` then I get changes to a lot of
unrelated crates, so not sure if I should run it to avoid the fmt
failure of the CI
- Should I hunt down references to fungible/fungibles documentation and
update it accordingly?

**Polkadot address:** `15fj1UhQp8Xes7y7LSmDYTy349mXvUwrbNmLaP5tQKBxsQY1`

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>

cc https://github.com/paritytech/ci_cd/issues/1035

cc https://github.com/paritytech/ci_cd/issues/1023

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Maksym H <1177472+mordamax@users.noreply.github.com>
Co-authored-by: gui <gui.thiolliere@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>

- This will ensure that a correct product name
(Polkadot/Cumulus/Substrate) is referenced in license headers.
- Closes https://github.com/paritytech/license-scanner/issues/49

Closes: https://github.com/paritytech/polkadot-sdk/issues/5545

- add missing template for frame & xcm benchmarks
- fix `git pull` ->
https://github.com/paritytech/polkadot-sdk/actions/runs/10644887539/job/29510118915
- respect runtimes headers - use GNU instead of apache for runtimes
- adds tests for cmd.py

Tip: review this one with Whitespace hidden

![image](https://github.com/user-attachments/assets/3bcdc6c2-7371-428f-9962-556ca81c1467)

---------

Co-authored-by: GitHub Action <action@github.com>

Introduce a dynamic proposal deposit mechanism influenced by the total
number of active proposals, with the option to set the deposit to none.

The potential cost (e.g., balance hold) for proposal submission and
storage is determined by the implementation of the `Consideration`
trait. The footprint is defined as `proposal_count`, representing the
total number of active proposals in the system, excluding the one
currently being proposed. This cost may vary based on the proposal
count. The pallet also offers various types to define a cost strategy
based on the number of proposals.

Two new calls are introduced:
- kill(origin, proposal_hash): the cancellation of a proposal,
accompanied by the burning of the associated cost/consideration ticket.
- release_proposal_cost(origin, proposal_hash): the release of the cost
for a non-active proposal.

Additionally change: 
- benchmarks have been upgraded to benchmarks::v2 for collective pallet;
- `ensure_successful` function added to the `Consideration` under
`runtime-benchmarks` feature.

---------

Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: GitHub Action <action@github.com>

Currently, when the pallet is compiled with the `insecure_zero_ed flag`,
benchmarks fail because the minimum balance is set to zero.

The PR aims to resolve this issue by implementing a placeholder value
for the minimum balance when the `insecure_zero_ed` flag is active. it
ensures that benchmarks run successfully regardless of whether this flag
is used or not

Introduce `MaybeConsideration` extension trait for `Consideration`.

The trait allows for the management of tickets that may represent no
cost. While the `MaybeConsideration` still requires proper handling, it
introduces the ability to determine if a ticket represents no cost and
can be safely forgotten without any side effects.

The new trait is particularly useful when a consumer expects the cost to
be zero under certain conditions (e.g., when the proposal count is below
a threshold N) and does not want to store such consideration tickets in
storage. The extension approach allows us to avoid breaking changes to
the existing trait and to continue using it as a non-optional version
for migrating pallets that utilize the `Currency` and `fungible` traits
for `holds` and `freezes`, without requiring any storage migration.

Make ticket non-optional and add ensure_successful method to
Consideration trait.

Reverts the optional return ticket type for the new function introduced
in
[polkadot-sdk/4596](https://github.com/paritytech/polkadot-sdk/pull/4596)
and adds a helper `ensure_successful` function for the runtime
benchmarks.
Since the existing FRAME pallet represents zero cost with a zero balance
rather than `None` in an option, maintaining the ticket type as a
non-optional balance is beneficial for backward compatibility and helps
avoid unnecessary migrations.

As per #3326, removes usage of the `pallet::getter` macro from the
balances pallet. The syntax `StorageItem::<T, I>::get()` should be used
instead.

Also, adds public functions for compatibility.

cc @muraca

polkadot address: 5GsLutpKjbzsbTphebs9Uy4YK6gTN47MAaz6njPktidjR5cp

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Make `on_unbalanceds` work with `fungibles` `imbalances`.

The `fungibles` `imbalances` cannot be handled by the default
implementation of `on_unbalanceds` from the `OnUnbalanced` trait. This
is because the `fungibles` `imbalances` types do not implement the
`Imbalance` trait (and cannot with its current semantics). The
`on_unbalanceds` function requires only the `merge` function for the
imbalance type. In this PR, we provide the `TryMerge` trait, which can
be implemented by all imbalance types and make `OnUnbalanced` require it
instead `Imbalance`.

### Migration for `OnUnbalanced` trait implementations:
In case if you have a custom implementation of `on_unbalanceds` trait
function, remove it's `<B>` type argument.

### Migration for custom imbalance types:
If you have your own imbalance types implementations, implement the
`TryMerge` trait for it introduced with this update.
        
The applicability of the `on_unbalanceds` function to fungibles
imbalances is useful in cases like -
[link](https://github.com/paritytech/polkadot-sdk/blob/3a8e675e

/substrate/frame/transaction-payment/asset-conversion-tx-payment/src/payment.rs#L267)
from https://github.com/paritytech/polkadot-sdk/pull/4488.

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

# Description

Previously, in the `Currency` impl, the implementation of
`pallet_balances` was not emitting any instances of `Issued` and
`Rescinded` events, even though the `Fungible` equivalent was.

This PR adds the `Issued` and `Rescinded` events in appropriate places
in `impl_currency` along with tests.

Closes #4028 

polkadot address: 5GsLutpKjbzsbTphebs9Uy4YK6gTN47MAaz6njPktidjR5cp

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Bastian Köcher <info@kchr.de>

This should remove nearly all usage of `sp-std` except:
- bridge and bridge-hubs
- a few of frames re-export `sp-std`, keep them for now
- there is a usage of `sp_std::Writer`, I don't have an idea how to move
it

Please review proc-macro carefully. I'm not sure I'm doing it the right
way.

Note: need `/bot fmt`

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>

`Consideration` trait generic over `Footprint` and indicates zero cost
for a give footprint.

`Consideration` trait is generic over `Footprint` (currently defined
over the type with the same name). This makes it possible to setup a
custom footprint (e.g. current number of proposals in the storage).

`Consideration::new` and `Consideration::update` return an
`Option<Self>` instead `Self`, this make it possible to indicate a no
cost for a specific footprint (e.g. if current number of proposals in
the storage < max_proposal_count / 2 then no cost).

These cases need to be handled for
https://github.com/paritytech/polkadot-sdk/pull/3151

Closes #3342

cc/ @liamaharon



TODO:

- [x] Improve docs.
- [x] Define public interface (See #3342).
  In case we define public calls to the pallet implementation:
  - Implement public calls.
  - Benchmarks.
  
polkadot address: 12gMhxHw8QjEwLQvnqsmMVY1z5gFa54vND74aMUbhhwN6mJR

---------

Co-authored-by: command-bot <>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

This PR aligns the settings for `MaxFreezes`, `RuntimeFreezeReason`, and
`FreezeIdentifier`.

#### Future work and improvements
https://github.com/paritytech/polkadot-sdk/issues/2997 (remove
`MaxFreezes` and `FreezeIdentifier`)

Co-authored-by: command-bot <>

#### Problem
`take()` consumes only 1 read worth of weight in
`single-block-migrations` example, while `take()`
[is](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/support/src/storage/unhashed.rs#L63)
`get() + kill()`, i.e should be 1 read + 1 write. I think this could
mislead developers who follow this example to write their migrations

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Adds an additional extrinsic call to the `pallet-balances` to _burn_
tokens.
Depending on the `keep_alive` flag, the call might or might not reap the
account.

Required modification of the _fungible's_ `Mutate` trait, `burn_from`
function to allow the `Preservation` argument.

**TODO**
- [x] run benchmarks & update weights
- [x] make sure prdoc is required & properly formatted

Related issue: https://github.com/paritytech/polkadot-sdk/issues/3943

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>

Allow for 0 existential deposit in benchmarks for `pallet_staking`, `pallet_session`, and `pallet_balances` (#4346)

This PR ensures non-zero values are available in benchmarks for
`pallet_staking`, `pallet_session`, and `pallet_balances` where required
for them to run.

This small change makes it possible to run the benchmarks for
`pallet_staking`, `pallet_session`, and `pallet_balances` in a runtime
for which existential deposit is set to 0.

The benchmarks for `pallet_staking` and `pallet_session` will still fail
in runtimes that use `U128CurrencyToVote`, but that is easy to work
around by creating a new `CurrencyToVote` implementation for
benchmarking.

The changes are implemented by checking if existential deposit equals 0
and using 1 if so.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

There are some accounts that do not have a consumer ref while having a
reserve.
This adds a fail-safe mechanism to trigger in the case that
`does_consume` is true, but the assumption of `consumer>0` is not.

This should prevent those accounts from loosing balance and the TI from
getting messed up even more, but is not an "ideal" fix. TBH an ideal fix
is not possible, since on-chain data is in an invalid state.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Current benchmarking macro returns a closure with the captured
benchmarked code.
This can cause issues when the benchmarked code has complex lifetime
requirements.

This PR updates the existing macro by injecting the recording parameter
and invoking the start / stop method around the benchmarked block
instead of returning a closure

One other added benefit is that you can write this kind of code now as
well:

```rust
let v;
#[block]
{ v = func.call(); }
dbg!(v); // or assert something on v
```


[Weights compare
link](https://weights.tasty.limo/compare?unit=weight&ignore_errors=true&threshold=10&method=asymptotic&repo=polkadot-sdk&old=pg/fix-weights&new=pg/bench_update&path_pattern=substrate/frame/**/src/weights.rs,polkadot/runtime/*/src/weights/**/*.rs,polkadot/bridges/modules/*/src/weights.rs,cumulus/**/weights/*.rs,cumulus/**/weights/xcm/*.rs,cumulus/**/src/weights.rs)

---------

Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Part of https://github.com/paritytech/polkadot-sdk/issues/226 
Related https://github.com/paritytech/polkadot-sdk/issues/1833

- Deprecate `CurrencyAdapter` and introduce `FungibleAdapter`
- Deprecate `ToStakingPot` and replace usage with `ResolveTo`
- Required creating a new `StakingPotAccountId` struct that implements
`TypedGet` for the staking pot account ID
- Update parachain common utils `DealWithFees`, `ToAuthor` and
`AssetsToBlockAuthor` implementations to use `fungible`
- Update runtime XCM Weight Traders to use `ResolveTo` instead of
`ToStakingPot`
- Update runtime Transaction Payment pallets to use `FungibleAdapter`
instead of `CurrencyAdapter`
- [x] Blocked by https://github.com/paritytech/polkadot-sdk/pull/1296,
needs the `Unbalanced::decrease_balance` fix

Closes https://github.com/paritytech/polkadot-sdk-docs/issues/70

WIP PR for an overview of how to develop tokens in FRAME. 

- [x] Tokens in Substrate Ref Doc
  - High-level overview of the token-related logic in FRAME
- Improve docs with better explanation of how holds, freezes, ed, free
balance, etc, all work
- [x] Update `pallet_balances` docs
  - Clearly mark what is deprecated (currency)
- [x] Write fungible trait docs
- [x] Evaluate and if required update `pallet_assets`, `pallet_uniques`,
`pallet_nfts` docs
- [x] Absorb https://github.com/paritytech/polkadot-sdk/pull/2683/
- [x] Audit individual trait method docs, and improve if possible

Feel free to suggest additional TODOs for this PR in the comments

---------

Co-authored-by: Bill Laboon <laboon@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

This PR adds a new extrinsic `Call::restore_ledger ` gated by
`StakingAdmin` origin that restores a corrupted staking ledger. This
extrinsic will be used to recover ledgers that were affected by the
issue discussed in
https://github.com/paritytech/polkadot-sdk/issues/3245.

The extrinsic will re-write the storage items associated with a stash
account provided as input parameter. The data used to reset the ledger
can be either i) fetched on-chain or ii) partially/totally set by the
input parameters of the call.

In order to use on-chain data to restore the staking locks, we need a
way to read the current lock in the balances pallet. This PR adds a
`InspectLockableCurrency` trait and implements it in the pallet
balances. An alternative would be to tightly couple staking with the
pallet balances but that's inelegant (an example of how it would look
like in [this
branch](https://github.com/paritytech/polkadot-sdk/tree/gpestana/ledger-badstate-clean_tightly)).

More details on the type of corruptions and corresponding fixes
https://hackmd.io/DLb5jEYWSmmvqXC9ae4yRg?view#/

We verified that the `Call::restore_ledger` does fix all current
corrupted ledgers in Polkadot and Kusama. You can verify it here
https://hackmd.io/v-XNrEoGRpe7APR-EZGhOA.

**Changes introduced**
- Adds `Call::restore_ledger ` extrinsic to recover a corrupted ledger;
- Adds trait `frame_support::traits::currency::InspectLockableCurrency`
to allow external pallets to read current locks given an account and
lock ID;
- Implements the `InspectLockableCurrency` in the pallet-balances.
- Adds staking locks try-runtime checks
(https://github.com/paritytech/polkadot-sdk/issues/3751)

**Todo**
- [x] benchmark `Call::restore_ledger`
- [x] throughout testing of all ledger recovering cases
- [x] consider adding the staking locks try-runtime checks to this PR
(https://github.com/paritytech/polkadot-sdk/issues/3751)
- [x] simulate restoring all ledgers
(https://hackmd.io/Dsa2tvhISNSs7zcqriTaxQ?view) in Polkadot and Kusama
using chopsticks -- https://hackmd.io/v-XNrEoGRpe7APR-EZGhOA

Related to https://github.com/paritytech/polkadot-sdk/issues/3245
Closes https://github.com/paritytech/polkadot-sdk/issues/3751

---------

Co-authored-by: command-bot <>

**Update:** Pushed additional changes based on the review comments.

**This pull request fixes various spelling mistakes in this
repository.**

Most of the changes are contained in the first **3** commits:

- `Fix spelling mistakes in comments and docs`

- `Fix spelling mistakes in test names`

- `Fix spelling mistakes in error messages, panic messages, logs and
tracing`

Other source code spelling mistakes are separated into individual
commits for easier reviewing:

- `Fix the spelling of 'authority'`

- `Fix the spelling of 'REASONABLE_HEADERS_IN_JUSTIFICATION_ANCESTRY'`

- `Fix the spelling of 'prev_enqueud_messages'`

- `Fix the spelling of 'endpoint'`

- `Fix the spelling of 'children'`

- `Fix the spelling of 'PenpalSiblingSovereignAccount'`

- `Fix the spelling of 'PenpalSudoAccount'`

- `Fix the spelling of 'insufficient'`

- `Fix the spelling of 'PalletXcmExtrinsicsBenchmark'`

- `Fix the spelling of 'subtracted'`

- `Fix the spelling of 'CandidatePendingAvailability'`

- `Fix the spelling of 'exclusive'`

- `Fix the spelling of 'until'`

- `Fix the spelling of 'discriminator'`

- `Fix the spelling of 'nonexistent'`

- `Fix the spelling of 'subsystem'`

- `Fix the spelling of 'indices'`

- `Fix the spelling of 'committed'`

- `Fix the spelling of 'topology'`

- `Fix the spelling of 'response'`

- `Fix the spelling of 'beneficiary'`

- `Fix the spelling of 'formatted'`

- `Fix the spelling of 'UNKNOWN_PROOF_REQUEST'`

- `Fix the spelling of 'succeeded'`

- `Fix the spelling of 'reopened'`

- `Fix the spelling of 'proposer'`

- `Fix the spelling of 'InstantiationNonce'`

- `Fix the spelling of 'depositor'`

- `Fix the spelling of 'expiration'`

- `Fix the spelling of 'phantom'`

- `Fix the spelling of 'AggregatedKeyValue'`

- `Fix the spelling of 'randomness'`

- `Fix the spelling of 'defendant'`

- `Fix the spelling of 'AquaticMammal'`

- `Fix the spelling of 'transactions'`

- `Fix the spelling of 'PassingTracingSubscriber'`

- `Fix the spelling of 'TxSignaturePayload'`

- `Fix the spelling of 'versioning'`

- `Fix the spelling of 'descendant'`

- `Fix the spelling of 'overridden'`

- `Fix the spelling of 'network'`

Let me know if this structure is adequate.

**Note:** The usage of the words `Merkle`, `Merkelize`, `Merklization`,
`Merkelization`, `Merkleization`, is somewhat inconsistent but I left it
as it is.

~~**Note:** In some places the term `Receival` is used to refer to
message reception, IMO `Reception` is the correct word here, but I left
it as it is.~~

~~**Note:** In some places the term `Overlayed` is used instead of the
more acceptable version `Overlaid` but I also left it as it is.~~

~~**Note:** In some places the term `Applyable` is used instead of the
correct version `Applicable` but I also left it as it is.~~

**Note:** Some usage of British vs American english e.g. `judgement` vs
`judgment`, `initialise` vs `initialize`, `optimise` vs `optimize` etc.
are both present in different places, but I suppose that's
understandable given the number of contributors.

~~**Note:** There is a spelling mistake in `.github/CODEOWNERS` but it
triggers errors in CI when I make changes to it, so I left it as it
is.~~

Step in https://github.com/paritytech/polkadot-sdk/issues/171

This PR removes `as [disambiguation_path]` syntax from `derive_impl`
usage across the polkadot-sdk as introduced in
https://github.com/paritytech/polkadot-sdk/pull/3505

Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280)" (#3665)

This PR reverts #2280 which introduced `TransactionExtension` to replace
`SignedExtension`.

As a result of the discussion
[here](https://github.com/paritytech/polkadot-sdk/pull/3623#issuecomment-1986789700),
the changes will be reverted for now with plans to reintroduce the
concept in the future.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>

Closes #2160

First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415)

Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for the latter. (Ultimately, the latter will be phased out as
a type of transaction, and Bare will only be used for Inherents.)

Types of extrinsic are now therefore:
- Bare (no hardcoded signature, no Extra data; used to be known as
"Unsigned")
- Bare transactions (deprecated): Gossiped, validated with
`ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
`TransactionExtension` (deprecated).
  - Inherents: Not gossiped, validated with `ProvideInherent`.
- Extended (Extra data): Gossiped, validated via `TransactionExtension`.
  - Signed transactions (with a hardcoded signature).
  - General transactions (without a hardcoded signature).

`TransactionExtension` differs from `SignedExtension` because:
- A signature on the underlying transaction may validly not be present.
- It may alter the origin during validation.
- `pre_dispatch` is renamed to `prepare` and need not contain the checks
present in `validate`.
- `validate` and `prepare` is passed an `Origin` rather than a
`AccountId`.
- `validate` may pass arbitrary information into `prepare` via a new
user-specifiable type `Val`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`. It is encoded *for the entire transaction* and
passed in to each extension as a new argument to `validate`. This
facilitates the ability of extensions to acts as underlying crypto.

There is a new `DispatchTransaction` trait which contains only default
function impls and is impl'ed for any `TransactionExtension` impler. It
provides several utility functions which reduce some of the tedium from
using `TransactionExtension` (indeed, none of its regular functions
should now need to be called directly).

Three transaction version discriminator ("versions") are now
permissible:
- 0b000000100: Bare (used to be called "Unsigned"): contains Signature
or Extra (extension data). After bare transactions are no longer
supported, this will strictly identify an Inherents only.
- 0b100000100: Old-school "Signed" Transaction: contains Signature and
Extra (extension data).
- 0b010000100: New-school "General" Transaction: contains Extra
(extension data), but no Signature.

For the New-school General Transaction, it becomes trivial for authors
to publish extensions to the mechanism for authorizing an Origin, e.g.
through new kinds of key-signing schemes, ZK proofs, pallet state,
mutations over pre-authenticated origins or any combination of the
above.

## Code Migration

### NOW: Getting it to build

Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
accompanied by renaming your aggregate type in line with the new
terminology. E.g. Before:

```rust
/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
	/* snip */
	MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
```

After:

```rust
/// The extension to the basic transaction logic.
pub type TxExtension = (
	/* snip */
	AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
```

You'll also need to alter any transaction building logic to add a
`.into()` to make the conversion happen. E.g. Before:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let extra: SignedExtra = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */),
	);
	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		extra,
	)
}
```

After:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let tx_ext: TxExtension = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */).into(),
	);
	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		tx_ext,
	)
}
```

### SOON: Migrating to `TransactionExtension`

Most `SignedExtension`s can be trivially converted to become a
`TransactionExtension`. There are a few things to know.

- Instead of a single trait like `SignedExtension`, you should now
implement two traits individually: `TransactionExtensionBase` and
`TransactionExtension`.
- Weights are now a thing and must be provided via the new function `fn
weight`.

#### `TransactionExtensionBase`

This trait takes care of anything which is not dependent on types
specific to your runtime, most notably `Call`.

- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`.
- Weight must be returned by implementing the `weight` function. If your
extension is associated with a pallet, you'll probably want to do this
via the pallet's existing benchmarking infrastructure.

#### `TransactionExtension`

Generally:
- `pre_dispatch` is now `prepare` and you *should not reexecute the
`validate` functionality in there*!
- You don't get an account ID any more; you get an origin instead. If
you need to presume an account ID, then you can use the trait function
`AsSystemOriginSigner::as_system_origin_signer`.
- You get an additional ticket, similar to `Pre`, called `Val`. This
defines data which is passed from `validate` into `prepare`. This is
important since you should not be duplicating logic from `validate` to
`prepare`, you need a way of passing your working from the former into
the latter. This is it.
- This trait takes two type parameters: `Call` and `Context`. `Call` is
the runtime call type which used to be an associated type; you can just
move it to become a type parameter for your trait impl. `Context` is not
currently used and you can safely implement over it as an unbounded
type.
- There's no `AccountId` associated type any more. Just remove it.

Regarding `validate`:
- You get three new parameters in `validate`; all can be ignored when
migrating from `SignedExtension`.
- `validate` returns a tuple on success; the second item in the tuple is
the new ticket type `Self::Val` which gets passed in to `prepare`. If
you use any information extracted during `validate` (off-chain and
on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
pass it through with this. For the tuple's last item, just return the
`origin` argument.

Regarding `prepare`:
- This is renamed from `pre_dispatch`, but there is one change:
- FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
`validate`!!
- (This is different to `SignedExtension` which was required to run the
same checks in `pre_dispatch` as in `validate`.)

Regarding `post_dispatch`:
- Since there are no unsigned transactions handled by
`TransactionExtension`, `Pre` is always defined, so the first parameter
is `Self::Pre` rather than `Option<Self::Pre>`.

If you make use of `SignedExtension::validate_unsigned` or
`SignedExtension::pre_dispatch_unsigned`, then:
- Just use the regular versions of these functions instead.
- Have your logic execute in the case that the `origin` is `None`.
- Ensure your transaction creation logic creates a General Transaction
rather than a Bare Transaction; this means having to include all
`TransactionExtension`s' data.
- `ValidateUnsigned` can still be used (for now) if you need to be able
to construct transactions which contain none of the extension data,
however these will be phased out in stage 2 of the Transactions Horizon,
so you should consider moving to an extension-centric design.

## TODO

- [x] Introduce `CheckSignature` impl of `TransactionExtension` to
ensure it's possible to have crypto be done wholly in a
`TransactionExtension`.
- [x] Deprecate `SignedExtension` and move all uses in codebase to
`TransactionExtension`.
  - [x] `ChargeTransactionPayment`
  - [x] `DummyExtension`
  - [x] `ChargeAssetTxPayment` (asset-tx-payment)
  - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment)
  - [x] `CheckWeight`
  - [x] `CheckTxVersion`
  - [x] `CheckSpecVersion`
  - [x] `CheckNonce`
  - [x] `CheckNonZeroSender`
  - [x] `CheckMortality`
  - [x] `CheckGenesis`
  - [x] `CheckOnlySudoAccount`
  - [x] `WatchDummy`
  - [x] `PrevalidateAttests`
  - [x] `GenericSignedExtension`
  - [x] `SignedExtension` (chain-polkadot-bulletin)
  - [x] `RefundSignedExtensionAdapter`
- [x] Implement `fn weight` across the board.
- [ ] Go through all pre-existing extensions which assume an account
signer and explicitly handle the possibility of another kind of origin.
- [x] `CheckNonce` should probably succeed in the case of a non-account
origin.
- [x] `CheckNonZeroSender` should succeed in the case of a non-account
origin.
- [x] `ChargeTransactionPayment` and family should fail in the case of a
non-account origin.
  - [ ] 
- [x] Fix any broken tests.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com>
Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Marcin S. <marcin@realemail.net>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Javier Bullrich <javier@bullrich.dev>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Ross Bulat <ross@parity.io>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Julian Eager <eagr@tutanota.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: yjh <yjh465402634@gmail.com>
Co-authored-by: Tom Mi <tommi@niemi.lol>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Will | Paradox | ParaNodes.io <79228812+paradox-tt@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: Joshy Orndorff <git-user-email.h0ly5@simplelogin.com>
Co-authored-by: PG Herveou <pgherveou@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Juan Girini <juangirini@gmail.com>
Co-authored-by: bader y <ibnbassem@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Parth <desaiparth08@gmail.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Jonathan Udd <jonathan@dwellir.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Evgeny Snitko <evgeny@parity.io>
Co-authored-by: Just van Stam <vstam1@users.noreply.github.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
Co-authored-by: dzmitry-lahoda <dzmitry@lahoda.pro>
Co-authored-by: zhiqiangxu <652732310@qq.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Anwesh <anweshknayak@gmail.com>
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Jegor Sidorenko <5252494+jsidorenko@users.noreply.github.com>
Co-authored-by: Muharem <ismailov.m.h@gmail.com>
Co-authored-by: joepetrowski <joe@parity.io>
Co-authored-by: Alexandru Gheorghe <49718502+alexggh@users.noreply.github.com>
Co-authored-by: Gabriel Facco de Arruda <arrudagates@gmail.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: georgepisaltu <george.pisaltu@parity.io>
Co-authored-by: command-bot <>

Closes https://github.com/paritytech/polkadot-sdk-docs/issues/55

- Changes 'current storage version' terminology to less ambiguous
'in-code storage version' (suggestion by @ggwpez

)
- Adds a new example pallet `pallet-example-single-block-migrations`
- Adds a new reference doc to replace
https://docs.substrate.io/maintain/runtime-upgrades/ (temporarily living
in the pallet while we wait for developer hub PR to merge)
- Adds documentation for the `storage_alias` macro
- Improves `trait Hooks` docs 
- Improves `trait GetStorageVersion` docs
- Update the suggested patterns for using `VersionedMigration`, so that
version unchecked migrations are never exported
- Prevents accidental usage of version unchecked migrations in runtimes

https://github.com/paritytech/substrate/pull/14421#discussion_r1255467895
- Unversioned migration code is kept inside `mod version_unchecked`,
versioned code is kept in `pub mod versioned`
- It is necessary to use modules to limit visibility because the inner
migration must be `pub`. See
https://github.com/rust-lang/rust/issues/30905 and

https://internals.rust-lang.org/t/lang-team-minutes-private-in-public-rules/4504/40
for more.

### todo

- [x] move to reference docs to proper place within sdk-docs (now that
https://github.com/paritytech/polkadot-sdk/pull/2102 is merged)
- [x] prdoc

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Juan <juangirini@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>

# Description

This PR removes redundant type definition from test definition config
implementations like
```
#[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)]
impl frame_system::Config for Test {
    type A = A;
    ...
}
```

This changes avoid redundancies in the code as the macro `derive_impl`
defines the relevant types. To implement the changes, it was a simple
fact of running tests and making sure that the tests would still run
while the definition would be removed.

Closes #3237

As a note, here is a brief account of things done from the Issue's
description statement
```
alliance migrate alliance, fast-unstake and bags list to use derive-impl #1636
asset-conversion                                                                                            DONE
asset-rate                                                                                                  DONE
assets                                                                                                      DONE
atomic-swap                                                                                                 DONE
aura                                                                                                        DONE
authority-discovery                                                                                         DONE                                                                     
authorship  migrate babe and authorship to use derive-impl #1790
babe  migrate babe and authorship to use derive-impl #1790
bags-list migrate alliance, fast-unstake and bags list to use derive-impl #1636
balances                                                                                                    DONE
beefy                                                                                                       NOTHING TO DO --- also noted this error without failing tests Feb 13 13:49:08.941 ERROR runtime::timestamp: `pallet_timestamp::UnixTime::now` is called at genesis, invalid value returned: 0
beefy-mmr                                                                                                   NOTHING TO DO
bounties                                                                                                    DONE
child-bounties                                                                                              DONE
collective                                                                                                  DONE
contracts                                                                                                   DONE
conviction-voting                                                                                           DONE
core-fellowship                                                                                             NOTHING TO DO
democracy                                                                                                   DONE
election-provider-multi-phase                                                                               NOTHING TO DO
elections-phragmen                                                                                          DONE
executive                                                                                                   NOTHING TO DO
fast-unstake migrate alliance, fast-unstake and bags list to use derive-impl #1636
glutton                                                                                                     DONE
grandpa                                                                                                     DONE
identity                                                                                                    DONE
im-online                                                                                                   NOTHING TO DO
indices Refactor indices pallet #1789
insecure-randomness-collective-flip                                                                         DONE
lottery                                                                                                     DONE
membership                                                                                                  DONE
merkle-mountain-range                                                                                       NOTHING TO DO
message-queue                                                                                               DONE
multisig add frame_system::DefaultConfig to individual pallet DefaultConfigs substrate#14453
nft-fractionalization                                                                                       DONE
nfts                                                                                                        DONE
nicks Refactor pallet-state-trie-migration to fungible::* traits #1801                                      NOT IN REPO
nis                                                                                                         DONE
node-authorization                                                                                          DONE
nomination-pools                                                                                            NOTHING TO DO -- ONLY impl for Runtime
offences                                                                                                    DELETED EVERYTHING -- IS THAT CORRECT??
preimage                                                                                                    DONE
proxy add frame_system::DefaultConfig to individual pallet DefaultConfigs substrate#14453
ranked-collective                                                                                           NOTHING TO DO
recovery                                                                                                    DONE
referenda                                                                                                   DONE
remark                                                                                                      DONE
root-offences                                                                                               DONE
root-testing                                                                                                NOTHING TO DO
salary                                                                                                      NOTHING TO DO
scheduler                                                                                                   DONE
scored-pool                                                                                                 DONE
session                                                                                                     DONE -- substrate/frame/session/benchmarking/src/mock.rs untouched
society                                                                                                     NOTHING TO DO
staking                                                                                                     DONE
staking-bags-benchmarks                                                                                     NOT IN REPO
state-trie-migration                                                                                        NOTHING TO DO
statement                                                                                                   DONE
sudo                                                                                                        DONE
system                                                                                                      DONE
timestamp                                                                                                   DONE
tips                                                                                                        DONE
transaction-payment                                                                                         NOTHING TO DO
transaction-storage                                                                                         NOTHING TO DO
treasury                                                                                                    DONE
try-runtime                                                                                                 NOTHING TO DO -- no specific mention of 'for Test'
uniques                                                                                                     DONE
utility                                                                                                     DONE
vesting                                                                                                     DONE
whitelist                                                                                                   DONE
```

---------

Co-authored-by: command-bot <>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>

[frame] `#[pallet::composite_enum]` improved variant count handling + removed `pallet_balances`'s `MaxHolds` config (#2657)

I started this investigation/issue based on @liamaharon

 question
[here](https://github.com/paritytech/polkadot-sdk/pull/1801#discussion_r1410452499).

## Problem

The `pallet_balances` integrity test should correctly detect that the
runtime has correct distinct `HoldReasons` variant count. I assume the
same situation exists for RuntimeFreezeReason.

It is not a critical problem, if we set `MaxHolds` with a sufficiently
large value, everything should be ok. However, in this case, the
integrity_test check becomes less useful.

**Situation for "any" runtime:**
- `HoldReason` enums from different pallets:
```rust
        /// from pallet_nis
        #[pallet::composite_enum]
	pub enum HoldReason {
		NftReceipt,
	}

        /// from pallet_preimage
        #[pallet::composite_enum]
	pub enum HoldReason {
		Preimage,
	}

        // from pallet_state-trie-migration
        #[pallet::composite_enum]
	pub enum HoldReason {
		SlashForContinueMigrate,
		SlashForMigrateCustomTop,
		SlashForMigrateCustomChild,
	}
```

- generated `RuntimeHoldReason` enum looks like:
```rust
pub enum RuntimeHoldReason {

    #[codec(index = 32u8)]
    Preimage(pallet_preimage::HoldReason),

    #[codec(index = 38u8)]
    Nis(pallet_nis::HoldReason),

    #[codec(index = 42u8)]
    StateTrieMigration(pallet_state_trie_migration::HoldReason),
}
```

- composite enum `RuntimeHoldReason` variant count is detected as `3`
- we set `type MaxHolds = ConstU32<3>`
- `pallet_balances::integrity_test` is ok with `3`(at least 3)

However, the real problem can occur in a live runtime where some
functionality might stop working. This is due to a total of 5 distinct
hold reasons (for pallets with multi-instance support, it is even more),
and not all of them can be used because of an incorrect `MaxHolds`,
which is deemed acceptable according to the `integrity_test`:
  ```
  // pseudo-code - if we try to call all of these:

T::Currency::hold(&pallet_nis::HoldReason::NftReceipt.into(),
&nft_owner, deposit)?;
T::Currency::hold(&pallet_preimage::HoldReason::Preimage.into(),
&nft_owner, deposit)?;

T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForContinueMigrate.into(),
&nft_owner, deposit)?;

  // With `type MaxHolds = ConstU32<3>` these two will fail

T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForMigrateCustomTop.into(),
&nft_owner, deposit)?;

T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForMigrateCustomChild.into(),
&nft_owner, deposit)?;
  ```  


## Solutions

A macro `#[pallet::*]` expansion is extended of `VariantCount`
implementation for the `#[pallet::composite_enum]` enum type. This
expansion generates the `VariantCount` implementation for pallets'
`HoldReason`, `FreezeReason`, `LockId`, and `SlashReason`. Enum variants
must be plain enum values without fields to ensure a deterministic
count.

The composite runtime enum, `RuntimeHoldReason` and
`RuntimeFreezeReason`, now sets `VariantCount::VARIANT_COUNT` as the sum
of pallets' enum `VariantCount::VARIANT_COUNT`:
```rust
#[frame_support::pallet(dev_mode)]
mod module_single_instance {

	#[pallet::composite_enum]
	pub enum HoldReason {
		ModuleSingleInstanceReason1,
		ModuleSingleInstanceReason2,
	}
...
}

#[frame_support::pallet(dev_mode)]
mod module_multi_instance {

	#[pallet::composite_enum]
	pub enum HoldReason<I: 'static = ()> {
		ModuleMultiInstanceReason1,
		ModuleMultiInstanceReason2,
		ModuleMultiInstanceReason3,
	}
...
}


impl self::sp_api_hidden_includes_construct_runtime::hidden_include::traits::VariantCount
    for RuntimeHoldReason
{
    const VARIANT_COUNT: u32 = 0
        + module_single_instance::HoldReason::VARIANT_COUNT
        + module_multi_instance::HoldReason::<module_multi_instance::Instance1>::VARIANT_COUNT
        + module_multi_instance::HoldReason::<module_multi_instance::Instance2>::VARIANT_COUNT
        + module_multi_instance::HoldReason::<module_multi_instance::Instance3>::VARIANT_COUNT;
}
```

In addition, `MaxHolds` is removed (as suggested
[here](https://github.com/paritytech/polkadot-sdk/pull/2657#discussion_r1443324573))
from `pallet_balances`, and its `Holds` are now bounded to
`RuntimeHoldReason::VARIANT_COUNT`. Therefore, there is no need to let
the runtime specify `MaxHolds`.


## For reviewers

Relevant changes can be found here:
- `substrate/frame/support/procedural/src/lib.rs` 
-  `substrate/frame/support/procedural/src/pallet/parse/composite.rs`
-  `substrate/frame/support/procedural/src/pallet/expand/composite.rs`
-
`substrate/frame/support/procedural/src/construct_runtime/expand/composite_helper.rs`
-
`substrate/frame/support/procedural/src/construct_runtime/expand/hold_reason.rs`
-
`substrate/frame/support/procedural/src/construct_runtime/expand/freeze_reason.rs`
- `substrate/frame/support/src/traits/misc.rs`

And the rest of the files is just about removed `MaxHolds` from
`pallet_balances`

## Next steps

Do the same for `MaxFreezes`
https://github.com/paritytech/polkadot-sdk/issues/2997.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>

Add `Balances::force_adjust_total_issuance` as preparation for fixing
https://github.com/polkadot-fellows/runtimes/issues/147.
Important changes in `substrate/frame/balances/src/lib.rs`.

TODO:
- [x] Update weights

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>