This PR updates the `validate_unsigned` hook for `frame_system` to allow
valid tasks to be submitted as unsigned transactions. It also updates
the task example to be able to submit such transactions via an off-chain
worker.

---------

Co-authored-by: Bastian Köcher <[email protected]>

Improve doc:

* the pallet macro is actually referring to 2 places, for the module and
for the struct placeholder but doesn't really clarify it (I should have
named the latter just `pallet_struct` or something but it is a bit late)

* The doc of `with_default` is a bit confusing too IMO.

CC @Kianenigma



---------

Co-authored-by: Liam Aharon <[email protected]>

Introduce `PalletId` as an additional seed parameter for pool's account
id derivation.

The PR also introduces the `pallet_asset_conversion_ops` pallet with a
call to migrate a given pool to thew new account. Additionally
`fungibles::lifetime::ResetTeam` and `fungible::lifetime::Refund`
traits, to facilitate the migration of pools.

---------

Co-authored-by: command-bot <>

Introduce types to define 1:1 balance conversion for different relative
asset ids/locations of native asset.

Examples:
native asset on Asset Hub presented as `VersionedLocatableAsset` type in
the context of Relay Chain is
```
{
  `location`: (0, Parachain(1000)),
  `asset_id`: (1, Here),
}
```
and it's balance should be converted 1:1 by implementations of
`ConversionToAssetBalance` trait.

---------

Co-authored-by: Branislav Kontur <[email protected]>

Current benchmarking macro returns a closure with the captured
benchmarked code.
This can cause issues when the benchmarked code has complex lifetime
requirements.

This PR updates the existing macro by injecting the recording parameter
and invoking the start / stop method around the benchmarked block
instead of returning a closure

One other added benefit is that you can write this kind of code now as
well:

```rust
let v;
#[block]
{ v = func.call(); }
dbg!(v); // or assert something on v
```


[Weights compare
link](https://weights.tasty.limo/compare?unit=weight&ignore_errors=true&threshold=10&method=asymptotic&repo=polkadot-sdk&old=pg/fix-weights&new=pg/bench_update&path_pattern=substrate/frame/**/src/weights.rs,polkadot/runtime/*/src/weights/**/*.rs,polkadot/bridges/modules/*/src/weights.rs,cumulus/**/weights/*.rs,cumulus/**/weights/xcm/*.rs,cumulus/**/src/weights.rs)

---------

Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@pari...>

Signed-off-by: hongkuang <[email protected]>

The runtime now can provide a number of predefined presets of
`RuntimeGenesisConfig` struct. This presets are intended to be used in
different deployments, e.g.: `local`, `staging`, etc, and should be
included into the corresponding chain-specs.

Having `GenesisConfig` presets in runtime allows to fully decouple node
from runtime types (the problem is described in #1984).

**Summary of changes:**
- The `GenesisBuilder` API was adjusted to enable this functionality
(and provide better naming - #150):
   ```rust
    fn preset_names() -> Vec<PresetId>;
fn get_preset(id: Option<PresetId>) -> Option<serde_json::Value>;
//`None` means default
    fn build_state(value: serde_json::Value);
    pub struct PresetId(Vec<u8>);
   ```

- **Breaking change**: Old `create_default_config` method was removed,
`build_config` was renamed to `build_state`. As a consequence a node
won't be able to interact with genesis config for older runtimes. The
cleanup was made for sake of API simplicity. Also IMO maintaining
compatibility with old API is not so crucial.
- Reference implementation was provided for `substrate-test-runtime` and
`rococo` runtimes. For rococo new
[`genesis_configs_presets`](https://github.com/paritytech/polkadot-sdk/blob/3b41d66b/polkadot/runtime/rococo/src/genesis_config_presets.rs#L530)
module was added and is used in `GenesisBuilder`
[_presets-related_](https://github.com/paritytech/polkadot-sdk/blob/3b41d66b/polkadot/runtime/rococo/src/lib.rs#L2462-L2485)
methods.

- The `chain-spec-builder` util was also improved and allows to
([_doc_](https://github.com/paritytech/polkadot-sdk/blob/3b41d66b/substrate/bin/utils/chain-spec-builder/src/lib.rs#L19)):
   - list presets provided by given runtime (`list-presets`),
- display preset or default config provided by the runtime
(`display-preset`),
   - build chain-spec using named preset (`create ... named-preset`),


- The `ChainSpecBuilder` is extended with
[`with_genesis_config_preset_name`](https://github.com/paritytech/polkadot-sdk/blob/3b41d66b/substrate/client/chain-spec/src/chain_spec.rs#L447)
method which allows to build chain-spec using named preset provided by
the runtime. Sample usage on the node side
[here](https://github.com/paritytech/polkadot-sdk/blob/2caffaae

/polkadot/node/service/src/chain_spec.rs#L404).

Implementation of #1984.
fixes: #150
part of: #25

---------

Co-authored-by: Sebastian Kunert <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

## Basic example showcasing a migration using the MBM framework

This PR has been built on top of
https://github.com/paritytech/polkadot-sdk/pull/1781 and adds two new
example crates to the `examples` pallet

### Changes Made:

Added the `pallet-example-mbm` crate: This crate provides a minimal
example of a pallet that uses MBM. It showcases a storage migration
where values are migrated from a `u32` to a `u64`.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

migrations: prevent accidentally using unversioned migrations instead of `VersionedMigration` (#3835)

closes #1324 

#### Problem
Currently, it is possible to accidentally use inner unversioned
migration instead of `VersionedMigration` since both implement
`OnRuntimeUpgrade`.

#### Solution

With this change, we make it clear that value of `Inner` is not intended
to be used directly. It is achieved by bounding `Inner` to new trait
`UncheckedOnRuntimeUpgrade`, which has the same interface (except
`unchecked_` prefix) as `OnRuntimeUpgrade`.

#### `try-runtime` functions

Since developers can implement `try-runtime` for `Inner` value in
`VersionedMigration` and have custom logic for it, I added the same
`try-runtime` functions to `UncheckedOnRuntimeUpgrade`. I looked for a
ways to not duplicate functions, but couldn't find anything that doesn't
significantly change the codebase. So I would appreciate If you have any
suggestions to improve this

cc @liamaharon

 @xlc 

polkadot address: 16FqwPZ8GRC5U5D4Fu7W33nA55ZXzXGWHwmbnE1eT6pxuqcT

---------

Co-authored-by: Liam Aharon <[email protected]>

Closes https://github.com/paritytech/polkadot-sdk-docs/issues/70

WIP PR for an overview of how to develop tokens in FRAME. 

- [x] Tokens in Substrate Ref Doc
  - High-level overview of the token-related logic in FRAME
- Improve docs with better explanation of how holds, freezes, ed, free
balance, etc, all work
- [x] Update `pallet_balances` docs
  - Clearly mark what is deprecated (currency)
- [x] Write fungible trait docs
- [x] Evaluate and if required update `pallet_assets`, `pallet_uniques`,
`pallet_nfts` docs
- [x] Absorb https://github.com/paritytech/polkadot-sdk/pull/2683/
- [x] Audit individual trait method docs, and improve if possible

Feel free to suggest additional TODOs for this PR in the comments

---------

Co-authored-by: Bill Laboon <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Sebastian Kunert <[email protected]>

This PR add `#[deprecated]` attribute to v1 and v2 of the schedule
trait. Proposed in this issue:
https://github.com/paritytech/polkadot-sdk/issues/3676

```rust
#[allow(deprecated)]
#[deprecated = "traits::schedule::v1 is deprecated. Please use v3 instead."]
pub mod v1 {
...
}

#[allow(deprecated)]
#[deprecated = "traits::schedule::v2 is deprecated. Please use v3 instead."]
pub mod v2 {
...
}
```

polkadot address: 19nSqFQorfF2HxD3oBzWM3oCh4SaCRKWt1yvmgaPYGCo71J

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

This PR adds a new extrinsic `Call::restore_ledger ` gated by
`StakingAdmin` origin that restores a corrupted staking ledger. This
extrinsic will be used to recover ledgers that were affected by the
issue discussed in
https://github.com/paritytech/polkadot-sdk/issues/3245.

The extrinsic will re-write the storage items associated with a stash
account provided as input parameter. The data used to reset the ledger
can be either i) fetched on-chain or ii) partially/totally set by the
input parameters of the call.

In order to use on-chain data to restore the staking locks, we need a
way to read the current lock in the balances pallet. This PR adds a
`InspectLockableCurrency` trait and implements it in the pallet
balances. An alternative would be to tightly couple staking with the
pallet balances but that's inelegant (an example of how it would look
like in [this
branch](https://github.com/paritytech/polkadot-sdk/tree/gpestana/ledger-badstate-clean_tightly)).

More details on the type of corruptions and corresponding fixes
https://hackmd.io/DLb5jEYWSmmvqXC9ae4yRg?view#/

We verified that the `Call::restore_ledger` does fix all current
corrupted ledgers in Polkadot and Kusama. You can verify it here
https://hackmd.io/v-XNrEoGRpe7APR-EZGhOA.

**Changes introduced**
- Adds `Call::restore_ledger ` extrinsic to recover a corrupted ledger;
- Adds trait `frame_support::traits::currency::InspectLockableCurrency`
to allow external pallets to read current locks given an account and
lock ID;
- Implements the `InspectLockableCurrency` in the pallet-balances.
- Adds staking locks try-runtime checks
(https://github.com/paritytech/polkadot-sdk/issues/3751)

**Todo**
- [x] benchmark `Call::restore_ledger`
- [x] throughout testing of all ledger recovering cases
- [x] consider adding the staking locks try-runtime checks to this PR
(https://github.com/paritytech/polkadot-sdk/issues/3751)
- [x] simulate restoring all ledgers
(https://hackmd.io/Dsa2tvhISNSs7zcqriTaxQ?view) in Polkadot and Kusama
using chopsticks -- https://hackmd.io/v-XNrEoGRpe7APR-EZGhOA

Related to https://github.com/paritytech/polkadot-sdk/issues/3245
Closes https://github.com/paritytech/polkadot-sdk/issues/3751

---------

Co-authored-by: command-bot <>

**Update:** Pushed additional changes based on the review comments.

**This pull request fixes various spelling mistakes in this
repository.**

Most of the changes are contained in the first **3** commits:

- `Fix spelling mistakes in comments and docs`

- `Fix spelling mistakes in test names`

- `Fix spelling mistakes in error messages, panic messages, logs and
tracing`

Other source code spelling mistakes are separated into individual
commits for easier reviewing:

- `Fix the spelling of 'authority'`

- `Fix the spelling of 'REASONABLE_HEADERS_IN_JUSTIFICATION_ANCESTRY'`

- `Fix the spelling of 'prev_enqueud_messages'`

- `Fix the spelling of 'endpoint'`

- `Fix the spelling of 'children'`

- `Fix the spelling of 'PenpalSiblingSovereignAccount'`

- `Fix the spelling of 'PenpalSudoAccount'`

- `Fix the spelling of 'insufficient'`

- `Fix the spelling of 'PalletXcmExtrinsicsBenchmark'`

- `Fix the spelling of 'subtracted'`

- `Fix the spelling of 'CandidatePendingAvailability'`

- `Fix the spelling of 'exclusive'`

- `Fix the spelling of 'until'`

- `Fix the spelling of 'discriminator'`

- `Fix the spelling of 'nonexistent'`

- `Fix the spelling of 'subsystem'`

- `Fix the spelling of 'indices'`

- `Fix the spelling of 'committed'`

- `Fix the spelling of 'topology'`

- `Fix the spelling of 'response'`

- `Fix the spelling of 'beneficiary'`

- `Fix the spelling of 'formatted'`

- `Fix the spelling of 'UNKNOWN_PROOF_REQUEST'`

- `Fix the spelling of 'succeeded'`

- `Fix the spelling of 'reopened'`

- `Fix the spelling of 'proposer'`

- `Fix the spelling of 'InstantiationNonce'`

- `Fix the spelling of 'depositor'`

- `Fix the spelling of 'expiration'`

- `Fix the spelling of 'phantom'`

- `Fix the spelling of 'AggregatedKeyValue'`

- `Fix the spelling of 'randomness'`

- `Fix the spelling of 'defendant'`

- `Fix the spelling of 'AquaticMammal'`

- `Fix the spelling of 'transactions'`

- `Fix the spelling of 'PassingTracingSubscriber'`

- `Fix the spelling of 'TxSignaturePayload'`

- `Fix the spelling of 'versioning'`

- `Fix the spelling of 'descendant'`

- `Fix the spelling of 'overridden'`

- `Fix the spelling of 'network'`

Let me know if this structure is adequate.

**Note:** The usage of the words `Merkle`, `Merkelize`, `Merklization`,
`Merkelization`, `Merkleization`, is somewhat inconsistent but I left it
as it is.

~~**Note:** In some places the term `Receival` is used to refer to
message reception, IMO `Reception` is the correct word here, but I left
it as it is.~~

~~**Note:** In some places the term `Overlayed` is used instead of the
more acceptable version `Overlaid` but I also left it as it is.~~

~~**Note:** In some places the term `Applyable` is used instead of the
correct version `Applicable` but I also left it as it is.~~

**Note:** Some usage of British vs American english e.g. `judgement` vs
`judgment`, `initialise` vs `initialize`, `optimise` vs `optimize` etc.
are both present in different places, but I suppose that's
understandable given the number of contributors.

~~**Note:** There is a spelling mistake in `.github/CODEOWNERS` but it
triggers errors in CI when I make changes to it, so I left it as it
is.~~

Step in https://github.com/paritytech/polkadot-sdk/issues/171

This PR removes `as [disambiguation_path]` syntax from `derive_impl`
usage across the polkadot-sdk as introduced in
https://github.com/paritytech/polkadot-sdk/pull/3505

Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280)" (#3665)

This PR reverts #2280 which introduced `TransactionExtension` to replace
`SignedExtension`.

As a result of the discussion
[here](https://github.com/paritytech/polkadot-sdk/pull/3623#issuecomment-1986789700),
the changes will be reverted for now with plans to reintroduce the
concept in the future.

---------

Signed-off-by: georgepisaltu <[email protected]>

Moved from https://github.com/paritytech/substrate/pull/14788

----

Fixes https://github.com/paritytech/polkadot-sdk/issues/232

This PR introduces outer-macro approach for `construct_runtime` as
discussed in the linked issue. It looks like the following:
```rust
#[frame_support::runtime]
mod runtime {
	#[runtime::runtime]
        #[runtime::derive(
		RuntimeCall,
		RuntimeEvent,
		RuntimeError,
		RuntimeOrigin,
		RuntimeFreezeReason,
		RuntimeHoldReason,
		RuntimeSlashReason,
		RuntimeLockId,
                RuntimeTask,
	)]
	pub struct Runtime;

	#[runtime::pallet_index(0)]
	pub type System = frame_system;

	#[runtime::pallet_index(1)]
	pub type Timestamp = pallet_timestamp;

	#[runtime::pallet_index(2)]
	pub type Aura = pallet_aura;

	#[runtime::pallet_index(3)]
	pub type Grandpa = pallet_grandpa;

	#[runtime::pallet_index(4)]
	pub type Balances = pallet_balances;

	#[runtime::pallet_index(5)]
	pub type TransactionPayment = pallet_transaction_payment;

	#[runtime::pallet_index(6)]
	pub type Sudo = pallet_sudo;

	// Include the custom logic from the pallet-template in the runtime.
	#[runtime::pallet_index(7)]
	pub type TemplateModule = pallet_template;
}
```

## Features
- `#[runtime::runtime]` attached to a struct defines the main runtime
- `#[runtime::derive]` attached to this struct defines the types
generated by runtime
- `#[runtime::pallet_index]` must be attached to a pallet to define its
index
- `#[runtime::disable_call]` can be optionally attached to a pallet to
disable its calls
- `#[runtime::disable_unsigned]` can be optionally attached to a pallet
to disable unsigned calls
- A pallet instance can be defined as `TemplateModule:
pallet_template<Instance>`
- An optional attribute can be defined as
`#[frame_support::runtime(legacy_ordering)]` to ensure that the order of
hooks is same as the order of pallets (and not based on the
pallet_index). This is to support legacy runtimes and should be avoided
for new ones.

## Todo
- [x] Update the latest syntax in kitchensink and tests
- [x] Update UI tests
- [x] Docs

## Extension
- Abstract away the Executive similar to
https://github.com/paritytech/substrate/pull/14742
- Optionally avoid the need to specify all runtime types (TBD)

---------

Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: Nikhil Gupta <>

# Description

*Removes the deprecated `trait Store` feature from the code base*

Fixes #222

---------

Co-authored-by: Dónal Murray <[email protected]>

substrate.io deprecation companion:
https://github.com/substrate-developer-hub/substrate-docs/pull/2139
pba-content companion:
https://github.com/Polkadot-Blockchain-Academy/pba-content/pull/978

partially inspired by:
https://github.com/paritytech/polkadot-sdk/issues/3535

---------

Co-authored-by: Ankan <[email protected]>

Changes:
- `QueueFootprint` gets a new field; `ready_pages` that contains the
non-overweight and not yet processed pages.
- `XCMP` queue pallet is change to use the `ready_pages` instead of
`pages` to calculate the channel suspension thresholds.

This should give the XCMP queue pallet a more correct view of when to
suspend channels.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Closes #2160

First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415)

Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for the latter. (Ultimately, the latter will be phased out as
a type of transaction, and Bare will only be used for Inherents.)

Types of extrinsic are now therefore:
- Bare (no hardcoded signature, no Extra data; used to be known as
"Unsigned")
- Bare transactions (deprecated): Gossiped, validated with
`ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
`TransactionExtension` (deprecated).
  - Inherents: Not gossiped, validated with `ProvideInherent`.
- Extended (Extra data): Gossiped, validated via `TransactionExtension`.
  - Signed transactions (with a hardcoded signature).
  - General transactions (without a hardcoded signature).

`TransactionExtension` differs from `SignedExtension` because:
- A signature on the underlying transaction may validly not be present.
- It may alter the origin during validation.
- `pre_dispatch` is renamed to `prepare` and need not contain the checks
present in `validate`.
- `validate` and `prepare` is passed an `Origin` rather than a
`AccountId`.
- `validate` may pass arbitrary information into `prepare` via a new
user-specifiable type `Val`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`. It is encoded *for the entire transaction* and
passed in to each extension as a new argument to `validate`. This
facilitates the ability of extensions to acts as underlying crypto.

There is a new `DispatchTransaction` trait which contains only default
function impls and is impl'ed for any `TransactionExtension` impler. It
provides several utility functions which reduce some of the tedium from
using `TransactionExtension` (indeed, none of its regular functions
should now need to be called directly).

Three transaction version discriminator ("versions") are now
permissible:
- 0b000000100: Bare (used to be called "Unsigned"): contains Signature
or Extra (extension data). After bare transactions are no longer
supported, this will strictly identify an Inherents only.
- 0b100000100: Old-school "Signed" Transaction: contains Signature and
Extra (extension data).
- 0b010000100: New-school "General" Transaction: contains Extra
(extension data), but no Signature.

For the New-school General Transaction, it becomes trivial for authors
to publish extensions to the mechanism for authorizing an Origin, e.g.
through new kinds of key-signing schemes, ZK proofs, pallet state,
mutations over pre-authenticated origins or any combination of the
above.

## Code Migration

### NOW: Getting it to build

Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
accompanied by renaming your aggregate type in line with the new
terminology. E.g. Before:

```rust
/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
	/* snip */
	MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
```

After:

```rust
/// The extension to the basic transaction logic.
pub type TxExtension = (
	/* snip */
	AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
```

You'll also need to alter any transaction building logic to add a
`.into()` to make the conversion happen. E.g. Before:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let extra: SignedExtra = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */),
	);
	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		extra,
	)
}
```

After:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let tx_ext: TxExtension = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */).into(),
	);
	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		tx_ext,
	)
}
```

### SOON: Migrating to `TransactionExtension`

Most `SignedExtension`s can be trivially converted to become a
`TransactionExtension`. There are a few things to know.

- Instead of a single trait like `SignedExtension`, you should now
implement two traits individually: `TransactionExtensionBase` and
`TransactionExtension`.
- Weights are now a thing and must be provided via the new function `fn
weight`.

#### `TransactionExtensionBase`

This trait takes care of anything which is not dependent on types
specific to your runtime, most notably `Call`.

- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`.
- Weight must be returned by implementing the `weight` function. If your
extension is associated with a pallet, you'll probably want to do this
via the pallet's existing benchmarking infrastructure.

#### `TransactionExtension`

Generally:
- `pre_dispatch` is now `prepare` and you *should not reexecute the
`validate` functionality in there*!
- You don't get an account ID any more; you get an origin instead. If
you need to presume an account ID, then you can use the trait function
`AsSystemOriginSigner::as_system_origin_signer`.
- You get an additional ticket, similar to `Pre`, called `Val`. This
defines data which is passed from `validate` into `prepare`. This is
important since you should not be duplicating logic from `validate` to
`prepare`, you need a way of passing your working from the former into
the latter. This is it.
- This trait takes two type parameters: `Call` and `Context`. `Call` is
the runtime call type which used to be an associated type; you can just
move it to become a type parameter for your trait impl. `Context` is not
currently used and you can safely implement over it as an unbounded
type.
- There's no `AccountId` associated type any more. Just remove it.

Regarding `validate`:
- You get three new parameters in `validate`; all can be ignored when
migrating from `SignedExtension`.
- `validate` returns a tuple on success; the second item in the tuple is
the new ticket type `Self::Val` which gets passed in to `prepare`. If
you use any information extracted during `validate` (off-chain and
on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
pass it through with this. For the tuple's last item, just return the
`origin` argument.

Regarding `prepare`:
- This is renamed from `pre_dispatch`, but there is one change:
- FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
`validate`!!
- (This is different to `SignedExtension` which was required to run the
same checks in `pre_dispatch` as in `validate`.)

Regarding `post_dispatch`:
- Since there are no unsigned transactions handled by
`TransactionExtension`, `Pre` is always defined, so the first parameter
is `Self::Pre` rather than `Option<Self::Pre>`.

If you make use of `SignedExtension::validate_unsigned` or
`SignedExtension::pre_dispatch_unsigned`, then:
- Just use the regular versions of these functions instead.
- Have your logic execute in the case that the `origin` is `None`.
- Ensure your transaction creation logic creates a General Transaction
rather than a Bare Transaction; this means having to include all
`TransactionExtension`s' data.
- `ValidateUnsigned` can still be used (for now) if you need to be able
to construct transactions which contain none of the extension data,
however these will be phased out in stage 2 of the Transactions Horizon,
so you should consider moving to an extension-centric design.

## TODO

- [x] Introduce `CheckSignature` impl of `TransactionExtension` to
ensure it's possible to have crypto be done wholly in a
`TransactionExtension`.
- [x] Deprecate `SignedExtension` and move all uses in codebase to
`TransactionExtension`.
  - [x] `ChargeTransactionPayment`
  - [x] `DummyExtension`
  - [x] `ChargeAssetTxPayment` (asset-tx-payment)
  - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment)
  - [x] `CheckWeight`
  - [x] `CheckTxVersion`
  - [x] `CheckSpecVersion`
  - [x] `CheckNonce`
  - [x] `CheckNonZeroSender`
  - [x] `CheckMortality`
  - [x] `CheckGenesis`
  - [x] `CheckOnlySudoAccount`
  - [x] `WatchDummy`
  - [x] `PrevalidateAttests`
  - [x] `GenericSignedExtension`
  - [x] `SignedExtension` (chain-polkadot-bulletin)
  - [x] `RefundSignedExtensionAdapter`
- [x] Implement `fn weight` across the board.
- [ ] Go through all pre-existing extensions which assume an account
signer and explicitly handle the possibility of another kind of origin.
- [x] `CheckNonce` should probably succeed in the case of a non-account
origin.
- [x] `CheckNonZeroSender` should succeed in the case of a non-account
origin.
- [x] `ChargeTransactionPayment` and family should fail in the case of a
non-account origin.
  - [ ] 
- [x] Fix any broken tests.

---------

Signed-off-by: georgepisaltu <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Signed-off-by: Alexandru Gheorghe <[email protected]>
Signed-off-by: Andrei Sandu <[email protected]>
Co-authored-by: Nikhil Gupta <[email protected]>
Co-authored-by: georgepisaltu <[email protected]>
Co-authored-by: Chevdor <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Maciej <[email protected]>
Co-authored-by: Javier Viola <[email protected]>
Co-authored-by: Marcin S. <[email protected]>
Co-authored-by: Tsvetomir Dimitrov <[email protected]>
Co-authored-by: Javier Bullrich <[email protected]>
Co-authored-by: Koute <[email protected]>
Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: Vladimir Istyufeev <[email protected]>
Co-authored-by: Ross Bulat <[email protected]>
Co-authored-by: Gonçalo Pestana <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: s0me0ne-unkn0wn <[email protected]>
Co-authored-by: ordian <[email protected]>
Co-authored-by: Sebastian Kunert <[email protected]>
Co-authored-by: Aaro Altonen <[email protected]>
Co-authored-by: Dmitry Markin <[email protected]>
Co-authored-by: Alexandru Vasile <[email protected]>
Co-authored-by: Alexander Samusev <[email protected]>
Co-authored-by: Julian Eager <[email protected]>
Co-authored-by: Michal Kucharczyk <[email protected]>
Co-authored-by: Davide Galassi <[email protected]>
Co-authored-by: Dónal Murray <[email protected]>
Co-authored-by: yjh <[email protected]>
Co-authored-by: Tom Mi <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Will | Paradox | ParaNodes.io <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Joshy Orndorff <[email protected]>
Co-authored-by: Joshy Orndorff <[email protected]>
Co-authored-by: PG Herveou <[email protected]>
Co-authored-by: Alexander Theißen <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Juan Girini <[email protected]>
Co-authored-by: bader y <[email protected]>
Co-authored-by: James Wilson <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: asynchronous rob <[email protected]>
Co-authored-by: Parth <[email protected]>
Co-authored-by: Andrew Jones <[email protected]>
Co-authored-by: Jonathan Udd <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: Egor_P <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: Evgeny Snitko <[email protected]>
Co-authored-by: Just van Stam <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: gupnik <[email protected]>
Co-authored-by: dzmitry-lahoda <[email protected]>
Co-authored-by: zhiqiangxu <[email protected]>
Co-authored-by: Nazar Mokrynskyi <[email protected]>
Co-authored-by: Anwesh <[email protected]>
Co-authored-by: cheme <[email protected]>
Co-authored-by: Sam Johnson <[email protected]>
Co-authored-by: kianenigma <[email protected]>
Co-authored-by: Jegor Sidorenko <[email protected]>
Co-authored-by: Muharem <[email protected]>
Co-authored-by: joepetrowski <[email protected]>
Co-authored-by: Alexandru Gheorghe <[email protected]>
Co-authored-by: Gabriel Facco de Arruda <[email protected]>
Co-authored-by: Squirrel <[email protected]>
Co-authored-by: Andrei Sandu <[email protected]>
Co-authored-by: georgepisaltu <[email protected]>
Co-authored-by: command-bot <>

Closes https://github.com/paritytech/polkadot-sdk-docs/issues/35

- Moves pallet proc macro docs to `frame_support`
- Adds missing docs
- Revise revise existing docs, adding compiling doctests where
appropriate

---------

Co-authored-by: command-bot <>
Co-authored-by: Kian Paimani <[email protected]>

This MR is the merge of
https://github.com/paritytech/substrate/pull/14414 and
https://github.com/paritytech/substrate/pull/14275. It implements
[RFC#13](https://github.com/polkadot-fellows/RFCs/pull/13), closes
https://github.com/paritytech/polkadot-sdk/issues/198.

----- 

This Merge request introduces three major topicals:

1. Multi-Block-Migrations
1. New pallet `poll` hook for periodic service work
1. Replacement hooks for `on_initialize` and `on_finalize` in cases
where `poll` cannot be used

and some more general changes to FRAME.  
The changes for each topical span over multiple crates. They are listed
in topical order below.

# 1.) Multi-Block-Migrations

Multi-Block-Migrations are facilitated by creating `pallet_migrations`
and configuring `System::Config::MultiBlockMigrator` to point to it.
Executive picks this up and triggers one step of the migrations pallet
per block.
The chain is in lockdown mode for as long as an MBM is ongoing.
Executive does this by polling `MultiBlockMigrator::ongoing` and not
allowing any transaction in a block, if true.

A MBM is defined through trait `SteppedMigration`. A condensed version
looks like this:
```rust
/// A migration that can proceed in multiple steps.
pub trait SteppedMigration {
	type Cursor: FullCodec + MaxEncodedLen;
	type Identifier: FullCodec + MaxEncodedLen;

	fn id() -> Self::Identifier;

	fn max_steps() -> Option<u32>;

	fn step(
		cursor: Option<Self::Cursor>,
		meter: &mut WeightMeter,
	) -> Result<Option<Self::Cursor>, SteppedMigrationError>;
}
```

`pallet_migrations` can be configured with an aggregated tuple of these
migrations. It then starts to migrate them one-by-one on the next
runtime upgrade.
Two things are important here:
- 1. Doing another runtime upgrade while MBMs are ongoing is not a good
idea and can lead to messed up state.
- 2. **Pallet Migrations MUST BE CONFIGURED IN `System::Config`,
otherwise it is not used.**

The pallet supports an `UpgradeStatusHandler` that can be used to notify
external logic of upgrade start/finish (for example to pause XCM
dispatch).

Error recovery is very limited in the case that a migration errors or
times out (exceeds its `max_steps`). Currently the runtime dev can
decide in `FailedMigrationHandler::failed` how to handle this. One
follow-up would be to pair this with the `SafeMode` pallet and enact
safe mode when an upgrade fails, to allow governance to rescue the
chain. This is currently not possible, since governance is not
`Mandatory`.

## Runtime API

- `Core`: `initialize_block` now returns `ExtrinsicInclusionMode` to
inform the Block Author whether they can push transactions.

### Integration

Add it to your runtime implementation of `Core` and `BlockBuilder`:
```patch
diff --git a/runtime/src/lib.rs b/runtime/src/lib.rs
@@ impl_runtime_apis! {
	impl sp_block_builder::Core<Block> for Runtime {
-		fn initialize_block(header: &<Block as BlockT>::Header) {
+		fn initialize_block(header: &<Block as BlockT>::Header) -> RuntimeExecutiveMode {
			Executive::initialize_block(header)
		}

		...
	}
```

# 2.) `poll` hook

A new pallet hook is introduced: `poll`. `Poll` is intended to replace
mostly all usage of `on_initialize`.
The reason for this is that any code that can be called from
`on_initialize` cannot be migrated through an MBM. Currently there is no
way to statically check this; the implication is to use `on_initialize`
as rarely as possible.
Failing to do so can result in broken storage invariants.

The implementation of the poll hook depends on the `Runtime API` changes
that are explained above.

# 3.) Hard-Deadline callbacks

Three new callbacks are introduced and configured on `System::Config`:
`PreInherents`, `PostInherents` and `PostTransactions`.
These hooks are meant as replacement for `on_initialize` and
`on_finalize` in cases where the code that runs cannot be moved to
`poll`.
The reason for this is to make the usage of HD-code (hard deadline) more
explicit - again to prevent broken invariants by MBMs.

# 4.) FRAME (general changes)

## `frame_system` pallet

A new memorize storage item `InherentsApplied` is added. It is used by
executive to track whether inherents have already been applied.
Executive and can then execute the MBMs directly between inherents and
transactions.

The `Config` gets five new items:
- `SingleBlockMigrations` this is the new way of configuring migrations
that run in a single block. Previously they were defined as last generic
argument of `Executive`. This shift is brings all central configuration
about migrations closer into view of the developer (migrations that are
configured in `Executive` will still work for now but is deprecated).
- `MultiBlockMigrator` this can be configured to an engine that drives
MBMs. One example would be the `pallet_migrations`. Note that this is
only the engine; the exact MBMs are injected into the engine.
- `PreInherents` a callback that executes after `on_initialize` but
before inherents.
- `PostInherents` a callback that executes after all inherents ran
(including MBMs and `poll`).
- `PostTransactions` in symmetry to `PreInherents`, this one is called
before `on_finalize` but after all transactions.

A sane default is to set all of these to `()`. Example diff suitable for
any chain:
```patch
@@ impl frame_system::Config for Test {
 	type MaxConsumers = ConstU32<16>;
+	type SingleBlockMigrations = ();
+	type MultiBlockMigrator = ();
+	type PreInherents = ();
+	type PostInherents = ();
+	type PostTransactions = ();
 }
```

An overview of how the block execution now looks like is here. The same
graph is also in the rust doc.

<details><summary>Block Execution Flow</summary>
<p>

![Screenshot 2023-12-04 at 19 11
29](https://github.com/paritytech/polkadot-sdk/assets/10380170/e88a80c4-ef11-4faa-8df5-8b33a724c054)

</p>
</details> 

## Inherent Order

Moved to https://github.com/paritytech/polkadot-sdk/pull/2154

---------------


## TODO

- [ ] Check that `try-runtime` still works
- [ ] Ensure backwards compatibility with old Runtime APIs
- [x] Consume weight correctly
- [x] Cleanup

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Juan Girini <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

- deprecation companion:
https://github.com/substrate-developer-hub/substrate-docs/pull/2136
- inspired by
https://substrate.stackexchange.com/questions/11058/how-can-i-create-ocw-that-wont-activates-every-block-but-will-activates-only-w/11060#11060

---------

Co-authored-by: Sergej Sakac <[email protected]>

Closes https://github.com/paritytech/polkadot-sdk-docs/issues/55

- Changes 'current storage version' terminology to less ambiguous
'in-code storage version' (suggestion by @ggwpez

)
- Adds a new example pallet `pallet-example-single-block-migrations`
- Adds a new reference doc to replace
https://docs.substrate.io/maintain/runtime-upgrades/ (temporarily living
in the pallet while we wait for developer hub PR to merge)
- Adds documentation for the `storage_alias` macro
- Improves `trait Hooks` docs 
- Improves `trait GetStorageVersion` docs
- Update the suggested patterns for using `VersionedMigration`, so that
version unchecked migrations are never exported
- Prevents accidental usage of version unchecked migrations in runtimes

https://github.com/paritytech/substrate/pull/14421#discussion_r1255467895
- Unversioned migration code is kept inside `mod version_unchecked`,
versioned code is kept in `pub mod versioned`
- It is necessary to use modules to limit visibility because the inner
migration must be `pub`. See
https://github.com/rust-lang/rust/issues/30905 and

https://internals.rust-lang.org/t/lang-team-minutes-private-in-public-rules/4504/40
for more.

### todo

- [x] move to reference docs to proper place within sdk-docs (now that
https://github.com/paritytech/polkadot-sdk/pull/2102 is merged)
- [x] prdoc

---------

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Juan <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: gupnik <[email protected]>

Introduce storage attr macro `#[disable_try_decode_storage]` and set it on `System::Events` and `ParachainSystem::HostConfiguration` (#3454)

Closes https://github.com/paritytech/polkadot-sdk/issues/2560

Allows marking storage items with `#[disable_try_decode_storage]`, and
uses it with `System::Events`.

Question: what's the recommended way to write a test for this? I
couldn't find a test for similar existing macro `#[whitelist_storage]`.

Does the following: 

- Add a reference doc page named `frame_runtime_types`, which explains
what types like `RuntimeOrigin`, `RuntimeCall` etc are.
- On top of it, it adds a reference doc page called `frame_origin` which
explains a few important patterns that we use around origins
- And finally brushes up `#[frame::origin]` docs. 
- Updates the theme, sidebar and favicon to look like: 

<img width="1728" alt="Screenshot 2024-02-20 at 12 16 00"
src="https://github.com/paritytech/polkadot-sdk/assets/5588131/6d60a16b-2081-411b-8869-43b91920cca9">


All of this was inspired by
https://substrate.stackexchange.com/questions/10992/how-do-you-find-the-public-key-for-the-medium-spender-track-origin/10993

closes https://github.com/paritytech/polkadot-sdk-docs/issues/45
closes https://github.com/paritytech/polkadot-sdk-docs/issues/43
contributes / overlaps with
https://github.com/paritytech/polkadot-sdk/pull/2638 cc @liamaharon


deprecation companion:
https://github.com/substrate-developer-hub/substrate-docs/pull/2131
pba-content companion:
https://github.com/Polkadot-Blockchain-Academy/pba-content/pull/977

---------

Co-authored-by: Radha <[email protected]>
Co-authored-by: Sebastian Kunert <[email protected]>
Co-authored-by: Gonçalo Pestana <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

Closes https://github.com/paritytech/polkadot-sdk/issues/2713

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: André Silva <[email protected]>

Closes #169  

Fork of the `orml-parameters-pallet` as introduced by
https://github.com/open-web3-stack/open-runtime-module-library/pull/927
(cc @xlc)
It greatly changes how the macros work, but keeps the pallet the same.
The downside of my code is now that it does only support constant keys
in the form of types, not value-bearing keys.
I think this is an acceptable trade off, give that it can be used by
*any* pallet without any changes.

The pallet allows to dynamically set parameters that can be used in
pallet configs while also restricting the updating on a per-key basis.
The rust-docs contains a complete example.

Changes:
- Add `parameters-pallet`
- Use in the kitchensink as demonstration
- Add experimental attribute to define dynamic params in the runtime.
- Adding a bunch of traits to `frame_support::traits::dynamic_params`
that can be re-used by the ORML macros

## Example

First to define the parameters in the runtime file. The syntax is very
explicit about the codec index and errors if there is no.
```rust
#[dynamic_params(RuntimeParameters, pallet_parameters::Parameters::<Runtime>))]
pub mod dynamic_params {
	use super::*;

	#[dynamic_pallet_params]
	#[codec(index = 0)]
	pub mod storage {
		/// Configures the base deposit of storing some data.
		#[codec(index = 0)]
		pub static BaseDeposit: Balance = 1 * DOLLARS;

		/// Configures the per-byte deposit of storing some data.
		#[codec(index = 1)]
		pub static ByteDeposit: Balance = 1 * CENTS;
	}

	#[dynamic_pallet_params]
	#[codec(index = 1)]
	pub mod contracts {
		#[codec(index = 0)]
		pub static DepositPerItem: Balance = deposit(1, 0);

		#[codec(index = 1)]
		pub static DepositPerByte: Balance = deposit(0, 1);
	}
}
```

Then the pallet is configured with the aggregate:  
```rust
impl pallet_parameters::Config for Runtime {
	type AggregratedKeyValue = RuntimeParameters;
	type AdminOrigin = EnsureRootWithSuccess<AccountId, ConstBool<true>>;
	...
}
```

And then the parameters can be used in a pallet config:
```rust
impl pallet_preimage::Config for Runtime {
	type DepositBase = dynamic_params::storage::DepositBase;
}
```

A custom origin an be defined like this:  
```rust
pub struct DynamicParametersManagerOrigin;

impl EnsureOriginWithArg<RuntimeOrigin, RuntimeParametersKey> for DynamicParametersManagerOrigin {
	type Success = ();

	fn try_origin(
		origin: RuntimeOrigin,
		key: &RuntimeParametersKey,
	) -> Result<Self::Success, RuntimeOrigin> {
		match key {
			RuntimeParametersKey::Storage(_) => {
				frame_system::ensure_root(origin.clone()).map_err(|_| origin)?;
				return Ok(())
			},
			RuntimeParametersKey::Contract(_) => {
				frame_system::ensure_root(origin.clone()).map_err(|_| origin)?;
				return Ok(())
			},
		}
	}

	#[cfg(feature = "runtime-benchmarks")]
	fn try_successful_origin(_key: &RuntimeParametersKey) -> Result<RuntimeOrigin, ()> {
		Ok(RuntimeOrigin::Root)
	}
}
```

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Nikhil Gupta <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: command-bot <>

First in a series of PRs that reduces our use of sp-std with a view to
deprecating it.

This is just looking at /substrate and moving some of the references
from `sp-std` to `core`.
These particular changes should be uncontroversial.

Where macros are used `::core` should be used to remove any ambiguity.

part of https://github.com/paritytech/polkadot-sdk/issues/2101

# Description

While methods' names on [`VoteTally`][1] trait might be self-explanatory
at first sight, the distinction between `support` and `approval` can be
a bit ambiguous for some readers. This PR aims to clarify the
distinction and inform about the expected values for every not yet
documented method on this trait.

# Checklist

- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works (if applicable)

[1]:
https://docs.rs/frame-support/latest/frame_support/traits/trait.VoteTally.html

---------

Co-authored-by: Bastian Köcher <[email protected]>

Fixup for https://github.com/paritytech/polkadot-sdk/pull/2587 to make
the `core-fellowship` crate work with swapped members.

Adds a `MemberSwappedHandler` to the `ranked-collective` pallet that are
implemented by `core-fellowship+salary`.
There is are exhaustive tests
[here](https://github.com/paritytech/polkadot-sdk/blob/72aa7ac1/substrate/frame/core-fellowship/src/tests/integration.rs#L338)
and
[here](https://github.com/paritytech/polkadot-sdk/blob/ab3cdb05

/substrate/frame/salary/src/tests/integration.rs#L224)
to check that adding member `1` is equivalent to adding member `0` and
then swapping.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

[frame] `#[pallet::composite_enum]` improved variant count handling + removed `pallet_balances`'s `MaxHolds` config (#2657)

I started this investigation/issue based on @liamaharon

 question
[here](https://github.com/paritytech/polkadot-sdk/pull/1801#discussion_r1410452499).

## Problem

The `pallet_balances` integrity test should correctly detect that the
runtime has correct distinct `HoldReasons` variant count. I assume the
same situation exists for RuntimeFreezeReason.

It is not a critical problem, if we set `MaxHolds` with a sufficiently
large value, everything should be ok. However, in this case, the
integrity_test check becomes less useful.

**Situation for "any" runtime:**
- `HoldReason` enums from different pallets:
```rust
        /// from pallet_nis
        #[pallet::composite_enum]
	pub enum HoldReason {
		NftReceipt,
	}

        /// from pallet_preimage
        #[pallet::composite_enum]
	pub enum HoldReason {
		Preimage,
	}

        // from pallet_state-trie-migration
        #[pallet::composite_enum]
	pub enum HoldReason {
		SlashForContinueMigrate,
		SlashForMigrateCustomTop,
		SlashForMigrateCustomChild,
	}
```

- generated `RuntimeHoldReason` enum looks like:
```rust
pub enum RuntimeHoldReason {

    #[codec(index = 32u8)]
    Preimage(pallet_preimage::HoldReason),

    #[codec(index = 38u8)]
    Nis(pallet_nis::HoldReason),

    #[codec(index = 42u8)]
    StateTrieMigration(pallet_state_trie_migration::HoldReason),
}
```

- composite enum `RuntimeHoldReason` variant count is detected as `3`
- we set `type MaxHolds = ConstU32<3>`
- `pallet_balances::integrity_test` is ok with `3`(at least 3)

However, the real problem can occur in a live runtime where some
functionality might stop working. This is due to a total of 5 distinct
hold reasons (for pallets with multi-instance support, it is even more),
and not all of them can be used because of an incorrect `MaxHolds`,
which is deemed acceptable according to the `integrity_test`:
  ```
  // pseudo-code - if we try to call all of these:

T::Currency::hold(&pallet_nis::HoldReason::NftReceipt.into(),
&nft_owner, deposit)?;
T::Currency::hold(&pallet_preimage::HoldReason::Preimage.into(),
&nft_owner, deposit)?;

T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForContinueMigrate.into(),
&nft_owner, deposit)?;

  // With `type MaxHolds = ConstU32<3>` these two will fail

T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForMigrateCustomTop.into(),
&nft_owner, deposit)?;

T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForMigrateCustomChild.into(),
&nft_owner, deposit)?;
  ```  


## Solutions

A macro `#[pallet::*]` expansion is extended of `VariantCount`
implementation for the `#[pallet::composite_enum]` enum type. This
expansion generates the `VariantCount` implementation for pallets'
`HoldReason`, `FreezeReason`, `LockId`, and `SlashReason`. Enum variants
must be plain enum values without fields to ensure a deterministic
count.

The composite runtime enum, `RuntimeHoldReason` and
`RuntimeFreezeReason`, now sets `VariantCount::VARIANT_COUNT` as the sum
of pallets' enum `VariantCount::VARIANT_COUNT`:
```rust
#[frame_support::pallet(dev_mode)]
mod module_single_instance {

	#[pallet::composite_enum]
	pub enum HoldReason {
		ModuleSingleInstanceReason1,
		ModuleSingleInstanceReason2,
	}
...
}

#[frame_support::pallet(dev_mode)]
mod module_multi_instance {

	#[pallet::composite_enum]
	pub enum HoldReason<I: 'static = ()> {
		ModuleMultiInstanceReason1,
		ModuleMultiInstanceReason2,
		ModuleMultiInstanceReason3,
	}
...
}


impl self::sp_api_hidden_includes_construct_runtime::hidden_include::traits::VariantCount
    for RuntimeHoldReason
{
    const VARIANT_COUNT: u32 = 0
        + module_single_instance::HoldReason::VARIANT_COUNT
        + module_multi_instance::HoldReason::<module_multi_instance::Instance1>::VARIANT_COUNT
        + module_multi_instance::HoldReason::<module_multi_instance::Instance2>::VARIANT_COUNT
        + module_multi_instance::HoldReason::<module_multi_instance::Instance3>::VARIANT_COUNT;
}
```

In addition, `MaxHolds` is removed (as suggested
[here](https://github.com/paritytech/polkadot-sdk/pull/2657#discussion_r1443324573))
from `pallet_balances`, and its `Holds` are now bounded to
`RuntimeHoldReason::VARIANT_COUNT`. Therefore, there is no need to let
the runtime specify `MaxHolds`.


## For reviewers

Relevant changes can be found here:
- `substrate/frame/support/procedural/src/lib.rs` 
-  `substrate/frame/support/procedural/src/pallet/parse/composite.rs`
-  `substrate/frame/support/procedural/src/pallet/expand/composite.rs`
-
`substrate/frame/support/procedural/src/construct_runtime/expand/composite_helper.rs`
-
`substrate/frame/support/procedural/src/construct_runtime/expand/hold_reason.rs`
-
`substrate/frame/support/procedural/src/construct_runtime/expand/freeze_reason.rs`
- `substrate/frame/support/src/traits/misc.rs`

And the rest of the files is just about removed `MaxHolds` from
`pallet_balances`

## Next steps

Do the same for `MaxFreezes`
https://github.com/paritytech/polkadot-sdk/issues/2997.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Dónal Murray <[email protected]>
Co-authored-by: gupnik <[email protected]>

This PR is related to
[this](https://github.com/paritytech/polkadot-sdk/issues/154) issue.

The idea is to add `OrdNoBound` and `PartialOrdNoBound` macros to the
substrate `*NoBound` macros.

closes #2198 
✄
-----------------------------------------------------------------------------

Thank you for your Pull Request! 🙏

 Please make sure it follows the
contribution guidelines outlined in
[this
document](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CONTRIBUTING.md)
and fill
out the sections below. Once you're ready to submit your PR for review,
please
delete this section and leave only the text under the "Description"
heading.

# Description

*Please include a summary of the changes and the related issue. Please
also include relevant motivation and context,
including:*

- What does this PR do?
- Why are these changes needed?
- How were these changes implemented and what do they affect?

*Use [Github semantic

linking](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)
to address any open issues this PR relates to or closes.*

Fixes # (issue number, *if applicable*)

Closes # (issue number, *if applicable*)

# Checklist

- [ ] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

You can remove the "Checklist" section once all have been checked. Thank
you for your contribution!

✄
-----------------------------------------------------------------------------

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

This is a rather big change in jsonrpsee, the major things in this bump
are:
- Server backpressure (the subscription impls are modified to deal with
that)
- Allow custom error types / return types (remove jsonrpsee::core::Error
and jsonrpee::core::CallError)
- Bug fixes (graceful shutdown in particular not used by substrate
anyway)
   - Less dependencies for the clients in particular
   - Return type requires Clone in method call responses
   - Moved to tokio channels
   - Async subscription API (not used in this PR)

Major changes in this PR:
- The subscriptions are now bounded and if subscription can't keep up
with the server it is dropped
- CLI: add parameter to configure the jsonrpc server bounded message
buffer (default is 64)
- Add our own subscription helper to deal with the unbounded streams in
substrate

The most important things in this PR to review is the added helpers
functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
much chore.

Regarding the "bounded buffer limit" it may cause the server to handle
the JSON-RPC calls
slower than before.

The message size limit is bounded by "--rpc-response-size" thus "by
default 10MB * 64 = 640MB"
but the subscription message size is not covered by this limit and could
be capped as well.

Hopefully the last release prior to 1.0, sorry in advance for a big PR

Previous attempt: https://github.com/paritytech/substrate/pull/13992

Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
https://github.com/paritytech/polkadot-sdk/issues/627

Step towards https://github.com/paritytech/polkadot-sdk/issues/1975

As reported
https://github.com/paritytech/polkadot-sdk/issues/1975#issuecomment-1774534225
I'd like to encapsulate crypto related stuff in a dedicated folder.

Currently all cryptographic primitive wrappers are all sparsed in
`substrate/core` which contains "misc core" stuff.

To simplify the process, as the first step with this PR I propose to
move the cryptographic hashing there.

The `substrate/crypto` folder was already created to contains `ec-utils`
crate.

Notes:
- rename `sp-core-hashing` to `sp-crypto-hashing`
- rename `sp-core-hashing-proc-macro` to `sp-crypto-hashing-proc-macro`
- As the crates name is changed I took the freedom to restart fresh from
version 0.1.0 for both crates

---------

Co-authored-by: Robert Hambrock <[email protected]>

Clean up all the old syntax.

---------

Co-authored-by: command-bot <>
Co-authored-by: gupnik <[email protected]>
Co-authored-by: Nikhil Gupta <[email protected]>
Co-authored-by: Maksym H <[email protected]>

Closes #1323 

cc @xlc

---------

Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Backport of https://github.com/paritytech/polkadot-sdk/pull/1863 to
master

Extend candidate sanitation in paras_inherent by removing backing votes
from disabled validators. Check
https://github.com/paritytech/polkadot-sdk/issues/1592 for more details.

This change is related to the disabling strategy implementation
(https://github.com/paritytech/polkadot-sdk/pull/2226).

---------

Co-authored-by: ordian <[email protected]>
Co-authored-by: ordian <[email protected]>
Co-authored-by: Maciej <[email protected]>

# Note for reviewer

Most changes are just syntax changes necessary for the new version.
Most important files should be the ones under the `xcm` folder.

# Description 

Added XCMv4.

## Removed `Multi` prefix
The following types have been renamed:
- MultiLocation -> Location
- MultiAsset -> Asset
- MultiAssets -> Assets
- InteriorMultiLocation -> InteriorLocation
- MultiAssetFilter -> AssetFilter
- VersionedMultiAsset -> VersionedAsset
- WildMultiAsset -> WildAsset
- VersionedMultiLocation -> VersionedLocation

In order to fix a name conflict, the `Assets` in `xcm-executor` were
renamed to `HoldingAssets`, as they represent assets in holding.

## Removed `Abstract` asset id

It was not being used anywhere and this simplifies the code.

Now assets are just constructed as follows:

```rust
let asset: Asset = (AssetId(Location::new(1, Here)), 100u128).into();
```

No need for specifying `Concrete` anymore.

## Outcome is now a named fields struct

Instead of

```rust
pub enum Outcome {
  Complete(Weight),
  Incomplete(Weight, Error),
  Error(Error),
}
```

we now have

```rust
pub enum Outcome {
  Complete { used: Weight },
  Incomplete { used: Weight, error: Error },
  Error { error: Error },
}
```

## Added Reanchorable trait

Now both locations and assets implement this trait, making it easier to
reanchor both.

## New syntax for building locations and junctions

Now junctions are built using the following methods:

```rust
let location = Location {
    parents: 1,
    interior: [Parachain(1000), PalletInstance(50), GeneralIndex(1984)].into()
};
```

or

```rust
let location = Location::new(1, [Parachain(1000), PalletInstance(50), GeneralIndex(1984)]);
```

And they are matched like so:

```rust
match location.unpack() {
  (1, [Parachain(id)]) => ...
  (0, Here) => ...,
  (1, [_]) => ...,
}
```

This syntax is mandatory in v4, and has been also implemented for v2 and
v3 for easier migration.

This was needed to make all sizes smaller.

# TODO
- [x] Scaffold v4
- [x] Port github.com/paritytech/polkadot/pull/7236
- [x] Remove `Multi` prefix
- [x] Remove `Abstract` asset id

---------

Co-authored-by: command-bot <>
Co-authored-by: Keith Yeung <[email protected]>

Original PR https://github.com/paritytech/substrate/pull/14655

---

Partial https://github.com/paritytech/polkadot-sdk/issues/225

- [x] Adds conformance tests for Unbalanced
- [x] Adds conformance tests for Balanced
- Several minor fixes to fungible default implementations and the
Balances pallet
- [x] `Unbalanced::decrease_balance` can reap account when
`Preservation` is `Preserve`
- [x] `Balanced::pair` can return pairs of imbalances which do not
cancel each other out
   - [x] Balances pallet `active_issuance` 'underflow'
- [x] Refactors the conformance test file structure to match the
fungible file structure: tests for traits in regular.rs go into a test
file named regular.rs, tests for traits in freezes.rs go into a test
file named freezes.rs, etc.
 - [x] Improve doc comments
 - [x] Simplify macros

## Fixes

### `Unbalanced::decrease_balance` can reap account when called with
`Preservation::Preserve`
There is a potential issue in the default implementation of
`Unbalanced::decrease_balance`. The implementation can delete an account
even when it is called with `preservation: Preservation::Preserve`. This
seems to contradict the documentation of `Preservation::Preserve`:

```rust
	/// The account may not be killed and our provider reference must remain (in the context of
	/// tokens, this means that the account may not be dusted).
	Preserve,
```

I updated `Unbalanced::decrease_balance` to return
`Err(TokenError::BelowMinimum)` when a withdrawal would cause the
account to be reaped and `preservation: Preservation::Preserve`.

- [ ] TODO Confirm with @gavofyork that this is correct behavior

Test for this behavior:


https://github.com/paritytech/polkadot-sdk/blob/e5c876dd/substrate/frame/support/src/traits/tokens/fungible/conformance_tests/regular.rs#L912-L937

### `Balanced::pair` returning non-canceling pairs

`Balanced::pair` is supposed to create a pair of imbalances that cancel
each other out. However this is not the case when the method is called
with an amount greater than the total supply.

In the existing default implementation, `Balanced::pair` creates a pair
by first rescinding the balance, creating `Debt`, and then issuing the
balance, creating `Credit`.

When creating `Debt`, if the amount to create exceeds the
`total_supply`, `total_supply` units of `Debt` are created *instead* of
`amount` units of `Debt`. This can lead to non-canceling amount of
`Credit` and `Debt` being created.

To address this, I create the credit and debt directly in the method
instead of calling `issue` and `rescind`.

Test for this behavior:


https://github.com/paritytech/polkadot-sdk/blob/e5c876dd/substrate/frame/support/src/traits/tokens/fungible/conformance_tests/regular.rs#L1323-L1346

### `Balances` pallet `active_issuance` 'underflow'

This PR resolves an issue in the `Balances` pallet that can lead to odd
behavior of `active_issuance`.

Currently, the Balances pallet doesn't check if `InactiveIssuance`
remains less than or equal to `TotalIssuance` when supply is
deactivated. This allows `InactiveIssuance` to be greater than
`TotalIssuance`, which can result in unexpected behavior from the
perspective of the fungible API.

`active_issuance` is derived from
`TotalIssuance.saturating_sub(InactiveIssuance)`.

If an `amount` is deactivated that causes `InactiveIssuance` to become
greater TotalIssuance, `active_issuance` will return 0. However once in
that state, reactivating an amount will not increase `active_issuance`
by the reactivated `amount` as expected.

Consider this test where the last assertion would fail due to this
issue:


https://github.com/paritytech/polkadot-sdk/blob/e5c876dd

/substrate/frame/support/src/traits/tokens/fungible/conformance_tests/regular.rs#L1036-L1071

To address this, I've modified the `deactivate` function to ensure
`InactiveIssuance` never surpasses `TotalIssuance`.

---------

Co-authored-by: Muharem <[email protected]>