Skip to content
  1. Mar 14, 2024
    • Gonçalo Pestana's avatar
      Staking ledger bonding fixes (#3639) · 606664e1
      Gonçalo Pestana authored 
      Currently, the staking logic does not prevent a controller from becoming
a stash of *another* ledger (introduced by [removing this
check](https://github.com/paritytech/polkadot-sdk/pull/1484/files#diff-3aa6ceab5aa4e0ab2ed73a7245e0f5b42e0832d8ca5b1ed85d7b2a52fb196524L850)).
Given that the remaining of the code expects that never happens, bonding
a ledger with a stash that is a controller of another ledger may lead to
data inconsistencies and data losses in bonded ledgers. For more
detailed explanation of this issue:
https://hackmd.io/@gpestana/HJoBm2tqo/%2FTPdi28H7Qc2mNUqLSMn15w

In a nutshell, when fetching a ledger with a given controller, we may be
end up getting the wrong ledger which can lead to unexpected ledger
states.

This PR also ensures that `set_controller` does not lead to data
inconsistencies in the staking ledger and bonded storage in the case
when a controller of a stash is a stash of *another* ledger. and
improves the staking `try-runtime` checks to catch potential issues with
the storage preemptively.

In summary, there are two important cases here:

1. **"Sane" double bonded ledger**

When a controller of a ledger is a stash of *another* ledger. In this
case, we have:

```
> Bonded(stash, controller)
(A, B)  // stash A with controller B
(B, C) // B is also a stash of another ledger
(C, D)

> Ledger(controller)
Ledger(B) = L_a (stash = A)
Ledger(C) = L_b (stash = B)
Ledger(D) = L_c (stash = C)
```

In this case, the ledgers can be mutated and all operations are OK.
However, we should not allow `set_controller` to be called if it means
it results in a "corrupt" double bonded ledger (see below).

3. **"Corrupt" double bonded ledger**

```
> Bonded(stash, controller)
(A, B)  // stash A with controller B
(B, B)
(C, D)
```
In this case, B is a stash and controller AND is corrupted, since B is
responsible for 2 ledgers which is not correct and will lead to
inconsistent states. Thus, in this case, in this PR we are preventing
these ledgers from mutating (i.e. operations like bonding extra etc)
until the ledger is brought back to a consistent state.

--- 

**Changes**:
- Checks if stash is already a controller when calling `Call::bond`
(fixes the regression introduced by [removing this
check](https://github.com/paritytech/polkadot-sdk/pull/1484/files#diff-3aa6ceab5aa4e0ab2ed73a7245e0f5b42e0832d8ca5b1ed85d7b2a52fb196524L850));
- Ensures that all fetching ledgers from storage are done through the
`StakingLedger` API;
- Ensures that -- when fetching a ledger from storage using the
`StakingLedger` API --, a `Error::BadState` is returned if the ledger
bonding is in a bad state. This prevents bad ledgers from mutating (e.g.
`bond_extra`, `set_controller`, etc) its state and avoid further data
inconsistencies.
- Prevents stashes which are controllers or another ledger from calling
`set_controller`, since that may lead to a bad state.
- Adds further try-state runtime checks that check if there are ledgers
in a bad state based on their bonded metadata.

Related to https://github.com/paritytech/polkadot-sdk/issues/3245



---------

Co-authored-by: default avatarKian Paimani <[email protected]>
Co-authored-by: default avatarkianenigma <[email protected]>
      606664e1
  3. Mar 13, 2024
    • georgepisaltu's avatar
      Revert "FRAME: Create `TransactionExtension` as a replacement for... · bbd51ce8
      georgepisaltu authored 
      Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280)" (#3665)

This PR reverts #2280 which introduced `TransactionExtension` to replace
`SignedExtension`.

As a result of the discussion
[here](https://github.com/paritytech/polkadot-sdk/pull/3623#issuecomment-1986789700

),
the changes will be reverted for now with plans to reintroduce the
concept in the future.

---------

Signed-off-by: default avatargeorgepisaltu <[email protected]>
      bbd51ce8
    • gupnik's avatar
      Construct Runtime v2 (#1378) · 82f3c3e2
      gupnik authored 
      Moved from https://github.com/paritytech/substrate/pull/14788

----

Fixes https://github.com/paritytech/polkadot-sdk/issues/232

This PR introduces outer-macro approach for `construct_runtime` as
discussed in the linked issue. It looks like the following:
```rust
#[frame_support::runtime]
mod runtime {
	#[runtime::runtime]
        #[runtime::derive(
		RuntimeCall,
		RuntimeEvent,
		RuntimeError,
		RuntimeOrigin,
		RuntimeFreezeReason,
		RuntimeHoldReason,
		RuntimeSlashReason,
		RuntimeLockId,
                RuntimeTask,
	)]
	pub struct Runtime;

	#[runtime::pallet_index(0)]
	pub type System = frame_system;

	#[runtime::pallet_index(1)]
	pub type Timestamp = pallet_timestamp;

	#[runtime::pallet_index(2)]
	pub type Aura = pallet_aura;

	#[runtime::pallet_index(3)]
	pub type Grandpa = pallet_grandpa;

	#[runtime::pallet_index(4)]
	pub type Balances = pallet_balances;

	#[runtime::pallet_index(5)]
	pub type TransactionPayment = pallet_transaction_payment;

	#[runtime::pallet_index(6)]
	pub type Sudo = pallet_sudo;

	// Include the custom logic from the pallet-template in the runtime.
	#[runtime::pallet_index(7)]
	pub type TemplateModule = pallet_template;
}
```

## Features
- `#[runtime::runtime]` attached to a struct defines the main runtime
- `#[runtime::derive]` attached to this struct defines the types
generated by runtime
- `#[runtime::pallet_index]` must be attached to a pallet to define its
index
- `#[runtime::disable_call]` can be optionally attached to a pallet to
disable its calls
- `#[runtime::disable_unsigned]` can be optionally attached to a pallet
to disable unsigned calls
- A pallet instance can be defined as `TemplateModule:
pallet_template<Instance>`
- An optional attribute can be defined as
`#[frame_support::runtime(legacy_ordering)]` to ensure that the order of
hooks is same as the order of pallets (and not based on the
pallet_index). This is to support legacy runtimes and should be avoided
for new ones.

## Todo
- [x] Update the latest syntax in kitchensink and tests
- [x] Update UI tests
- [x] Docs

## Extension
- Abstract away the Executive similar to
https://github.com/paritytech/substrate/pull/14742


- Optionally avoid the need to specify all runtime types (TBD)

---------

Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
Co-authored-by: Nikhil Gupta <>
      82f3c3e2
  5. Mar 12, 2024
  7. Mar 11, 2024
    • philoniare's avatar
      [Deprecation] Remove the deprecated Store trait (#3532) · d3f81056
      philoniare authored 
      

# Description

*Removes the deprecated `trait Store` feature from the code base*

Fixes #222

---------

Co-authored-by: default avatarDónal Murray <[email protected]>
      d3f81056
    • s0me0ne-unkn0wn's avatar
      Remove getters from `im-online` pallet (#3589) · 4249a3d6
      s0me0ne-unkn0wn authored 
      As I've been dancing around this pallet for quite some time anyway, I
decided to remove getters at once. There were only a few leftovers in
tests.

Related: #3326 
CC @muraca
      4249a3d6
    • Dónal Murray's avatar
      [pallet_broker] Fix `adapt_price` behaviour at zero (#3636) · aa353283
      Dónal Murray authored 
      

This fixes the behaviour of `Linear` which is the default implementation
of the `AdaptPrice` trait in the broker pallet. Previously if cores were
offered but not sold in only one sale, the price would be set to zero
and due to the logic being purely multiplicative, the price would stay
at 0 indefinitely.

This could be further paired with a configurable minimum in the broker
pallet itself, which will be a future PR.

This affects the Rococo and Westend Coretime chains, but Kusama has a
different implementation so this isn't required for the Kusama launch. I
actually thought I opened this a while ago.

---------

Co-authored-by: default avatarBastian Köcher <[email protected]>
      aa353283
  9. Mar 10, 2024
  11. Mar 09, 2024
  13. Mar 08, 2024
  15. Mar 07, 2024
  17. Mar 06, 2024
  19. Mar 05, 2024
  21. Mar 04, 2024
    • PG Herveou's avatar
      Contracts: Fix typo (#3563) · 0708cf38
      PG Herveou authored
      0708cf38
    • Gavin Wood's avatar
      FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280) · fd5f9292
      Gavin Wood authored 
      Closes #2160

First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415

)

Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for the latter. (Ultimately, the latter will be phased out as
a type of transaction, and Bare will only be used for Inherents.)

Types of extrinsic are now therefore:
- Bare (no hardcoded signature, no Extra data; used to be known as
"Unsigned")
- Bare transactions (deprecated): Gossiped, validated with
`ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
`TransactionExtension` (deprecated).
  - Inherents: Not gossiped, validated with `ProvideInherent`.
- Extended (Extra data): Gossiped, validated via `TransactionExtension`.
  - Signed transactions (with a hardcoded signature).
  - General transactions (without a hardcoded signature).

`TransactionExtension` differs from `SignedExtension` because:
- A signature on the underlying transaction may validly not be present.
- It may alter the origin during validation.
- `pre_dispatch` is renamed to `prepare` and need not contain the checks
present in `validate`.
- `validate` and `prepare` is passed an `Origin` rather than a
`AccountId`.
- `validate` may pass arbitrary information into `prepare` via a new
user-specifiable type `Val`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`. It is encoded *for the entire transaction* and
passed in to each extension as a new argument to `validate`. This
facilitates the ability of extensions to acts as underlying crypto.

There is a new `DispatchTransaction` trait which contains only default
function impls and is impl'ed for any `TransactionExtension` impler. It
provides several utility functions which reduce some of the tedium from
using `TransactionExtension` (indeed, none of its regular functions
should now need to be called directly).

Three transaction version discriminator ("versions") are now
permissible:
- 0b000000100: Bare (used to be called "Unsigned"): contains Signature
or Extra (extension data). After bare transactions are no longer
supported, this will strictly identify an Inherents only.
- 0b100000100: Old-school "Signed" Transaction: contains Signature and
Extra (extension data).
- 0b010000100: New-school "General" Transaction: contains Extra
(extension data), but no Signature.

For the New-school General Transaction, it becomes trivial for authors
to publish extensions to the mechanism for authorizing an Origin, e.g.
through new kinds of key-signing schemes, ZK proofs, pallet state,
mutations over pre-authenticated origins or any combination of the
above.

## Code Migration

### NOW: Getting it to build

Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
accompanied by renaming your aggregate type in line with the new
terminology. E.g. Before:

```rust
/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
	/* snip */
	MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
```

After:

```rust
/// The extension to the basic transaction logic.
pub type TxExtension = (
	/* snip */
	AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
```

You'll also need to alter any transaction building logic to add a
`.into()` to make the conversion happen. E.g. Before:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let extra: SignedExtra = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */),
	);
	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		extra,
	)
}
```

After:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let tx_ext: TxExtension = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */).into(),
	);
	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		tx_ext,
	)
}
```

### SOON: Migrating to `TransactionExtension`

Most `SignedExtension`s can be trivially converted to become a
`TransactionExtension`. There are a few things to know.

- Instead of a single trait like `SignedExtension`, you should now
implement two traits individually: `TransactionExtensionBase` and
`TransactionExtension`.
- Weights are now a thing and must be provided via the new function `fn
weight`.

#### `TransactionExtensionBase`

This trait takes care of anything which is not dependent on types
specific to your runtime, most notably `Call`.

- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`.
- Weight must be returned by implementing the `weight` function. If your
extension is associated with a pallet, you'll probably want to do this
via the pallet's existing benchmarking infrastructure.

#### `TransactionExtension`

Generally:
- `pre_dispatch` is now `prepare` and you *should not reexecute the
`validate` functionality in there*!
- You don't get an account ID any more; you get an origin instead. If
you need to presume an account ID, then you can use the trait function
`AsSystemOriginSigner::as_system_origin_signer`.
- You get an additional ticket, similar to `Pre`, called `Val`. This
defines data which is passed from `validate` into `prepare`. This is
important since you should not be duplicating logic from `validate` to
`prepare`, you need a way of passing your working from the former into
the latter. This is it.
- This trait takes two type parameters: `Call` and `Context`. `Call` is
the runtime call type which used to be an associated type; you can just
move it to become a type parameter for your trait impl. `Context` is not
currently used and you can safely implement over it as an unbounded
type.
- There's no `AccountId` associated type any more. Just remove it.

Regarding `validate`:
- You get three new parameters in `validate`; all can be ignored when
migrating from `SignedExtension`.
- `validate` returns a tuple on success; the second item in the tuple is
the new ticket type `Self::Val` which gets passed in to `prepare`. If
you use any information extracted during `validate` (off-chain and
on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
pass it through with this. For the tuple's last item, just return the
`origin` argument.

Regarding `prepare`:
- This is renamed from `pre_dispatch`, but there is one change:
- FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
`validate`!!
- (This is different to `SignedExtension` which was required to run the
same checks in `pre_dispatch` as in `validate`.)

Regarding `post_dispatch`:
- Since there are no unsigned transactions handled by
`TransactionExtension`, `Pre` is always defined, so the first parameter
is `Self::Pre` rather than `Option<Self::Pre>`.

If you make use of `SignedExtension::validate_unsigned` or
`SignedExtension::pre_dispatch_unsigned`, then:
- Just use the regular versions of these functions instead.
- Have your logic execute in the case that the `origin` is `None`.
- Ensure your transaction creation logic creates a General Transaction
rather than a Bare Transaction; this means having to include all
`TransactionExtension`s' data.
- `ValidateUnsigned` can still be used (for now) if you need to be able
to construct transactions which contain none of the extension data,
however these will be phased out in stage 2 of the Transactions Horizon,
so you should consider moving to an extension-centric design.

## TODO

- [x] Introduce `CheckSignature` impl of `TransactionExtension` to
ensure it's possible to have crypto be done wholly in a
`TransactionExtension`.
- [x] Deprecate `SignedExtension` and move all uses in codebase to
`TransactionExtension`.
  - [x] `ChargeTransactionPayment`
  - [x] `DummyExtension`
  - [x] `ChargeAssetTxPayment` (asset-tx-payment)
  - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment)
  - [x] `CheckWeight`
  - [x] `CheckTxVersion`
  - [x] `CheckSpecVersion`
  - [x] `CheckNonce`
  - [x] `CheckNonZeroSender`
  - [x] `CheckMortality`
  - [x] `CheckGenesis`
  - [x] `CheckOnlySudoAccount`
  - [x] `WatchDummy`
  - [x] `PrevalidateAttests`
  - [x] `GenericSignedExtension`
  - [x] `SignedExtension` (chain-polkadot-bulletin)
  - [x] `RefundSignedExtensionAdapter`
- [x] Implement `fn weight` across the board.
- [ ] Go through all pre-existing extensions which assume an account
signer and explicitly handle the possibility of another kind of origin.
- [x] `CheckNonce` should probably succeed in the case of a non-account
origin.
- [x] `CheckNonZeroSender` should succeed in the case of a non-account
origin.
- [x] `ChargeTransactionPayment` and family should fail in the case of a
non-account origin.
  - [ ] 
- [x] Fix any broken tests.

---------

Signed-off-by: default avatargeorgepisaltu <[email protected]>
Signed-off-by: default avatarAlexandru Vasile <[email protected]>
Signed-off-by: default avatardependabot[bot] <[email protected]>
Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
Signed-off-by: default avatarAlexandru Gheorghe <[email protected]>
Signed-off-by: default avatarAndrei Sandu <[email protected]>
Co-authored-by: default avatarNikhil Gupta <[email protected]>
Co-authored-by: default avatargeorgepisaltu <[email protected]>
Co-authored-by: default avatarChevdor <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
Co-authored-by: default avatarMaciej <[email protected]>
Co-authored-by: default avatarJavier Viola <[email protected]>
Co-authored-by: default avatarMarcin S. <[email protected]>
Co-authored-by: default avatarTsvetomir Dimitrov <[email protected]>
Co-authored-by: default avatarJavier Bullrich <[email protected]>
Co-authored-by: default avatarKoute <[email protected]>
Co-authored-by: default avatarAdrian Catangiu <[email protected]>
Co-authored-by: Vladimir Istyufeev's avatarVladimir Istyufeev <[email protected]>
Co-authored-by: default avatarRoss Bulat <[email protected]>
Co-authored-by: default avatarGonçalo Pestana <[email protected]>
Co-authored-by: default avatarLiam Aharon <[email protected]>
Co-authored-by: default avatarSvyatoslav Nikolsky <[email protected]>
Co-authored-by: default avatarAndré Silva <[email protected]>
Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatars0me0ne-unkn0wn <[email protected]>
Co-authored-by: default avatarordian <[email protected]>
Co-authored-by: default avatarSebastian Kunert <[email protected]>
Co-authored-by: default avatarAaro Altonen <[email protected]>
Co-authored-by: default avatarDmitry Markin <[email protected]>
Co-authored-by: default avatarAlexandru Vasile <[email protected]>
Co-authored-by: default avatarAlexander Samusev <[email protected]>
Co-authored-by: default avatarJulian Eager <[email protected]>
Co-authored-by: default avatarMichal Kucharczyk <[email protected]>
Co-authored-by: default avatarDavide Galassi <[email protected]>
Co-authored-by: default avatarDónal Murray <[email protected]>
Co-authored-by: default avataryjh <[email protected]>
Co-authored-by: default avatarTom Mi <[email protected]>
Co-authored-by: default avatardependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: default avatarWill | Paradox | ParaNodes.io <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
Co-authored-by: default avatarJoshy Orndorff <[email protected]>
Co-authored-by: default avatarJoshy Orndorff <[email protected]>
Co-authored-by: default avatarPG Herveou <[email protected]>
Co-authored-by: default avatarAlexander Theißen <[email protected]>
Co-authored-by: default avatarKian Paimani <[email protected]>
Co-authored-by: default avatarJuan Girini <[email protected]>
Co-authored-by: default avatarbader y <[email protected]>
Co-authored-by: default avatarJames Wilson <[email protected]>
Co-authored-by: default avatarjoe petrowski <[email protected]>
Co-authored-by: default avatarasynchronous rob <[email protected]>
Co-authored-by: default avatarParth <[email protected]>
Co-authored-by: default avatarAndrew Jones <[email protected]>
Co-authored-by: default avatarJonathan Udd <[email protected]>
Co-authored-by: default avatarSerban Iorga <[email protected]>
Co-authored-by: default avatarEgor_P <[email protected]>
Co-authored-by: default avatarBranislav Kontur <[email protected]>
Co-authored-by: default avatarEvgeny Snitko <[email protected]>
Co-authored-by: default avatarJust van Stam <[email protected]>
Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
Co-authored-by: default avatargupnik <[email protected]>
Co-authored-by: default avatardzmitry-lahoda <[email protected]>
Co-authored-by: default avatarzhiqiangxu <[email protected]>
Co-authored-by: default avatarNazar Mokrynskyi <[email protected]>
Co-authored-by: default avatarAnwesh <[email protected]>
Co-authored-by: default avatarcheme <[email protected]>
Co-authored-by: default avatarSam Johnson <[email protected]>
Co-authored-by: default avatarkianenigma <[email protected]>
Co-authored-by: default avatarJegor Sidorenko <[email protected]>
Co-authored-by: default avatarMuharem <[email protected]>
Co-authored-by: default avatarjoepetrowski <[email protected]>
Co-authored-by: default avatarAlexandru Gheorghe <[email protected]>
Co-authored-by: default avatarGabriel Facco de Arruda <[email protected]>
Co-authored-by: default avatarSquirrel <[email protected]>
Co-authored-by: default avatarAndrei Sandu <[email protected]>
Co-authored-by: default avatargeorgepisaltu <[email protected]>
Co-authored-by: command-bot <>
      fd5f9292
  23. Mar 03, 2024
  25. Mar 02, 2024
    • gupnik's avatar
      Remove `as frame_system::DefaultConfig` from the required syntax in `derive_impl` (#3505) · cdc8d197
      gupnik authored 
      Step in https://github.com/paritytech/polkadot-sdk/issues/171

This PR removes the need to specify `as [disambiguation_path]` for cases
where the trait definition resides within the same scope as default impl
path.

For example, in the following macro invocation
```rust
#[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)]
impl frame_system::Config for Runtime {
   ...
}
```
the trait `DefaultConfig` lies within the `frame_system` scope and
`TestDefaultConfig` impls the `DefaultConfig` trait. Using this
information, we can compute the disambiguation path internally, thus
removing the need of an explicit specification.

In cases where the trait lies outside this scope, we would still need to
specify it explicitly, but this should take care of most (if not all)
uses of `derive_impl` within FRAME's context.
      cdc8d197
  27. Feb 29, 2024
    • Kian Paimani's avatar
      Fix call enum's metadata regression (#3513) · c0e52a9e
      Kian Paimani authored 
      This fixes an issue introduced in
https://github.com/paritytech/substrate/pull/14101, in which I removed
the `Call` enum's documentation and replaced it with a link to the
`Pallet` struct, but this also removed any docs related to call from the
metadata.

I tried to add a regression test for this, but it seems to me that this
is not possible, given that using `type-info` we only assert in type-ids
for `Call`, `Event` and `Error`. I removed some doc comments from a test
setup in `frame-support-test` to demonstrate the issue there. @jsdw do
you have any comments on this?

I also fixed a small issue in the custom html/css of `polkadot-sdk-doc`
crate, making sure it does not affect the rust-doc page of all other
crates.

- [x] Investigate a regression test
- [x] prdoc
      c0e52a9e
    • philoniare's avatar
      [Deprecation] Remove sp_weights::OldWeight (#3491) · a22319cd
      philoniare authored 
      

# Description

*Removes `sp_weights::OldWeight` and its usage*

Fixes #144

---------

Co-authored-by: default avatarLiam Aharon <[email protected]>
      a22319cd
  29. Feb 28, 2024
    • Oliver Tale-Yazdi's avatar
      Multi-Block-Migrations, `poll` hook and new System callbacks (#1781) · eefd5fe4
      Oliver Tale-Yazdi authored 
      This MR is the merge of
https://github.com/paritytech/substrate/pull/14414 and
https://github.com/paritytech/substrate/pull/14275. It implements
[RFC#13](https://github.com/polkadot-fellows/RFCs/pull/13), closes
https://github.com/paritytech/polkadot-sdk/issues/198.

----- 

This Merge request introduces three major topicals:

1. Multi-Block-Migrations
1. New pallet `poll` hook for periodic service work
1. Replacement hooks for `on_initialize` and `on_finalize` in cases
where `poll` cannot be used

and some more general changes to FRAME.  
The changes for each topical span over multiple crates. They are listed
in topical order below.

# 1.) Multi-Block-Migrations

Multi-Block-Migrations are facilitated by creating `pallet_migrations`
and configuring `System::Config::MultiBlockMigrator` to point to it.
Executive picks this up and triggers one step of the migrations pallet
per block.
The chain is in lockdown mode for as long as an MBM is ongoing.
Executive does this by polling `MultiBlockMigrator::ongoing` and not
allowing any transaction in a block, if true.

A MBM is defined through trait `SteppedMigration`. A condensed version
looks like this:
```rust
/// A migration that can proceed in multiple steps.
pub trait SteppedMigration {
	type Cursor: FullCodec + MaxEncodedLen;
	type Identifier: FullCodec + MaxEncodedLen;

	fn id() -> Self::Identifier;

	fn max_steps() -> Option<u32>;

	fn step(
		cursor: Option<Self::Cursor>,
		meter: &mut WeightMeter,
	) -> Result<Option<Self::Cursor>, SteppedMigrationError>;
}
```

`pallet_migrations` can be configured with an aggregated tuple of these
migrations. It then starts to migrate them one-by-one on the next
runtime upgrade.
Two things are important here:
- 1. Doing another runtime upgrade while MBMs are ongoing is not a good
idea and can lead to messed up state.
- 2. **Pallet Migrations MUST BE CONFIGURED IN `System::Config`,
otherwise it is not used.**

The pallet supports an `UpgradeStatusHandler` that can be used to notify
external logic of upgrade start/finish (for example to pause XCM
dispatch).

Error recovery is very limited in the case that a migration errors or
times out (exceeds its `max_steps`). Currently the runtime dev can
decide in `FailedMigrationHandler::failed` how to handle this. One
follow-up would be to pair this with the `SafeMode` pallet and enact
safe mode when an upgrade fails, to allow governance to rescue the
chain. This is currently not possible, since governance is not
`Mandatory`.

## Runtime API

- `Core`: `initialize_block` now returns `ExtrinsicInclusionMode` to
inform the Block Author whether they can push transactions.

### Integration

Add it to your runtime implementation of `Core` and `BlockBuilder`:
```patch
diff --git a/runtime/src/lib.rs b/runtime/src/lib.rs
@@ impl_runtime_apis! {
	impl sp_block_builder::Core<Block> for Runtime {
-		fn initialize_block(header: &<Block as BlockT>::Header) {
+		fn initialize_block(header: &<Block as BlockT>::Header) -> RuntimeExecutiveMode {
			Executive::initialize_block(header)
		}

		...
	}
```

# 2.) `poll` hook

A new pallet hook is introduced: `poll`. `Poll` is intended to replace
mostly all usage of `on_initialize`.
The reason for this is that any code that can be called from
`on_initialize` cannot be migrated through an MBM. Currently there is no
way to statically check this; the implication is to use `on_initialize`
as rarely as possible.
Failing to do so can result in broken storage invariants.

The implementation of the poll hook depends on the `Runtime API` changes
that are explained above.

# 3.) Hard-Deadline callbacks

Three new callbacks are introduced and configured on `System::Config`:
`PreInherents`, `PostInherents` and `PostTransactions`.
These hooks are meant as replacement for `on_initialize` and
`on_finalize` in cases where the code that runs cannot be moved to
`poll`.
The reason for this is to make the usage of HD-code (hard deadline) more
explicit - again to prevent broken invariants by MBMs.

# 4.) FRAME (general changes)

## `frame_system` pallet

A new memorize storage item `InherentsApplied` is added. It is used by
executive to track whether inherents have already been applied.
Executive and can then execute the MBMs directly between inherents and
transactions.

The `Config` gets five new items:
- `SingleBlockMigrations` this is the new way of configuring migrations
that run in a single block. Previously they were defined as last generic
argument of `Executive`. This shift is brings all central configuration
about migrations closer into view of the developer (migrations that are
configured in `Executive` will still work for now but is deprecated).
- `MultiBlockMigrator` this can be configured to an engine that drives
MBMs. One example would be the `pallet_migrations`. Note that this is
only the engine; the exact MBMs are injected into the engine.
- `PreInherents` a callback that executes after `on_initialize` but
before inherents.
- `PostInherents` a callback that executes after all inherents ran
(including MBMs and `poll`).
- `PostTransactions` in symmetry to `PreInherents`, this one is called
before `on_finalize` but after all transactions.

A sane default is to set all of these to `()`. Example diff suitable for
any chain:
```patch
@@ impl frame_system::Config for Test {
 	type MaxConsumers = ConstU32<16>;
+	type SingleBlockMigrations = ();
+	type MultiBlockMigrator = ();
+	type PreInherents = ();
+	type PostInherents = ();
+	type PostTransactions = ();
 }
```

An overview of how the block execution now looks like is here. The same
graph is also in the rust doc.

<details><summary>Block Execution Flow</summary>
<p>

![Screenshot 2023-12-04 at 19 11
29](https://github.com/paritytech/polkadot-sdk/assets/10380170/e88a80c4-ef11-4faa-8df5-8b33a724c054)

</p>
</details> 

## Inherent Order

Moved to https://github.com/paritytech/polkadot-sdk/pull/2154



---------------


## TODO

- [ ] Check that `try-runtime` still works
- [ ] Ensure backwards compatibility with old Runtime APIs
- [x] Consume weight correctly
- [x] Cleanup

---------

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatarLiam Aharon <[email protected]>
Co-authored-by: default avatarJuan Girini <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
Co-authored-by: default avatarGavin Wood <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
      eefd5fe4
    • Kian Paimani's avatar
      Add documentation around FRAME Offchain workers (#3463) · 14530269
      Kian Paimani authored 
      - deprecation companion:
https://github.com/substrate-developer-hub/substrate-docs/pull/2136
- inspired by
https://substrate.stackexchange.com/questions/11058/how-can-i-create-ocw-that-wont-activates-every-block-but-will-activates-only-w/11060#11060



---------

Co-authored-by: default avatarSergej Sakac <[email protected]>
      14530269
    • Liam Aharon's avatar
      Runtime Upgrade ref docs and Single Block Migration example pallet (#1554) · 12ce4f7d
      Liam Aharon authored 
      Closes https://github.com/paritytech/polkadot-sdk-docs/issues/55

- Changes 'current storage version' terminology to less ambiguous
'in-code storage version' (suggestion by @ggwpez)
- Adds a new example pallet `pallet-example-single-block-migrations`
- Adds a new reference doc to replace
https://docs.substrate.io/maintain/runtime-upgrades/ (temporarily living
in the pallet while we wait for developer hub PR to merge)
- Adds documentation for the `storage_alias` macro
- Improves `trait Hooks` docs 
- Improves `trait GetStorageVersion` docs
- Update the suggested patterns for using `VersionedMigration`, so that
version unchecked migrations are never exported
- Prevents accidental usage of version unchecked migrations in runtimes

https://github.com/paritytech/substrate/pull/14421#discussion_r1255467895
- Unversioned migration code is kept inside `mod version_unchecked`,
versioned code is kept in `pub mod versioned`
- It is necessary to use modules to limit visibility because the inner
migration must be `pub`. See
https://github.com/rust-lang/rust/issues/30905 and

https://internals.rust-lang.org/t/lang-team-minutes-private-in-public-rules/4504/40
for more.

### todo

- [x] move to reference docs to proper place within sdk-docs (now that
https://github.com/paritytech/polkadot-sdk/pull/2102

 is merged)
- [x] prdoc

---------

Co-authored-by: default avatarKian Paimani <[email protected]>
Co-authored-by: default avatarJuan <[email protected]>
Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: default avatargupnik <[email protected]>
      12ce4f7d
    • Liam Aharon's avatar
      Introduce storage attr macro `#[disable_try_decode_storage]` and set it on... · 95da6583
      Liam Aharon authored 
      Introduce storage attr macro `#[disable_try_decode_storage]` and set it on `System::Events` and `ParachainSystem::HostConfiguration` (#3454)

Closes https://github.com/paritytech/polkadot-sdk/issues/2560

Allows marking storage items with `#[disable_try_decode_storage]`, and
uses it with `System::Events`.

Question: what's the recommended way to write a test for this? I
couldn't find a test for similar existing macro `#[whitelist_storage]`.
      95da6583
  31. Feb 27, 2024
  33. Feb 26, 2024
  35. Feb 23, 2024
    • Sebastian Kunert's avatar
      PoV Reclaim Runtime Side (#3002) · 3386377b
      Sebastian Kunert authored 
      

# Runtime side for PoV Reclaim

## Implementation Overview
- Hostfunction to fetch the storage proof size has been added to the
PVF. It uses the size tracking recorder that was introduced in my
previous PR.
- Mechanisms to use the reclaim HostFunction have been introduced.
- 1. A SignedExtension that checks the node-reported proof size before
and after application of an extrinsic. Then it reclaims the difference.
- 2. A manual helper to make reclaiming easier when manual interaction
is required, for example in `on_idle` or other hooks.
- In order to utilize the manual reclaiming, I modified `WeightMeter` to
support the reduction of consumed weight, at least for storage proof
size.

## How to use
To enable the general functionality for a parachain:
1. Add the SignedExtension to your parachain runtime. 
2. Provide the HostFunction to the node
3. Enable proof recording during block import

## TODO
- [x] PRDoc

---------

Co-authored-by: default avatarDmitry Markin <[email protected]>
Co-authored-by: default avatarDavide Galassi <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
      3386377b
    • Dino Pačandi's avatar
      pallet-membership weights (#3324) · 5fc6d67b
      Dino Pačandi authored 
      

## Summary
* use benchamarked weights instead of hardcoded ones for
`pallet-membership`
* rename benchmark to match extrinsic name
* remove unnecessary dependency from `clear_prime`

---------

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      5fc6d67b