Skip to content
  1. Jan 12, 2024
  3. Jan 11, 2024
    • Sebastian Kunert's avatar
      Cumulus test service cleanup (#2887) · c93f5aba
      Sebastian Kunert authored 
      closes #2567 

Followup for https://github.com/paritytech/polkadot-sdk/pull/2331

This PR contains multiple internal cleanups:

1. This gets rid of the functionality in `generate_genesis_block` which
was only used in one benchmark
2. Fixed `transaction_pool` and `transaction_throughput` benchmarks
failing since they require a tokio runtime now.
3. Removed `parachain_id` CLI option from the test parachain
4. Removed `expect` call from `RuntimeResolver`
      c93f5aba
  5. Jan 10, 2024
    • Nazar Mokrynskyi's avatar
      Improve storage monitor API (#2899) · af2e30e3
      Nazar Mokrynskyi authored 
      This removes the need to unnecessarily provide a very specific data
structure `DatabaseSource` and removes huge `sc-client-db` dependency
from storage monitor. It is now possible to use storage monitor with any
path.

P.S. I still strongly dislike that it pulls `clap` dependency for such a
small feature, but many other crates do as well, so nothing special
here.
      af2e30e3
    • joe petrowski's avatar
      Unique Usernames in Identity Pallet (#2651) · d1f678c0
      joe petrowski authored 
      

This PR allows _username authorities_ to issue unique usernames that
correspond with an account. It also provides two-way lookup, that is
from `AccountId` to a single, "primary" `Username` (alongside
`Registration`) and multiple unique `Username`s to an `AccountId`.

Key features:

- Username Authorities added (and removed) via privileged origin.
- Authorities have a `suffix` and an `allocation`. They can grant up to
`allocation` usernames. Their `suffix` will be appended to the usernames
that they issue. A suffix may be up to 7 characters long.
- Users can ask an authority to grant them a username. This will take
the form `myusername.suffix`. The entire name (including suffix) must be
less than or equal to 32 alphanumeric characters.
- Users can approve a username for themselves in one of two ways (that
is, authorities cannot grant them arbitrarily):
- Pre-sign the entire username (including suffix) with a secret key that
corresponds to their `AccountId` (for keyed accounts, obviously); or
- Accept the username after it has been granted by an authority (it will
be queued until accepted) (for non-keyed accounts like pure proxies or
multisigs).
- The system does not require any funds or deposits. Users without an
identity will be given a default one (presumably all fields set to
`None`). If they update this info, they will need to place the normal
storage deposit.
- If a user does not have any username, their first one will be set as
`Primary`, and their `AccountId` will map to that one. If they get
subsequent usernames, they can choose which one to be their primary via
`set_primary_username`.
- There are some state cleanup functions to remove expired usernames
that have not been accepted and dangling usernames whose owners have
called `clear_identity`.

TODO:

- [x] Add migration to runtimes
- [x] Probably do off-chain migration into People Chain genesis
- [x] Address a few TODO questions in code (please review)

---------

Co-authored-by: default avatarLiam Aharon <[email protected]>
Co-authored-by: default avatarGonçalo Pestana <[email protected]>
Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatarDónal Murray <[email protected]>
      d1f678c0
  7. Jan 09, 2024
  9. Jan 07, 2024
  11. Jan 06, 2024
  13. Jan 04, 2024
  15. Dec 21, 2023
  17. Dec 20, 2023
  19. Dec 19, 2023
  21. Dec 15, 2023
  23. Dec 13, 2023
  25. Dec 12, 2023
    • Ross Bulat's avatar
      Staking: Add `deprecate_controller_batch` AdminOrigin call (#2589) · 048a9c27
      Ross Bulat authored 
      

Partially Addresses #2500

Adds a `deprecate_controller_batch` call to the staking pallet that is
callable by `Root` and `StakingAdmin`. To be used for controller account
deprecation and removed thereafter. Adds
`MaxControllersDeprecationBatch` pallet constant that defines max
possible deprecations per call.

- [x] Add `deprecate_controller_batch` call, and
`MaxControllersInDeprecationBatch` constant.
- [x] Add tests, benchmark, weights. Tests that weight is only consumed
if unique pair.
- [x] Adds `StakingAdmin` origin to staking's `AdminOrigin` type in
westend runtime.
- [x] Determined that worst case 5,900 deprecations does fit into
`maxBlock` `proofSize` and `refTime` in both normal and operational
thresholds, meaning we can deprecate all controllers for each network in
one call.

## Block Weights

By querying `consts.system.blockWeights` we can see that the
`deprecate_controller_batch` weights fit within the `normal` threshold
on Polkadot.

#### `controller_deprecation_batch` where i = 5900:
#### Ref time: 69,933,325,300
#### Proof size: 21,040,390

### Polkadot 

```
// consts.query.blockWeights

maxBlock: {
        refTime: 2,000,000,000,000
        proofSize: 18,446,744,073,709,551,615
}
normal: {
 maxExtrinsic: {
	refTime: 1,479,873,955,000
	proofSize: 13,650,590,614,545,068,195
 }
 maxTotal: {
	refTime: 1,500,000,000,000
	proofSize: 13,835,058,055,282,163,711
 }
}
```

### Kusama

```
// consts.query.blockWeights

  maxBlock: {
    refTime: 2,000,000,000,000
    proofSize: 18,446,744,073,709,551,615
  }
    normal: {
      maxExtrinsic: {
        refTime: 1,479,875,294,000
        proofSize: 13,650,590,614,545,068,195
      }
      maxTotal: {
        refTime: 1,500,000,000,000
        proofSize: 13,835,058,055,282,163,711
      }
}
```

---------

Co-authored-by: command-bot <>
Co-authored-by: default avatarGonçalo Pestana <[email protected]>
      048a9c27
  27. Dec 11, 2023
  29. Dec 08, 2023
    • Sam Johnson's avatar
      Tasks: general system for recognizing and executing service work (#1343) · ac3f14d2
      Sam Johnson authored 
      `polkadot-sdk` version of original tasks PR located here:
https://github.com/paritytech/substrate/pull/14329

Fixes #206

## Status
- [x] Generic `Task` trait
- [x] `RuntimeTask` aggregated enum, compatible with
`construct_runtime!`
- [x] Casting between `Task` and `RuntimeTask` without needing `dyn` or
`Box`
- [x] Tasks Example pallet
- [x] Runtime tests for Tasks example pallet
- [x] Parsing for task-related macros
- [x] Retrofit parsing to make macros optional
- [x] Expansion for task-related macros
- [x] Adds support for args in tasks
- [x] Retrofit tasks example pallet to use macros instead of manual
syntax
- [x] Weights
- [x] Cleanup
- [x] UI tests
- [x] Docs

## Target Syntax
Adapted from
https://github.com/paritytech/polkadot-sdk/issues/206#issue-1865172283



```rust
// NOTE: this enum is optional and is auto-generated by the other macros if not present
#[pallet::task]
pub enum Task<T: Config> {
    AddNumberIntoTotal {
        i: u32,
    }
}

/// Some running total.
#[pallet::storage]
pub(super) type Total<T: Config<I>, I: 'static = ()> =
StorageValue<_, (u32, u32), ValueQuery>;

/// Numbers to be added into the total.
#[pallet::storage]
pub(super) type Numbers<T: Config<I>, I: 'static = ()> =
StorageMap<_, Twox64Concat, u32, u32, OptionQuery>;

#[pallet::tasks_experimental]
impl<T: Config<I>, I: 'static> Pallet<T, I> {
	/// Add a pair of numbers into the totals and remove them.
	#[pallet::task_list(Numbers::<T, I>::iter_keys())]
	#[pallet::task_condition(|i| Numbers::<T, I>::contains_key(i))]
	#[pallet::task_index(0)]
	pub fn add_number_into_total(i: u32) -> DispatchResult {
		let v = Numbers::<T, I>::take(i).ok_or(Error::<T, I>::NotFound)?;
		Total::<T, I>::mutate(|(total_keys, total_values)| {
			*total_keys += i;
			*total_values += v;
		});
		Ok(())
	}
}
```

---------

Co-authored-by: default avatarNikhil Gupta <[email protected]>
Co-authored-by: default avatarkianenigma <[email protected]>
Co-authored-by: Nikhil Gupta <>
Co-authored-by: default avatarGavin Wood <[email protected]>
Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatargupnik <[email protected]>
      ac3f14d2
  31. Dec 07, 2023
  33. Dec 06, 2023
  35. Dec 05, 2023
  37. Dec 01, 2023
  39. Nov 30, 2023
  41. Nov 29, 2023
  43. Nov 28, 2023
    • Aaro Altonen's avatar
      Rework the event system of `sc-network` (#1370) · e71c484d
      Aaro Altonen authored 
      This commit introduces a new concept called `NotificationService` which
allows Polkadot protocols to communicate with the underlying
notification protocol implementation directly, without routing events
through `NetworkWorker`. This implies that each protocol has its own
service which it uses to communicate with remote peers and that each
`NotificationService` is unique with respect to the underlying
notification protocol, meaning `NotificationService` for the transaction
protocol can only be used to send and receive transaction-related
notifications.

The `NotificationService` concept introduces two additional benefits:
  * allow protocols to start using custom handshakes
  * allow protocols to accept/reject inbound peers

Previously the validation of inbound connections was solely the
responsibility of `ProtocolController`. This caused issues with light
peers and `SyncingEngine` as `ProtocolController` would accept more
peers than `SyncingEngine` could accept which caused peers to have
differing views of their own states. `SyncingEngine` would reject excess
peers but these rejections were not properly communicated to those peers
causing them to assume that they were accepted.

With `NotificationService`, the local handshake is not sent to remote
peer if peer is rejected which allows it to detect that it was rejected.

This commit also deprecates the use of `NetworkEventStream` for all
notification-related events and going forward only DHT events are
provided through `NetworkEventStream`. If protocols wish to follow each
other's events, they must introduce additional abtractions, as is done
for GRANDPA and transactions protocols by following the syncing protocol
through `SyncEventStream`.

Fixes https://github.com/paritytech/polkadot-sdk/issues/512
Fixes https://github.com/paritytech/polkadot-sdk/issues/514
Fixes https://github.com/paritytech/polkadot-sdk/issues/515
Fixes https://github.com/paritytech/polkadot-sdk/issues/554
Fixes https://github.com/paritytech/polkadot-sdk/issues/556

---
These changes are transferred from
https://github.com/paritytech/substrate/pull/14197

 but there are no
functional changes compared to that PR

---------

Co-authored-by: default avatarDmitry Markin <[email protected]>
Co-authored-by: default avatarAlexandru Vasile <[email protected]>
      e71c484d
  45. Nov 22, 2023
  47. Nov 17, 2023
  49. Nov 14, 2023
  51. Nov 13, 2023
    • gupnik's avatar
      Adds syntax for marking calls feeless (#1926) · 60c77a2e
      gupnik authored 
      Fixes https://github.com/paritytech/polkadot-sdk/issues/1725



This PR adds the following changes:
1. An attribute `pallet::feeless_if` that can be optionally attached to
a call like so:
```rust
#[pallet::feeless_if(|_origin: &OriginFor<T>, something: &u32| -> bool {
	*something == 0
})]
pub fn do_something(origin: OriginFor<T>, something: u32) -> DispatchResult {
     ....
}
```
The closure passed accepts references to arguments as specified in the
call fn. It returns a boolean that denotes the conditions required for
this call to be "feeless".

2. A signed extension `SkipCheckIfFeeless<T: SignedExtension>` that
wraps a transaction payment processor such as
`pallet_transaction_payment::ChargeTransactionPayment`. It checks for
all calls annotated with `pallet::feeless_if` to see if the conditions
are met. If so, the wrapped signed extension is not called, essentially
making the call feeless.

In order to use this, you can simply replace your existing signed
extension that manages transaction payment like so:
```diff
- pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
+ pallet_skip_feeless_payment::SkipCheckIfFeeless<
+	Runtime,
+	pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
+ >,
```

### Todo
- [x] Tests
- [x] Docs
- [x] Prdoc

---------

Co-authored-by: Nikhil Gupta <>
Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
Co-authored-by: default avatarLiam Aharon <[email protected]>
      60c77a2e
  53. Nov 06, 2023
  55. Nov 05, 2023
  57. Nov 03, 2023
    • georgepisaltu's avatar
      Identity pallet improvements (#2048) · 21fbc00d
      georgepisaltu authored 
      This PR is a follow up to #1661 

- [x] rename the `simple` module to `legacy`
- [x] fix benchmarks to disregard the number of additional fields
- [x] change the storage deposits to charge per encoded byte of the
identity information instance, removing the need for `fn
additional(&self) -> usize` in `IdentityInformationProvider`
- [x] ~add an extrinsic to rejig deposits to account for the change
above~
- [ ] ~ensure through proper configuration that the new byte-based
deposit is always lower than whatever is reserved now~
- [x] remove `IdentityFields` from the `set_fields` extrinsic signature,
as per [this
discussion](https://github.com/paritytech/polkadot-sdk/pull/1661#discussion_r1371703403)

> ensure through proper configuration that the new byte-based deposit is
always lower than whatever is reserved now

Not sure this is needed anymore. If the new deposits are higher than
what is currently on chain and users don't have enough funds to reserve
what is needed, the extrinisc fails and they're basically grandfathered
and frozen until they add more funds and/or make a change to their
identity. This behavior seems fine to me. Original idea
[here](https://github.com/paritytech/polkadot-sdk/pull/1661#issuecomment-1779606319).

> add an extrinsic to rejig deposits to account for the change above

This was initially implemented but now removed from this PR in favor of
the implementation detailed
[here](https://github.com/paritytech/polkadot-sdk/pull/2088

).

---------

Signed-off-by: default avatargeorgepisaltu <[email protected]>
Co-authored-by: default avatarjoepetrowski <[email protected]>
      21fbc00d
    • Bastian Köcher's avatar
      `sc-block-builder`: Remove `BlockBuilderProvider` (#2099) · ca5f1056
      Bastian Köcher authored 
      The `BlockBuilderProvider` was a trait that was defined in
`sc-block-builder`. The trait was implemented for `Client`. This
basically meant that you needed to import `sc-block-builder` any way to
have access to the block builder. So, this trait was not providing any
real value. This pull request is removing the said trait. Instead of the
trait it introduces a builder for creating a `BlockBuilder`. The builder
currently has the quite fabulous name `BlockBuilderBuilder` (I'm open to
any better name 😅

). The rest of the pull request is about
replacing the old trait with the new builder.

# Downstream code changes

If you used `new_block` or `new_block_at` before you now need to switch
it over to the new `BlockBuilderBuilder` pattern:

```rust
// `new` requires a type that implements `CallApiAt`. 
let mut block_builder = BlockBuilderBuilder::new(client)
                // Then you need to specify the hash of the parent block the block will be build on top of
		.on_parent_block(at)
                // The block builder also needs the block number of the parent block. 
                // Here it is fetched from the given `client` using the `HeaderBackend`
                // However, there also exists `with_parent_block_number` for directly passing the number
		.fetch_parent_block_number(client)
		.unwrap()
                // Enable proof recording if required. This call is optional.
		.enable_proof_recording()
                // Pass the digests. This call is optional.
                .with_inherent_digests(digests)
		.build()
		.expect("Creates new block builder");
```

---------

Co-authored-by: default avatarSebastian Kunert <[email protected]>
Co-authored-by: command-bot <>
      ca5f1056
    • Alexandru Gheorghe's avatar
      substrate: sysinfo: Expose failed hardware requirements (#2144) · dca14239
      Alexandru Gheorghe authored 
      

The check_hardware functions does not give us too much information as to
what is failing, so let's return the list of failed metrics, so that callers can print 
it.

This would make debugging easier, rather than try to guess which
dimension is actually failing.

Signed-off-by: default avatarAlexandru Gheorghe <[email protected]>
      dca14239
  59. Nov 01, 2023
    • Ankan's avatar
      [NPoS] Paging reward payouts in order to scale rewardable nominators (#1189) · 00b85c51
      Ankan authored 
      helps https://github.com/paritytech/polkadot-sdk/issues/439.
closes https://github.com/paritytech/polkadot-sdk/issues/473.

PR link in the older substrate repository:
https://github.com/paritytech/substrate/pull/13498.

# Context
Rewards payout is processed today in a single block and limited to
`MaxNominatorRewardedPerValidator`. This number is currently 512 on both
Kusama and Polkadot.

This PR tries to scale the nominators payout to an unlimited count in a
multi-block fashion. Exposures are stored in pages, with each page
capped to a certain number (`MaxExposurePageSize`). Starting out, this
number would be the same as `MaxNominatorRewardedPerValidator`, but
eventually, this number can be lowered through new runtime upgrades to
limit the rewardeable nominators per dispatched call instruction.

The changes in the PR are backward compatible.

## How payouts would work like after this change
Staking exposes two calls, 1) the existing `payout_stakers` and 2)
`payout_stakers_by_page`.

### payout_stakers
This remains backward compatible with no signature change. If for a
given era a validator has multiple pages, they can call `payout_stakers`
multiple times. The pages are executed in an ascending sequence and the
runtime takes care of preventing double claims.

### payout_stakers_by_page
Very similar to `payout_stakers` but also accepts an extra param
`page_index`. An account can choose to payout rewards only for an
explicitly passed `page_index`.

**Lets look at an example scenario**
Given an active validator on Kusama had 1100 nominators,
`MaxExposurePageSize` set to 512 for Era e. In order to pay out rewards
to all nominators, the caller would need to call `payout_stakers` 3
times.

- `payout_stakers(origin, stash, e)` => will pay the first 512
nominators.
- `payout_stakers(origin, stash, e)` => will pay the second set of 512
nominators.
- `payout_stakers(origin, stash, e)` => will pay the last set of 76
nominators.
...
- `payout_stakers(origin, stash, e)` => calling it the 4th time would
return an error `InvalidPage`.

The above calls can also be replaced by `payout_stakers_by_page` and
passing a `page_index` explicitly.

## Commission note
Validator commission is paid out in chunks across all the pages where
each commission chunk is proportional to the total stake of the current
page. This implies higher the total stake of a page, higher will be the
commission. If all the pages of a validator's single era are paid out,
the sum of commission paid to the validator across all pages should be
equal to what the commission would have been if we had a non-paged
exposure.

### Migration Note
Strictly speaking, we did not need to bump our storage version since
there is no migration of storage in this PR. But it is still useful to
mark a storage upgrade for the following reasons:

- New storage items are introduced in this PR while some older storage
items are deprecated.
- For the next `HistoryDepth` eras, the exposure would be incrementally
migrated to its corresponding paged storage item.
- Runtimes using staking pallet would strictly need to wait at least
`HistoryDepth` eras with current upgraded version (14) for the migration
to complete. At some era `E` such that `E >
era_at_which_V14_gets_into_effect + HistoryDepth`, we will upgrade to
version X which will remove the deprecated storage items.
In other words, it is a strict requirement that E<sub>x</sub> -
E<sub>14</sub> > `HistoryDepth`, where
E<sub>x</sub> = Era at which deprecated storages are removed from
runtime,
E<sub>14</sub> = Era at which runtime is upgraded to version 14.
- For Polkadot and Kusama, there is a [tracker
ticket](https://github.com/paritytech/polkadot-sdk/issues/433) to clean
up the deprecated storage items.

### Storage Changes

#### Added
- ErasStakersOverview
- ClaimedRewards
- ErasStakersPaged

#### Deprecated
The following can be cleaned up after 84 eras which is tracked
[here](https://github.com/paritytech/polkadot-sdk/issues/433).

- ErasStakers.
- ErasStakersClipped.
- StakingLedger.claimed_rewards, renamed to
StakingLedger.legacy_claimed_rewards.

### Config Changes
- Renamed MaxNominatorRewardedPerValidator to MaxExposurePageSize.

### TODO
- [x] Tracker ticket for cleaning up the old code after 84 eras.
- [x] Add companion.
- [x] Redo benchmarks before merge.
- [x] Add Changelog for pallet_staking.
- [x] Pallet should be configurable to enable/disable paged rewards.
- [x] Commission payouts are distributed across pages.
- [x] Review documentation thoroughly.
- [x] Rename `MaxNominatorRewardedPerValidator` ->
`MaxExposurePageSize`.
- [x] NMap for `ErasStakersPaged`.
- [x] Deprecate ErasStakers.
- [x] Integrity tests.

### Followup issues
[Runtime api for deprecated ErasStakers storage
item](https://github.com/paritytech/polkadot-sdk/issues/426

)

---------

Co-authored-by: default avatarJavier Viola <[email protected]>
Co-authored-by: default avatarRoss Bulat <[email protected]>
Co-authored-by: command-bot <>
      00b85c51
  61. Oct 31, 2023
    • Lulu's avatar
      Add ci check for parity-publish and fix current check issues (#1887) · 495d24d7
      Lulu authored 
      

Co-authored-by: default avatarSergejs Kostjucenko <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
      495d24d7
    • Adel Arja's avatar
      1953 defensive testing extrinsic (#1998) · 6e2f94f8
      Adel Arja authored 
      

# Description

The `trigger_defensive` call has been added to the `root-testing`
pallet. The idea is to have this pallet running on `Rococo/Westend` and
use it to verify if the runtime monitoring works end-to-end.

To accomplish this, `trigger_defensive` dispatches an event when it is
called.

Closes #1953

# Checklist

- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

You can remove the "Checklist" section once all have been checked. Thank
you for your contribution!

✄
-----------------------------------------------------------------------------

---------

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      6e2f94f8