..without needing to provide your own `newtype` around it.  

This will allow `type Reason = u8` to be used as `FreezeReason` and
`HoldReason`, which I think is a nice simplification if one doens't want
to deal with the complications.

At the same time, it is a bit of an anti-pattern.

Putting it out there to check people's vibes.

fix #3631 

Types which are impacted and fixed here are `ItemTip`,
`PriceWithDirection`, `PreSignedMint`, `PreSignedAttributes`.

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

Update Identity pallet README.md according to the up-to-date docs,
particularly to explain the _username_ concept of the pallet.

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

closes https://github.com/paritytech-secops/srlabs_findings/issues/408.
This fixes how ProxyDelegator accounts are derived but may cause issues
in Westend since it would use the old derivative accounts. Does not
affect Polkadot/Kusama as this pallet is not deployed to them yet.

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>

## Context
Pool members using the old `TransferStake` strategy were able to
transfer all their funds to the pool. With `DelegateStake` changes, we
want to ensure similar behaviour by allowing members to delegate all
their stake to the pool.

## Changes
- Ensure all the balance including ED of an account can be delegated
(and used in the pool) by adding a provider for delegators.
- Gates calls that mutates the pool or pool member if they are in
unmigrated state. Closes
https://github.com/paritytech-secops/srlabs_findings/issues/409.
- Adds remote test to migrate all pools and members to `DelegateStake`
which can be used with `Kusama` and `Polkadot` runtime state. closes
https://github.com/paritytech/polkadot-sdk/issues/4629.
- Add new runtime apis to read pool and member balance.

## Addressing possible migration errors 
Pool members migrating can run into two types of errors:
- Already Staking: If the pool member is already staking, we cannot
migrate them to `DelegateStake` since this may mean they are able to use
the same staked funds in the pool. Users would need to withdraw all
their funds from staking, in order to migrate their pool funds.
- Pool contribution below ED: For these cases transfer from pool account
to member account would fail. The affected users can top up their
accounts and redo migration.

Another error that was earlier possible was when member's free balance
is below ED. This PR adds a provider to delegator allowing all user
balance including ED can be contributed towards the pool. This helps
`1095` accounts in Polkadot and `41` accounts in Kusama to migrate now
which would have earlier failed.

## Results from RemoteExternalities Tests.

### Kusama
`Migration stats: success: 3017, direct_stakers: 361, unexpected_errors:
0`

### Polkadot
`Migration stats: success: 42859, direct_stakers: 643,
unexpected_errors: 0`

## TODO
- [x] Add runtime api for member total balance.
- [x] New
[issue](https://github.com/paritytech/polkadot-sdk/issues/5009) to reap
pool members with contribution below ED.
- [x] Add provider for delegators so whole balance including ED can be
held while contributing to pools.
- [x] Gate all pool extrinsics if pool/member is in non-migrated state.

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>

The same exact `matches!()` was duplicated in
`Configuration::no_genesis()` method and inline in full node parts
creation. Since this is the same exact logic and reason, it makes sense
to de-duplicate them.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Related to #4523 

This PR adds benchmarks for `report_fork_voting()`.

**Important: Even though the benchmarks are now available, we still use
`Weight::MAX`. That's because I realized while working on this PR that
there's still one missing piece. We should also check that the ancestry
proof is optimal. I plan to do this in a future PR, hopefully the last
one related to #4523.**

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: command-bot <>

Make ticket non-optional and add ensure_successful method to
Consideration trait.

Reverts the optional return ticket type for the new function introduced
in
[polkadot-sdk/4596](https://github.com/paritytech/polkadot-sdk/pull/4596)
and adds a helper `ensure_successful` function for the runtime
benchmarks.
Since the existing FRAME pallet represents zero cost with a zero balance
rather than `None` in an option, maintaining the ticket type as a
non-optional balance is beneficial for backward compatibility and helps
avoid unnecessary migrations.

In the move from XCMv3 to XCMv4, the `AssetId` for `ForeignAssets` in
`asset-hub-rococo` and `asset-hub-westend` was left as `v3::Location` to
be later migrated to `v4::Location`.

This is that migration PR.

Because the encoding of `v3::Location` and `v4::Location` is the same,
we don't need to do any data migration, the keys will still be
decodable.
The [original idea by
Jan](https://github.com/paritytech/polkadot/pull/7236) was to make the
v4 changes in v3 since the ABI (the encoding/decoding) didn't change.
Corroborated the ABI is the same iterating over all storage, the code is
on [another
branch](https://github.com/paritytech/polkadot-sdk/blob/cisco-assert-v3-v4-encodings-equal/cumulus/parachains/runtimes/assets/migrations/src/foreign_assets_to_v4/mod.rs).

We will need a data migration when we want to update from `v4::Location`
to `v5::Location` because of [the accepted RFC changing the NetworkId
enum](https://github.com/polkadot-fellows/RFCs/pull/108).
I'll configure MBMs (Multi-Block Migrations) then and make the actual
migration.

Fixes https://github.com/paritytech/polkadot-sdk/issues/4128

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>

This PR performs minor code cleanup to reduce verbosity. Since the
compiler has already optimized out indirect calls in the existing code,
these changes improve readability but do not affect performance.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Resolves https://github.com/paritytech-secops/srlabs_findings/issues/412

## Changes
- Clear any dangling delegation when member is removed.
- Agents need to be killed explicitly when pools are destroyed.
- Member withdraw amount is max of their locked funds and the value of
their points.

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: command-bot <>

This PR adds an additional defensive check to the reclaim SE. 

Since it can happen that we miss some storage accesses on other SEs
pre-dispatch, we should double check
that the bookkeeping of the runtime stays ahead of the node-side
pov-size.

If we discover a mismatch and the node-side pov-size is indeed higher,
we should set the runtime bookkeeping to the node-side value. In cases
such as #5229, we would stop including extrinsics and not run `on_idle`
at least.

cc @gui1117

---------

Co-authored-by: command-bot <>

Hello
I found several spelling errors.
Br, Elias.

Trivial leftover from
https://github.com/paritytech/polkadot-sdk/pull/4844

Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

Added minor clarification on the genesis config patch
([link](https://substrate.stackexchange.com/questions/11813/in-the-genesis-config-what-does-the-patch-key-do/11825#11825))

---------

Co-authored-by: command-bot <>

https://github.com/paritytech/polkadot-sdk/pull/4527/files#r1706673828

Since `May 2023` after
https://github.com/paritytech/substrate/pull/13548 optimization,
`Blake2256` is faster with about 30%, that means that there is a
difference of ~30% between the benchmark values we ask validators to run
against and the machine we use for generating the weights.So if all
validators, just barely pass the benchmarks our weights are potentially
underestimated with about ~20%, so let's bring this two in sync.

Same thing happened when we merged
https://github.com/paritytech/polkadot-sdk/pull/2524 in `Nov 2023`
SR25519-Verify became faster with about 10-15%

## Results

Generated on machine from here:
https://github.com/paritytech/devops/pull/3210
```
+----------+----------------+--------------+-------------+-------------------+
| Category | Function       | Score        | Minimum     | Result            |
+============================================================================+
| CPU      | BLAKE2-256     | 1...

Follow-up to https://github.com/paritytech/polkadot-sdk/pull/4457, looks
like more things were missing

---------

Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>

This PR shows a warning when the `--public-addr` is not provided for
validators.

In the future, we'll transform this warning into a hard failure.
Validators are encouraged to provide this parameter for better
availability over the network.

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

I propose to have `ProofSizeExt` available during benchmarking so we can
improve the accuracy for extensions using it.

Another thing we could do is to also enable recording for the timing
benchmark here:

https://github.com/paritytech/polkadot-sdk/blob/035211d7/substrate/utils/frame/benchmarking-cli/src/pallet/command.rs#L232

Parachains will need to have recording enabled during import for
reclaim, so we could enable it here and provide a flag
`--disable-proof-recording` for scenarios where one does not want it.
Happy to hear opinions about this.

Closes #4517

Polkadot address: 12gMhxHw8QjEwLQvnqsmMVY1z5gFa54vND74aMUbhhwN6mJR

---------

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

Test currently failing, therefore improving to include a file from the
same crate to not trip up the caching.

R0 silent since this is only modifying unpublished crates.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Dónal Murray <donal.murray@parity.io>

[pallet_contracts] Increase the weight of the deposit_event host function to limit the memory used by events. (#4973)

This PR updates the weight of the `deposit_event` host function by
adding
a fixed ref_time of 60,000 picoseconds per byte. Given a block time of 2
seconds
and this specified ref_time, the total allocation size is 32MB.

---------

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

This PR adds functionality that allows tasks to enable auto-renewal.
Each task eligible for renewal can enable auto-renewal.

A new storage value is added to track all the cores with auto-renewal
enabled and the associated task running on the core. The `BoundedVec` is
sorted by `CoreIndex` to make disabling auto-renewal more efficient.

Cores are renewed at the start of a new bulk sale. If auto-renewal
fails(e.g. due to the sovereign account of the task not holding
sufficient balance), an event will be emitted, and the renewal will
continue for the other cores.

The two added extrinsics are:
- `enable_auto_renew`: Extrinsic for enabling auto renewal.
- `disable_auto_renew`: Extrinsic for disabling auto renewal.

TODOs:
- [x] Write benchmarks for the newly added extrinsics.

Closes: #4351

---------

Co-authored-by: Dónal Murray <donalm@seadanda.dev>

network/strategy: Backoff and ban overloaded peers to avoid submitting the same request multiple times (#5029)

This PR avoids submitting the same block or state request multiple times
to the same slow peer.

Previously, we submitted the same request to the same slow peer, which
resulted in reputation bans on the slow peer side.
Furthermore, the strategy selected the same slow peer multiple times to
submit queries to, although a better candidate may exist.

Instead, in this PR we:
- introduce a `DisconnectedPeers` via LRU with 512 peer capacity to only
track the state of disconnected peers with a request in flight
- when the `DisconnectedPeers` detects a peer disconnected with a
request in flight, the peer is backed off
  - on the first disconnection: 60 seconds
  - on second disconnection: 120 seconds
- on the third disconnection the peer is banned, and the peer remains
banned until the peerstore decays its reputation
  
This PR lifts the pressure from overloaded nodes that cannot process
requests in due time.
And if a peer is detected to be slow after backoffs, the peer is banned.

Theoretically, submitting the same request multiple times can still
happen when:
- (a) we backoff and ban the peer 
- (b) the network does not discover other peers -- this may also be a
test net
- (c) the peer gets reconnected after the reputation decay and is still
slow to respond



Aims to improve:
- https://github.com/paritytech/polkadot-sdk/issues/4924
- https://github.com/paritytech/polkadot-sdk/issues/531

Next Steps:
- Investigate the network after this is deployed, possibly bumping the
keep-alive timeout or seeing if there's something else misbehaving




This PR builds on top of:
- https://github.com/paritytech/polkadot-sdk/pull/4987


### Testing Done
- Added a couple of unit tests where test harness were set in place

- Local testnet

```bash
13:13:25.102 DEBUG tokio-runtime-worker sync::persistent_peer_state: Added first time peer 12D3KooWHdiAxVd8uMQR1hGWXccidmfCwLqcMpGwR6QcTP6QRMuD

13:14:39.102 DEBUG tokio-runtime-worker sync::persistent_peer_state: Remove known peer 12D3KooWHdiAxVd8uMQR1hGWXccidmfCwLqcMpGwR6QcTP6QRMuD state: DisconnectedPeerState { num_disconnects: 2, last_disconnect: Instant { tv_sec: 93355, tv_nsec: 942016062 } }, should ban: false

13:16:49.107 DEBUG tokio-runtime-worker sync::persistent_peer_state: Remove known peer 12D3KooWHdiAxVd8uMQR1hGWXccidmfCwLqcMpGwR6QcTP6QRMuD state: DisconnectedPeerState { num_disconnects: 3, last_disconnect: Instant { tv_sec: 93485, tv_nsec: 947551051 } }, should ban: true

13:16:49.108  WARN tokio-runtime-worker peerset: Report 12D3KooWHdiAxVd8uMQR1hGWXccidmfCwLqcMpGwR6QcTP6QRMuD: -2147483648 to -2147483648. Reason: Slow peer after backoffs. Banned, disconnecting.
```

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

While working on #5129 I noticed that after warp sync, nodes would
print:
```
2024-07-29 17:59:23.898 ERROR ⋮beefy: 🥩 Error: ConsensusReset. Restarting voter.    
```

After some debugging I found that we enter the following loop:
1. Wait for beefy pallet to be available: Pallet is detected available
directly after warp sync since we are at the tip.
2. Wait for headers from tip to beefy genesis to be available: During
this time we don't process finality notifications, since we later want
to inspect all the headers for authority set changes.
3. Gap sync finishes, route to beefy genesis is available.
4. The worker starts acting, tries to fetch beefy genesis block. It
fails, since we are acting on old finality notifications where the state
is already pruned.
5. Whole beefy subsystem is being restarted, loading the state from db
again and iterating a lot of headers.

This already happened before #5129.

This PR enables the `chainSpec_v1` class for the polkadot-parachian. 
The chainSpec is part of the rpc-v2 which is spec-ed at:
https://github.com/paritytech/json-rpc-interface-spec/blob/main/src/api/chainSpec.md.

This also paves the way for enabling a future `chainSpec_unstable_spec`
on all nodes.

Closes: https://github.com/paritytech/polkadot-sdk/issues/5191

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

Added a new adapter to xcm-builder, the `SingleAssetExchangeAdapter`.
This adapter makes it easy to use `pallet-asset-conversion` for
configuring the `AssetExchanger` XCM config item.

I also took the liberty of adding a new function to the `AssetExchange`
trait, with the following signature:

```rust
fn quote_exchange_price(give: &Assets, want: &Assets, maximal: bool) -> Option<Assets>;
```

The signature is meant to be fairly symmetric to that of
`exchange_asset`.
The way they interact can be seen in the doc comment for it in the
`AssetExchange` trait.

This is a breaking change but is needed for
https://github.com/paritytech/polkadot-sdk/pull/5131.
Another idea is to create a new trait for this but that would require
setting it in the XCM config which is also breaking.

Old PR: https://github.com/paritytech/polkadot-sdk/pull/4375.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

This PR replaces env_logger with sp_tracing because of an issue with
env_logger and gum #4660

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Andrei Eres <eresav@me.com>

This PR reduces the occurrences for identified observed addresses.

Litep2p discovers its external addresses by inspecting the
`IdentifyInfo::ObservedAddress` field reported by other peers.
After we get 5 confirmations of the same external observed address (the
address the peer dialed to reach us), the address is reported through
the network layer.

The PR effectively changes this from 5 to 2.
This has a subtle implication on freshly started nodes for the
authority-discovery discussed below.

The PR also makes the authority discovery a bit more robust by not
publishing records if the node doesn't have addresses yet to report.
This aims to fix a scenario where:
- the litep2p node has started, it has some pending observed addresses
but less than 5
- the authorit-discovery publishes a record, but at this time the node
doesn't have any addresses discovered and the record is published
without addresses -> this means other nodes w...

The test name is `test-frame-ui` I don't know if I can also change it to
`test-ui` without breaking other stuff. So I kept the name unchanged.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

As per #3326, removes usage of the `pallet::getter` macro from the
balances pallet. The syntax `StorageItem::<T, I>::get()` should be used
instead.

Also, adds public functions for compatibility.

cc @muraca

polkadot address: 5GsLutpKjbzsbTphebs9Uy4YK6gTN47MAaz6njPktidjR5cp

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

This PR reverts the removal of an [`if`
statement](https://github.com/paritytech/polkadot-sdk/commit/7ecf3f75#diff-8bf31ba8d9ebd6377983fd7ecc7f4e41cb1478a600db1a15a578d1ae0e8ed435L370)
merged recently, which affected test output verbosity of several pallets
(e.g. staking, EPM, and potentially others).

More generally, the `UnixTime::now` implementation of the timestamp
pallet should log an error *only* when called at the genesis block.

ISSUE
Link to the issue:
https://github.com/paritytech/polkadot-sdk/issues/3326

Deliverables

[Deprecation] remove pallet::getter usage from pallet-proxy

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

As per #3326, removes pallet::getter macro usage from the
election-provider-multi-phase pallet. The syntax `StorageItem::<T,
I>::get()` should be used instead.

cc @muraca

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Add the possibility of injecting session-keys in genesis for
non-validators. Currently all keys injected in genesis were considered
as part of the initial validators set, this PR allows to inject a new
vector with non-authority keys

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

An attempt to improve [the
docs](https://paritytech.github.io/polkadot-sdk/master/polkadot_sdk_docs/index.html)
by applying various corrections:

- grammar/stylistics,
- formatting,
- broken links,
- broken markdown table,
- outdated vscode setting name,
- typos,
- consistency,
- etc.

Part of https://github.com/paritytech/eng-automation/issues/10