Split from #4374

This PR helps to reduce dependencies and align versions, which would
help to move them to workspace dep

Needed for: https://github.com/polkadot-fellows/runtimes/issues/248

This PR introduces a new type `TypeWithDefault<T, D: Get<T>>` to be able
to provide a custom default for any type. This can, then, be used to
provide the nonce type that returns the current block number as the
default, to avoid replay of immortal transactions.

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

When a staker unbonds and withdraws, it is possible that their stash
will contain less currency than the existential deposit. If that
happens, their stash is reaped. But if the existential deposit is zero,
the reap is not triggered. This PR adjusts `pallet_staking` to reap a
stash in the special case that the stash value is zero and the
existential deposit is zero.

This change is important for blockchains built on substrate that require
an existential deposit of zero, becuase it conserves valued storage
space.

There are two places in which ledgers are checked to determine if their
value is less than the existential deposit and they should be reaped: in
the methods `do_withdraw_unbonded` and `reap_stash`. When the check is
made, the condition `ledger_total == 0` is also checked. If
`ledger_total` is zero, then it must be below any existential deposit
greater than zero and equal to an existential deposit of 0.

I added a new test for each method to confirm the change behaves as
expected.

Closes https://github.com/paritytech/polkadot-sdk/issues/4340

---------

Co-authored-by: command-bot <>

Related to
https://github.com/paritytech/parity-bridges-common/issues/2962
Relay companion:
https://github.com/paritytech/parity-bridges-common/pull/2978

Example usage:
```
./target/release/substrate-relay relay-parachain-head rococo-to-bridge-hub-westend \
    --source-host localhost --source-port 9942 \
    --target-host localhost --target-port 8945 --target-signer //Alice \
    --at-relay-block 61
```

Due to recent bump of Rococo/Westend versions + the fact that
https://github.com/paritytech/parity-bridges-common/pull/2894 has
finally reached this repo, tests now fail, because we've started
checking all client versions (even source) unless we specify
`--source-version-mode Auto` in CLI arguments. This looks like an
overkill, but all those version checks will be fixed by
https://github.com/paritytech/polkadot-sdk/pull/4256, so now it makes
sense just to add this CLI option. We also need to propagate it to
running relays eventually.

cc https://github.com/paritytech/ci_cd/issues/939

Allow for 0 existential deposit in benchmarks for `pallet_staking`, `pallet_session`, and `pallet_balances` (#4346)

This PR ensures non-zero values are available in benchmarks for
`pallet_staking`, `pallet_session`, and `pallet_balances` where required
for them to run.

This small change makes it possible to run the benchmarks for
`pallet_staking`, `pallet_session`, and `pallet_balances` in a runtime
for which existential deposit is set to 0.

The benchmarks for `pallet_staking` and `pallet_session` will still fail
in runtimes that use `U128CurrencyToVote`, but that is easy to work
around by creating a new `CurrencyToVote` implementation for
benchmarking.

The changes are implemented by checking if existential deposit equals 0
and using 1 if so.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

The native executor is deprecated and downstream users should stop using
it.

This PR:
- changes `xcm::v4` to `xcm::prelude` imports for coretime stuff
- changes `query_acceptable_payment_assets` /
`query_weight_to_asset_fee` implementations to be more resilient to the
XCM version change
- adds `xcm_fee_payment_runtime_api::XcmPaymentApi` to the
AssetHubRococo/Westend exposing a native token as acceptable payment
asset

Continuation of: https://github.com/paritytech/polkadot-sdk/pull/3607

Closes: https://github.com/paritytech/polkadot-sdk/issues/4297

## Possible follow-ups

- [ ] add all sufficient assets (`Assets`, `ForeignAssets`) as
acceptable payment assets ?

This crate uses this feature however it appears to still work without
this feature enabled. I believe this is due to feature unification of
the workspace. Some other crate enables this feature so it also ends up
enabled here. But when this crate is pushed to crates.io and compiled
individualy it fails to compile.

use `server-core` feature instead of `server` feature when defining the
rpc api

This adds a small guide on how to prepare your parachain for pov-reclaim
usage.

This PR updates the litep2p crate to the latest version.

This fixes the build for developers that want to perform `cargo update`
on all their dependencies:
https://github.com/paritytech/polkadot-sdk/pull/4343, by porting the
latest changes.

The peer records were introduced to litep2p to be able to distinguish
and update peers with outdated records.
It is going to be properly used in substrate via:
https://github.com/paritytech/polkadot-sdk/pull/3786, however that is
pending the commit to merge on litep2p master:
https://github.com/paritytech/litep2p/pull/96.

Closes: https://github.com/paritytech/polkadot-sdk/pull/4343



---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

see https://github.com/paritytech/parity-bridges-common/issues/2974 : we
still need to support unupgraded chains (BHK and BHP) in relay

We may need to revert this change when all chains are upgraded

Closes #4272.

It turned out that not only the node part but also the runtime wasn't
async backing ready. Now, both of them are using proper APIs and
producing 6-second blocks out of the box.

This PR backports version bumps and reorganization of the `prdocs` from
`1.11.0` release branch back to master

HRMP - set `DefaultChannelSizeAndCapacityWithSystem` with dynamic values according to the `ActiveConfig` (#4332)

## Summary
This PR enhances the capability to set
`DefaultChannelSizeAndCapacityWithSystem` for HRMP. Currently, all
testnets (Rococo, Westend) have a hard-coded value set as 'half of the
maximum' determined by the live `ActiveConfig`. While this approach
appears satisfactory, potential issues could arise if the live
`ActiveConfig` are adjusted below these hard-coded values, necessitating
a new runtime release with updated values. Additionally, hard-coded
values have consequences, such as Rococo's benchmarks not functioning:
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6082656.

## Solution
The proposed solution here is to utilize
`ActiveConfigHrmpChannelSizeAndCapacityRatio`, which reads the current
`ActiveConfig` and calculates `DefaultChannelSizeAndCapacityWithSystem`,
for example, "half of the maximum" based on live data. This way,
whenever `ActiveConfig` is modified,
`ActiveConfigHrmpChannelSizeAndCapacityRatio` automatically returns
adjusted values with the appropriate ratio. Thus, manual adjustments and
new runtime releases become unnecessary.


Relates to a comment/discussion:
https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1541001420
Relates to a comment/discussion:
https://github.com/paritytech/polkadot-sdk/pull/3721/files#r1549291588

---------

Co-authored-by: command-bot <>

- [x] Drop requests from a PeerID that is already being served by us.
- [x] Don't sent requests to a PeerID if we already are requesting
something from them at that moment (prioritise other requests or wait).
- [x] Tests
- [ ] ~~Add a small rep update for unsolicited requests (same peer
request)~~ not included in original PR due to potential issues with
nodes slowly updating
- [x] Add a metric to track the amount of dropped requests due to peer
rate limiting
- [x] Add a metric to track how many time a node reaches the max
parallel requests limit in v2+

Helps with but does not close yet:
https://github.com/paritytech-secops/srlabs_findings/issues/303

revert some reverted changes from #4266

Related to https://github.com/paritytech/polkadot-sdk/pull/1903

For #1903 we will need to add a Fisherman struct. This PR:
- defines a basic version of `Fisherman` and moves into it the logic
that we have now for reporting double voting equivocations
- splits the logic for generating the key ownership proofs into a more
generic separate method
- renames `EquivocationProof` to `DoubleVotingProof` since later we will
introduce a new type of equivocation

The PR doesn't contain any functional changes

This PR makes a few improvements in the docs for the minimal template.

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Closes #4308

Polkadot address: 12gMhxHw8QjEwLQvnqsmMVY1z5gFa54vND74aMUbhhwN6mJR

---------

Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>

This PR does a "developer experience" refactor of the XCM Simulator
Example.

I was looking for existing code / documentation where developers could
better learn about working with and configuring XCM.

The XCM Simulator was a natural starting point due to the fact that it
can emulate end to end XCM scenarios, without needing to spawn multiple
real chains.

However, the XCM Simulator Example was just 3 giant files with a ton of
configurations, runtime, pallets, and tests mashed together.

This PR breaks down the XCM Simulator Example in a way that I believe is
more approachable by a new developer who is looking to navigate the
various components of the end to end example, and modify it themselves.

The basic structure is:

- xcm simulator example
    - lib (tries to only use the xcm simulator macros)
    - tests
    - relay-chain
        - mod (basic runtime that developers should be familiar with)
        - xcm-config
            - mod (contains the `XcmConfig` type
            - various files for each custom configuration  
    - parachain
        - mock_msg_queue (custom pallet for simulator example)
        - mod (basic runtime that developers should be familiar with)
        - xcm-config
            - mod (contains the `XcmConfig` type
            - various files for each custom configuration

I would like to add more documentation to this too, but I think this is
a first step to be accepted which will affect how documentation is added
to the example

---------

Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Related to https://github.com/paritytech/polkadot-sdk/pull/3905.

Since virtual stakers does not have a real balance, they should not be
allowed to be reaped.

The proper reaping for agents slashed will be done in a separate PR.

# ISSUE
- Link to issue: https://github.com/paritytech/polkadot-sdk/issues/4237

# DESCRIPTION
Remove all ModuleError with hard-coded indices to pallet Error. For
example:
```rs
Err(DispatchError::Module(ModuleError {
	index: 4,
	error: [2, 0, 0, 0],
	message: Some("Filtered")
}))
```
To 
```rs
let expected_result = Err(crate::Error::<Test>::Filtered.into());
assert_eq!(result, expected_result);
```
# TEST OUTCOME
```
test result: ok. 74 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
```

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Bumps [snow](https://github.com/mcginty/snow) from 0.9.3 to 0.9.6.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/mcginty/snow/releases">snow's
releases</a>.</em></p>
<blockquote>
<h2>v0.9.6</h2>
<ul>
<li>Validate invalid PSK positions when building a Noise protocol.</li>
<li>Raise errors in various typos/mistakes in Noise patterns when
parsing.</li>
<li>Deprecate the <code>sodiumoxide</code> backend, as that crate is no
longer maintained. We may eventually migrate it to a maintaned version
of the crate, but for now it's best to warn users.</li>
<li>Set a hard limit in <code>read_message()</code> in transport mode to
65535 to be fully compliant with the Noise specification.</li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/mcginty/snow/compare/v0.9.5...v0.9.6">https://github.com/mcginty/snow/compare/v0.9.5...v0.9.6</a></p>
<h2>v0.9.5</h2>
<p>This is a security release that fixes a logic flaw in decryption in
<code>TransportState</code> (i.e. the stateful one), where the nonce
could increase even when decryption failed, which can cause a desync
between the sender and receiver, opening this up as a denial of service
vector if the attacker has the ability to inject packets in the channel
Noise is talking over.</p>
<p>More details can be found in the advisory: <a
href="https://github.com/mcginty/snow/security/advisories/GHSA-7g9j-g5jg-3vv3">https://github.com/mcginty/snow/security/advisories/GHSA-7g9j-g5jg-3vv3</a></p>
<p>All users are encouraged to update.</p>
<h2>v0.9.4</h2>
<p>This is a dependency version bump release because a couple of
important dependencies released new versions that needed a
<code>Cargo.toml</code> bump:</p>
<ul>
<li><code>ring</code> 0.17</li>
<li><code>pqcrypto-kyber</code> 0.8</li>
<li><code>aes-gcm</code> 0.10</li>
<li><code>chacha20poly1305</code> 0.10</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/mcginty/snow/commit/a4be73faa042c5967f39662aa66919f774831a9a"><code>a4be73f</code></a>
meta: v0.9.6 release</li>
<li><a
href="https://github.com/mcginty/snow/commit/9e53dcf5bbea869b5e3e9ed26866d683906bc848"><code>9e53dcf</code></a>
TransportState: limit read_message size to 65535</li>
<li><a
href="https://github.com/mcginty/snow/commit/faf05609e19f4106cd47b78123415dfeb9330861"><code>faf0560</code></a>
Deprecate sodiumoxide resolver</li>
<li><a
href="https://github.com/mcginty/snow/commit/308a24d23da13cb01a173f0ec23f140898801fb9"><code>308a24d</code></a>
Add warnings about multiple calls to same method in Builder</li>
<li><a
href="https://github.com/mcginty/snow/commit/f280991ae408685d72e098545314f2be160e57f9"><code>f280991</code></a>
Error when extraneous parameters are included in string to parse</li>
<li><a
href="https://github.com/mcginty/snow/commit/dbdcc4803aae6e5d9910163a7d52e0df8def4310"><code>dbdcc48</code></a>
Error on duplicate modifiers in parameter string</li>
<li><a
href="https://github.com/mcginty/snow/commit/8b1a819c93ceae98f9ba0a1be192fa61fdec78c2"><code>8b1a819</code></a>
Validate PSK index in pattern to avoid panic</li>
<li><a
href="https://github.com/mcginty/snow/commit/74e30cf591d6d89c8a1670ee713ecc4e9607e38f"><code>74e30cf</code></a>
meta: v0.9.5 release</li>
<li><a
href="https://github.com/mcginty/snow/commit/12e8ae55547ae297d5f70599e5c884ea891303eb"><code>12e8ae5</code></a>
Stateful nonce desync fix</li>
<li><a
href="https://github.com/mcginty/snow/commit/02c26b7551cb7e221792a9d3d3a94730e6a34e8a"><code>02c26b7</code></a>
Remove clap from simple example</li>
<li>Additional commits viewable in <a
href="https://github.com/mcginty/snow/compare/v0.9.3...v0.9.6">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=snow&package-manager=cargo&previous-version=0.9.3&new-version=0.9.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/paritytech/polkadot-sdk/network/alerts

).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Some traits are already included in the 2021 prelude and so shouldn't be
needed to use explicitly:

use `convert::TryFrom`, `convert::TryInto`, and `iter::FromIterator` are
removed.

( https://doc.rust-lang.org/core/prelude/rust_2021/ )

No breaking changes or change of functionality, so I think no PR doc is
needed in this case.

(Motivation: Removes some references to `sp-std`)

closes https://github.com/paritytech/polkadot-sdk/issues/426.
related to https://github.com/paritytech/polkadot-sdk/pull/1189.

Would help offchain programs to query if there are unclaimed pages of
rewards for a given era.

The logic could look like below

```js
// loop as long as all era pages are claimed.
while (api.call.stakingApi.pendingRewards(era, validator_stash)) {
  api.tx.staking.payout_stakers(validator_stash, era)
}
```

Just the upper-stream for
https://github.com/Snowfork/polkadot-sdk/pull/137

 and more context
there.

---------

Co-authored-by: Clara van Staden <claravanstaden64@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

Closes https://github.com/paritytech/polkadot-sdk/issues/1966,
https://github.com/paritytech/polkadot-sdk/issues/1963 and
https://github.com/paritytech/polkadot-sdk/issues/1962.

Disabling strategy specification
[here](https://github.com/paritytech/polkadot-sdk/pull/2955). (Updated
13/02/2024)

Implements:
* validator disabling for a whole era instead of just a session
* no more than 1/3 of the validators in the active set are disabled
Removes:
* `DisableStrategy` enum - now each validator committing an offence is
disabled.
* New era is not forced if too many validators are disabled.

Before this PR not all offenders were disabled. A decision was made
based on [`enum
DisableStrategy`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/primitives/staking/src/offence.rs#L54).
Some offenders were disabled for a whole era, some just for a session,
some were not disabled at all.

This PR changes the disabling behaviour. Now a validator committing an
offense is disabled immediately till the end of the current era.

Some implementation notes:
* `OffendingValidators` in pallet session keeps all offenders (this is
not changed). However its type is changed from `Vec<(u32, bool)>` to
`Vec<u32>`. The reason is simple - each offender is getting disabled so
the bool doesn't make sense anymore.
* When a validator is disabled it is first added to
`OffendingValidators` and then to `DisabledValidators`. This is done in
[`add_offending_validator`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/slashing.rs#L325)
from staking pallet.
* In
[`rotate_session`](https://github.com/paritytech/polkadot-sdk/blob/bdbe98297032e21a553bf191c530690b1d591405/substrate/frame/session/src/lib.rs#L623)
the `end_session` also calls
[`end_era`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/pallet/impls.rs#L490)
when an era ends. In this case `OffendingValidators` are cleared
**(1)**.
* Then in
[`rotate_session`](https://github.com/paritytech/polkadot-sdk/blob/bdbe98297032e21a553bf191c530690b1d591405/substrate/frame/session/src/lib.rs#L623)
`DisabledValidators` are cleared **(2)**
* And finally (still in `rotate_session`) a call to
[`start_session`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/pallet/impls.rs#L430

)
repopulates the disabled validators **(3)**.
* The reason for this complication is that session pallet knows nothing
abut eras. To overcome this on each new session the disabled list is
repopulated (points 2 and 3). Staking pallet knows when a new era starts
so with point 1 it ensures that the offenders list is cleared.

---------

Co-authored-by: ordian <noreply@reusable.software>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>

Currently we discard errors content
We should at least log it.

Code now is more similar to what is written in try_on_runtime_upgrade.

label should be R0

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Javier Bullrich <javier@bullrich.dev>

related to
https://github.com/paritytech/parity-bridges-common/issues/2962


silent, because the actual code for subcommand is added in the
`parity-bridges-common` repo, where binary lives

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

This PR includes two changes:
- added `workflow_dispatch` to review bot
- reverted #4271

### Added `workflow_dispatch` to review bot
This allows us, in the case that review-bot fails for some fork reasons,
to trigger it manually ensuring that we can overcame the problem with
the multiple actions while we look for a solution.

<img width="342" alt="image"
src="https://github.com/paritytech/polkadot-sdk/assets/8524599/f432f91b-829a-4da4-b4ca-54cc4fe280c8">

### Reverted #4271
Unfortunately, the changes added in #4271 do not work in forks.

Here is a lengthy discussion of many individuals facing the same problem
as me:
- [GitHub Action `pull_request` attribute empty in `workflow_run` event
object for PR from forked repo
#25220](https://github.com/orgs/community/discussions/25220)

So I had to revert it (but I updated the dependencies to latest).

#### Miscellaneous changes

I added a debug log at the end of review bot in case it fails so we can
easily debug it without having to make a lot of boilerplate and forks to
duplicate the environment.

TODOs:
- [x] wait and see if test `1` works;
- [x] ~think of whether we need remaining tests.~ I think we should keep
it - will try to revive and update it

There are some accounts that do not have a consumer ref while having a
reserve.
This adds a fail-safe mechanism to trigger in the case that
`does_consume` is true, but the assumption of `consumer>0` is not.

This should prevent those accounts from loosing balance and the TI from
getting messed up even more, but is not an "ideal" fix. TBH an ideal fix
is not possible, since on-chain data is in an invalid state.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

The solochain template links to parachain template instead of solochain.