Enables the `request_revenue` and `notify_revenue` parts of [RFC 5 -
Coretime
Interface](https://polkadot-fellows.github.io/RFCs/approved/0005-coretime-interface.html)

TODO:
- [x] Finish first pass at implementation
- [x] ~~Need to explicitly burn uncollected and dropped revenue~~
Accumulate it instead
- [x] Confirm working on zombienet
- [x] Tests 
- [ ] Enable XCM `request_revenue` sending on Coretime chain on Kusama
and Polkadot

Fixes: #2209

---------

Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

This PR upgrades `litep2p` to the latest version and includes the two
fixes:

1. Enables incoming DHT record validation with `litep2p` network
backend.
2. Sets `TCP_NODELAY` flag on TCP & WS sockets in `litep2p` backend, as
it is currently done in `libp2p` backend.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Add a `promote_fast` extrinsic to the `core-fellowship` pallet to allow
promotions that ignore the promotion cooldown. It comes with a new
`FastPromoteOrigin`.

Supersedes https://github.com/paritytech/polkadot-sdk/pull/4778

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: command-bot <>

Closes: https://github.com/paritytech/polkadot-sdk/issues/4298

This PR also merges `xcm-fee-payment-runtime-api` module to the
`xcm-runtime-api`.


## TODO

- [x] rename `convert` to `convert_location` and add new one
`convert_account` (opposite direction)
- [x] add to the all testnet runtimes
- [x] check polkadot-js if supports that automatically or if needs to be
added manually https://github.com/polkadot-js/api/pull/5917
- [ ] backport/patch for fellows and release (asap)

## Open questions
- [x] should we merge `xcm-runtime-api` and
`xcm-fee-payment-runtime-api` to the one module `xcm-runtime-api` ?

## Usage
Input:
 - `location:  VersionedLocation`
 
Output:
 - account_id bytes


![image](https://github.com/paritytech/polkadot-sdk/assets/8159517/4607b15a-77d2-462b-806c-606107776c0d)

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

This is PR updates jsonrpsee v0.23 which mainly changes:
- Add `Extensions` which we now is using to get the connection id (used
by the rpc spec v2 impl)
- Update hyper to v1.0, http v1.0, soketto and related crates
(hyper::service::make_service_fn is removed)
- The subscription API for the client is modified to know why a
subscription was closed.

Full changelog here:
https://github.com/paritytech/jsonrpsee/releases/tag/v0.23.0

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Introduce migration type to remove data associated with a specific
storage of a pallet.

Based on existing `RemovePallet` migration type.

Required for https://github.com/paritytech/polkadot-sdk/pull/3820

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

A few refactorings to block import and block verification that should
not be controversial.

Block verification before block import is stateless by design as
described in https://substrate.stackexchange.com/a/1322/25 and the fact
that it wasn't yet I consider to be a bug. Some code that requires it
had to use `Mutex`, but I do not expect it to have a measurable
performance impact.

Similarly with block import checking whether block preconditions should
not be an exclusive operation, there is nothing fundamentally wrong with
checking a few competing blocks whose parent blocks exist at the same
time (and even import them concurrently later, though IIRC this is not
yet implemented either).

They were originally a part of
https://github.com/paritytech/polkadot-sdk/pull/4842 and upstreaming
will help us to reduce the size of the patch we need to apply on top of
upstream code at Subspace every time we upgrade. There are no new
features introduced here, just refactoring to get rid of unnecessary
requirements.

Fixes https://github.com/paritytech/polkadot-sdk/issues/4723. Also,
closes https://github.com/paritytech/polkadot-sdk/issues/4622

As stated in the linked issue, this PR adds the ability to use a real
rust type for pallet alias in the new `runtime` macro:
```rust
#[runtime::pallet_index(0)]
pub type System = frame_system::Pallet<Runtime>;
```

Please note that the current syntax still continues to be supported.

CC: @shawntabrizi @Kianenigma



---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

Upgrade libp2p to 0.52.4, including a fix: 

* Set Kademlia to server mode
(https://github.com/paritytech/substrate/pull/14703)

### TODO
- [x] Fix 3 zombienet tests failing:
  - [x] `zombienet-substrate-0002-validators-warp-sync`
- [ ]
~`zombienet-polkadot-functional-0005-parachains-disputes-past-session`~
The test is also flaky in other PRs and is not required for CI to
succeed.
  - [x] `zombienet-polkadot-functional-0009-approval-voting-coalescing`
- [x] Uncomment and update to the actual libp2p API tests in
[`substrate/client/network/src/protocol/notifications/handler.rs`](https://github.com/paritytech/polkadot-sdk/blob/7331f179/substrate/client/network/src/protocol/notifications/handler.rs#L1009).
- [x] When upgrading `multihash` crate as part of libp2p upgrade to
version v0.19.1, uncomment the conversion code at
https://github.com/paritytech/polkadot-sdk/blob/7547c494

/substrate/client/network/types/src/multihash.rs#L159
- [x] Perform a burn-in.

---------

Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Bastian Köcher <git@kchr.de>

While working on https://github.com/paritytech/polkadot-sdk/pull/4600 I
found that it would be nice if `chain-spec-builder` supported
`codeSubstitutes`. After this PR is merged you can do:

```
chain-spec-builder add-code-substitute chain_spec.json my_runtime.compact.compressed.wasm 1234
```

In addition, the `chain-spec-builder` was silently removing
`relay_chain` and `para_id` fields when used on parachain chain-specs.
This is now fixed by providing a custom chain-spec extension that has
these fields marked as optional.

Some primitives have impl Hex related traits enabled by `rustc-hex`
feature. People wanna use H256/H160 maybe need these trait impls

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

We should not print a warning if the best block stays the same between
two finality events. There is no requirement that this changes. For sure
this doesn't need to be an error message.

Signed-off-by: dashangcun <jchaodaohang@foxmail.com>
Co-authored-by: dashangcun <jchaodaohang@foxmail.com>

Remove unused config parameters `ApproveOrigin` and `OnSlash` from the
treasury pallet. Add `OnSlash` config parameter to the bounties and tips
pallets.

part of https://github.com/paritytech/polkadot-sdk/issues/3800

Pull the Rust dependencies in advance to see if there is an issue with
timeouts.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Configuration for the maximum member count per rank, with the option for
no limit.

After preparing in https://github.com/paritytech/polkadot-sdk/pull/4633,
we can lift also all internal dependencies up to the workspace.

This does not actually change anything, but uses `workspace = true` for
all dependencies. You can check it with:
```bash
git checkout -q $(git merge-base oty-lift-all-deps origin/master)
cargo tree -e features > master.out

git checkout -q oty-lift-all-deps
cargo tree -e features > new.out
diff master.out new.out
```

It did not yet lift 100% of dependencies, some inside of `target.*` or
some that had conflicting aliases introduced recently. But i will do
these together in a follow-up with CI checks.

Can be reproduced with [zepter](https://github.com/ggwpez/zepter/):
`zepter transpose d lift-to-workspace "regex:.*" --version-resolver
highest --skip-package "polkadot-sdk" --ignore-errors --fix`.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

When `min_enactment_period == 0` and `desired == At(n)` where `n` is
smaller than the current block number, the scheduling would fail. This
happened for example here:
https://collectives.subsquare.io/fellowship/referenda/126

To ensure that this doesn't happen again, ensure that the earliest
allowed block is at minimum the next block.

### ISSUE
Link to the issue:
https://github.com/paritytech/polkadot-sdk/issues/1143

Deliverables
 - Implement trait `ContainsLengthBound` for pallet-membership
 

### Test Outcomes
___
Successful tests by running `cargo test -p pallet-membership --features
runtime-benchmarks`



running 22 tests
test tests::__construct_runtime_integrity_test::runtime_integrity_tests
... ok
test benchmark::bench_clear_prime ... ok
test tests::add_member_works ... ok
test tests::change_key_with_same_caller_as_argument_changes_nothing ...
ok
test tests::change_key_works ... ok
test benchmark::bench_set_prime ... ok
test benchmark::bench_remove_member ... ok
test benchmark::bench_change_key ... ok
test tests::change_key_works_that_does_not_change_order ... ok
test benchmark::bench_reset_members ... ok
test benchmark::bench_add_member ... ok
test tests::genesis_build_panics_with_duplicate_members - should panic
... ok
test benchmark::bench_swap_member ... ok
test tests::query_membership_works ... ok
test tests::prime_member_works ... ok
test tests::test_genesis_config_builds ... ok
test tests::remove_member_works ... ok
test tests::migration_v4 ... ok
test tests::swap_member_with_identical_arguments_changes_nothing ... ok
test tests::swap_member_works_that_does_not_change_order ... ok
test tests::swap_member_works ... ok
test tests::reset_members_works ... ok

test result: ok. 22 passed; 0 failed; 0 ignored; 0 measured; 0 filtered
out; finished in 0.01s

   Doc-tests pallet_membership

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered
out; finished in 0.00s

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

I carried these things in a fork for a long time, I think wouldn't hurt
to have it upstream.

Originally submitted as part of
https://github.com/paritytech/polkadot-sdk/pull/1598 that went nowhere.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Currently the `Initialize` and `Reinitialize` messages in the
collationGeneration subsystem fail if:
-  `Initialize` if there exists already another configuration and
- `Reinitialize` if another configuration does not exist

I propose to instead change the behaviour of `Reinitialize` to always
set the config regardless of whether one exists or not.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

As per #3326, removes pallet::getter macro usage from the
pallet-elections-phragmen. The syntax `StorageItem::<T, I>::get()`
should be used instead.

cc @muraca

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

`Consideration` trait generic over `Footprint` and indicates zero cost
for a give footprint.

`Consideration` trait is generic over `Footprint` (currently defined
over the type with the same name). This makes it possible to setup a
custom footprint (e.g. current number of proposals in the storage).

`Consideration::new` and `Consideration::update` return an
`Option<Self>` instead `Self`, this make it possible to indicate a no
cost for a specific footprint (e.g. if current number of proposals in
the storage < max_proposal_count / 2 then no cost).

These cases need to be handled for
https://github.com/paritytech/polkadot-sdk/pull/3151

Closes: https://github.com/paritytech/polkadot-sdk/issues/4748

Add `From` implementation used by downstream project.

Ref.
https://github.com/paritytech/polkadot-sdk/pull/4198#discussion_r1648676102

CC @nazar-pc

Closes #3342

cc/ @liamaharon



TODO:

- [x] Improve docs.
- [x] Define public interface (See #3342).
  In case we define public calls to the pallet implementation:
  - Implement public calls.
  - Benchmarks.
  
polkadot address: 12gMhxHw8QjEwLQvnqsmMVY1z5gFa54vND74aMUbhhwN6mJR

---------

Co-authored-by: command-bot <>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

Changes:
- The XCMP queue does not partially modify pages anymore by using
`try_mutate` instead of `mutate`.
- The XCMP queue max page size is now the min between the value that the
relay reports and the local limit.

Thanks to whom pointed this out to me via DM.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

This PR:

- Adds extra functionality to pallet-nfts's `BenchmarkHelper` to provide
signers and sign message.
- Abstracts away the explicit link with Sr25519 schema in the
benchmarks, allowing parachains with a different one to be able to run
them and calculate the weights.
- Adds a default implementation for the empty tuple that leaves the code
equivalent.

PR changes forklift authentication to gcs

cc https://github.com/paritytech/ci_cd/issues/987

Bumps [ws](https://github.com/websockets/ws) from 8.16.0 to 8.17.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/websockets/ws/releases">ws's
releases</a>.</em></p>
<blockquote>
<h2>8.17.1</h2>
<h1>Bug fixes</h1>
<ul>
<li>Fixed a DoS vulnerability (<a
href="https://redirect.github.com/websockets/ws/issues/2231">#2231</a>).</li>
</ul>
<p>A request with a number of headers exceeding
the[<code>server.maxHeadersCount</code>][]
threshold could be used to crash a ws server.</p>
<pre lang="js"><code>const http = require('http');
const WebSocket = require('ws');
<p>const wss = new WebSocket.Server({ port: 0 }, function () {
const chars =
&quot;!#$%&amp;'*+-.0123456789abcdefghijklmnopqrstuvwxyz^_`|~&quot;.split('');
const headers = {};
let count = 0;</p>
<p>for (let i = 0; i &lt; chars.length; i++) {
if (count === 2000) break;</p>
<pre><code>for (let j = 0; j &amp;lt; ch...

Bumps
[curve25519-dalek](https://github.com/dalek-cryptography/curve25519-dalek)
from 4.1.2 to 4.1.3.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/5312a0311ec40df95be953eacfa8a11b9a34bc54"><code>5312a03</code></a>
curve: Bump version to 4.1.3 (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/660">#660</a>)</li>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/b4f9e4df92a4689fb59e312a21f940ba06ba7013"><code>b4f9e4d</code></a>
SECURITY: fix timing variability in backend/serial/u32/scalar.rs (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/661">#661</a>)</li>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/415892acf1cdf9161bd6a4c99bc2f4cb8fae5e6a"><code>415892a</code></a>
SECURITY: fix timing variability in backend/serial/u64/scalar.rs (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/659">#659</a>)</li>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/56bf398d0caed63ef1d1edfbd35eb5335132aba2"><code>56bf398</code></a>
Updates license field to valid SPDX format (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/647">#647</a>)</li>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/9252fa5c0d09054fed4ac4d649e63c40fad7abaf"><code>9252fa5</code></a>
Mitigate check-cfg until MSRV 1.77 (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/652">#652</a>)</li>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/1efe6a93b176c4389b78e81e52b2cf85d728aac6"><code>1efe6a9</code></a>
Fix a minor typo in signing.rs (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/649">#649</a>)</li>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/cc3421a22fa7ee1f557cbe9243b450da53bbe962"><code>cc3421a</code></a>
Indicate that the rand_core feature is required (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/641">#641</a>)</li>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/858c4ca8ae03d33fe8b71b4504c4d3f5ff5b45c0"><code>858c4ca</code></a>
Address new nightly clippy unnecessary qualifications (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/639">#639</a>)</li>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/31ccb6705067d68782cb135e23c79b640a6a06ee"><code>31ccb67</code></a>
Remove platforms in favor using CARGO_CFG_TARGET_POINTER_WIDTH (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/636">#636</a>)</li>
<li><a
href="https://github.com/dalek-cryptography/curve25519-dalek/commit/19c7f4a5d5e577adc9cc65a837abef9ed7ebf0a4"><code>19c7f4a</code></a>
Fix new nightly redundant import lint warns (<a
href="https://redirect.github.com/dalek-cryptography/curve25519-dalek/issues/638">#638</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/dalek-cryptography/curve25519-dalek/compare/curve25519-4.1.2...curve25519-4.1.3">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=curve25519-dalek&package-manager=cargo&previous-version=4.1.2&new-version=4.1.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/paritytech/polkadot-sdk/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Previous attempt https://github.com/paritytech/substrate/pull/14314

Close #4443 

Ideally, we should move /health and /health/readiness to the prometheus
server but because it's was quite easy to implement on the RPC server
and that RPC server already exposes /health.

Manual tests on a polkadot node syncing:

```bash
➜ polkadot-sdk (na-fix-4443) ✗ curl -v localhost:9944/health
* Host localhost:9944 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:9944...
* connect to ::1 port 9944 from ::1 port 55024 failed: Connection refused
*   Trying 127.0.0.1:9944...
* Connected to localhost (127.0.0.1) port 9944
> GET /health HTTP/1.1
> Host: localhost:9944
> User-Agent: curl/8.5.0
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: application/json; charset=utf-8
< content-length: 53
< date: Fri, 14 Jun 2024 16:12:23 GMT
<
* Connection #0 to host localhost left intact
{"peers":0,"isSyncing":false,"shouldHavePeers":false}%
➜ polkadot-sdk (na-fix-4443) ✗ curl -v localhost:9944/health/readiness
* Host localhost:9944 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:9944...
* connect to ::1 port 9944 from ::1 port 54328 failed: Connection refused
*   Trying 127.0.0.1:9944...
* Connected to localhost (127.0.0.1) port 9944
> GET /health/readiness HTTP/1.1
> Host: localhost:9944
> User-Agent: curl/8.5.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< content-type: application/json; charset=utf-8
< content-length: 0
< date: Fri, 14 Jun 2024 16:12:36 GMT
<
* Connection #0 to host localhost left intact
```

//cc @BulatSaif

 you may be interested in this..

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

`ValueEnum` is apparently not using the `from_str`...

Closes: https://github.com/paritytech/polkadot-sdk/issues/4828

Implements most of
https://github.com/paritytech/polkadot-sdk/issues/1797

Core sharing (two parachains or more marachains scheduled on the same
core with the same `PartsOf57600` value) was not working correctly. The
expected behaviour is to have Backed and Included event in each block
for the paras sharing the core and the paras should take turns. E.g. for
two cores we expect: Backed(a); Included(a)+Backed(b);
Included(b)+Backed(a); etc. Instead of this each block contains just one
event and there are a lot of gaps (blocks w/o events) during the
session.

Core sharing should also work when collators are building collations
ahead of time

TODOs:

- [x] Add a zombienet test verifying that the behaviour mentioned above
works.
- [x] prdoc

---------

Co-authored-by: alindima <alin@parity.io>

Migrated commands individually to work as GitHub actions with a
[`workflow_dispatch`](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_dispatch)
event.

This will not disable the command-bot yet, but it's the first step
before disabling it.

### Commands migrated
- [x] bench-all
- [x] bench-overhead
- [x] bench
- [x] fmt
- [x] update-ui

Also created an action that will inform users about the new
documentation when they comment `bot`.

### Created documentation 
Created a detailed documentation on how to use this action. Found the
documentation
[here](https://github.com/paritytech/polkadot-sdk/blob/bullrich/cmd-action/.github/commands-readme.md).

---------

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Co-authored-by: Przemek Rzad <przemek@parity.io>

remove redundant words in comments.

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>