Writing down the processes to do our releases.

Status: please review & approve so we can go ahead.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Jegor Sidorenko <[email protected]>

Related https://github.com/paritytech/polkadot-sdk/issues/3032

---

Using https://github.com/liamaharon/cargo-workspace-version-tools/ 

`cargo run -- sync --path ../polkadot-sdk`

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

Fix the Pools `v7` migration.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>

Bumps the known_good_semver group with 1 update:
[serde_yaml](https://github.com/dtolnay/serde-yaml).

Updates `serde_yaml` from 0.9.27 to 0.9.30
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/serde-yaml/releases">serde_yaml's
releases</a>.</em></p>
<blockquote>
<h2>0.9.30</h2>
<ul>
<li>Update proc-macro2 to fix caching issue when using a rustc-wrapper
such as sccache</li>
</ul>
<h2>0.9.29</h2>
<ul>
<li>Turn on <code>deny(unsafe_op_in_unsafe_fn)</code> lint</li>
</ul>
<h2>0.9.28</h2>
<ul>
<li>Update <code>unsafe-libyaml</code> dependency to pull in unaligned
write fix</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/09ee25156f608f95150b27edd120bd5471db3c64"><code>09ee251</code></a>
Release 0.9.30</li>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/caea939ece85ab54ac41de0672d683905f1e406a"><code>caea939</code></a>
Pull in proc-macro2 sccache fix</li>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/d255918c141fd72d01f169bb5aa0152234981699"><code>d255918</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/serde-yaml/issues/403">#403</a>
from dtolnay/optionifletelse</li>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/8cfeedd766f83dab24d03b9fb11b72886a247425"><code>8cfeedd</code></a>
Remove option_if_let_else clippy suppression</li>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/b957d2b15d7f3d96279997800fa0610b41b8fe00"><code>b957d2b</code></a>
Release 0.9.29</li>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/007fc2d5c1987847a0f1ac95885c56f8e6e5f808"><code>007fc2d</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/serde-yaml/issues/401">#401</a>
from dtolnay/unsafeop</li>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/5bac2475b0017d6a635d641df17165d71b951f0e"><code>5bac247</code></a>
Fill in unsafe blocks inside unsafe functions</li>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/0f6dba18ab8db598c4963e9242afd490ee0202f0"><code>0f6dba1</code></a>
Turn on deny(unsafe_op_in_unsafe_fn)</li>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/1b6e44837f0db0a1c15537311fed3579a7e8c6d2"><code>1b6e448</code></a>
Release 0.9.28</li>
<li><a
href="https://github.com/dtolnay/serde-yaml/commit/ec1a3145d7b6b809f7b3aa2d9dcd7db30f0588d4"><code>ec1a314</code></a>
Force unsafe-libyaml version that contains unaligned write fix</li>
<li>Additional commits viewable in <a
href="https://github.com/dtolnay/serde-yaml/compare/0.9.27...0.9.30">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=serde_yaml&package-manager=cargo&previous-version=0.9.27&new-version=0.9.30)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <[email protected]>

closes https://github.com/paritytech/parity-bridges-common/issues/2796

This partially reverts the #2439 - there are some changes (unrelated to
CI) that we still want to keep. The reason of that removal is that with
async backing enabled for Rococo AH (and for other chains in the near
future), we see a lot of issues there (because we run `14` nodes +
additional standalone process within a same container and it causes a
lot of timeouts). There's no way known to me to fix it right now, so
we're removing those tests hopefully temporarily to keep CI green

🤦🏼



---------

Signed-off-by: Andrei Sandu <[email protected]>

also remove some dead code and deduplicate some error handling

the new release brings performance improvements and support for
systematic chunk recovery, needed in:
https://github.com/paritytech/polkadot-sdk/pull/1644

This PR introduces a new `NonFungibleAdapter`.

It will be useful for enabling cross-chain Coretime region transfers, as
the existing `NonFungiblesAdapter` is unsuitable for this purpose. This
is due to the fact that there is only one class of items within the
`pallet-broker`, i.e., the Coretime regions.

---------

Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: Dónal Murray <[email protected]>

This PR backports the changes from
https://github.com/paritytech/json-rpc-interface-spec/pull/107.

The `transaction` class becomes `transactionWatch`, and the other
functionality remains the same.

// cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>

Add subsystem benchmarks for `availability-distribution` and `biftield-distribution` (availability write) (#2970)

Introduce a new test objective : `DataAvailabilityWrite`.

The new benchmark measures the network and cpu usage of
`availability-distribution`, `biftield-distribution` and
`availability-store` subsystems from the perspective of a validator node
during the process when candidates are made available.

Additionally I refactored the networking emulation to support bandwidth
acounting and limits of incoming and outgoing requests.

Screenshot of succesful run


<img width="1293" alt="Screenshot 2024-01-17 at 19 17 44"
src="https://github.com/paritytech/polkadot-sdk/assets/54316454/fde11280-e25b-4dc3-9dc9-d4b9752f9b7a">

---------

Signed-off-by: Andrei Sandu <[email protected]>

We don't need dependency on `json_patch` there is implementation of
needed function in `sc_chain_spec` crate.
This PR makes this function publicly available and removes the dep.

We would like to add our RadiumBlock bootnode to collectives-westend
parachain

Fixes: https://github.com/paritytech/polkadot-sdk/issues/2138.

Especially on restart AuthorithyDiscovery cache is not populated so we
create an invalid topology and messages won't be routed correctly for
the entire session. This PR proposes to try to fix this by updating the
topology as soon as we now the Authority/PeerId mapping, that should
impact the situation dramatically.


[This issue was hit
yesterday](https://grafana.teleport.parity.io/goto/o9q2625Sg?orgId=1),
on Westend and resulted in stalling the finality.


# TODO

- [x] Unit tests
- [x] Test impact on versi

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>

Bumps [arbitrary](https://github.com/rust-fuzz/arbitrary) from 1.3.0 to
1.3.2.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-fuzz/arbitrary/blob/main/CHANGELOG.md">arbitrary's
changelog</a>.</em></p>
<blockquote>
<h2>1.3.2</h2>
<p>Released 2023-10-30.</p>
<h3>Added</h3>
<ul>
<li>Added <code>Arbitrary</code> implementations for
<code>Arc&lt;[T]&gt;</code> and
<code>Rc&lt;[T]&gt;</code>. <a
href="https://redirect.github.com/rust-fuzz/arbitrary/pull/160">#160</a></li>
</ul>
<hr />
<h2>1.3.1</h2>
<p>Released 2023-10-11.</p>
<h3>Fixed</h3>
<ul>
<li>Fixed an issue with generating collections of collections in
<code>arbitrary_take_rest</code> where
<code>&lt;Vec&lt;Vec&lt;u8&gt;&gt;&gt;::arbitrary_take_rest</code> would
never
generate <code>vec![vec![]]</code> for example. See
<a
href="https://redirect.github.com/rust-fuzz/arbitrary/pull/159">#159</a>
for details.</li>
</ul>
<hr />
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/66e75c5bf57275d400d3ebc746e0cee4f6ff9596"><code>66e75c5</code></a>
Bump to version 1.3.1</li>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/04054dfa1a0f07b233db0581c2d61615df737ade"><code>04054df</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-fuzz/arbitrary/issues/160">#160</a>
from kpreid/arcslice</li>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/ef5dff63e4f3079acc6455445f0a8080d4857813"><code>ef5dff6</code></a>
Implement <code>Arbitrary</code> for <code>Arc\&lt;[A]&gt;</code> and
<code>Rc\&lt;[A]&gt;</code>.</li>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/b3e8342ea8dc8437aff3d3a1f5b95b7c02bf57f5"><code>b3e8342</code></a>
Bump to version 1.3.1</li>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/c1fa740bb777940bda77a4154d035805b4ecce5b"><code>c1fa740</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-fuzz/arbitrary/issues/159">#159</a>
from fitzgen/arbitrary-take-rest-and-collections-of-c...</li>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/f19fd7a512fe953e902954d01fe046475d8f01a7"><code>f19fd7a</code></a>
Add clippy allow for existing code running afoul of new clippy lint</li>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/27560f182b5f0feb8dbd70791cbadd6fbd622411"><code>27560f1</code></a>
Fix <code>Unstructured::arbitrary_take_rest_iter</code> for collections
of collections</li>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/80d6bfe5e8c864a05ed8c1f0a107bca632ea8c61"><code>80d6bfe</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-fuzz/arbitrary/issues/157">#157</a>
from jyn514/ip-addr</li>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/7d3364edb6a39554c4b97f0d0548289f001121fe"><code>7d3364e</code></a>
impl Arbitrary for IpAddr</li>
<li><a
href="https://github.com/rust-fuzz/arbitrary/commit/0bdbec8a9fdf19a18e6cb8ffe4022b9a6a588cf2"><code>0bdbec8</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-fuzz/arbitrary/issues/151">#151</a>
from Ekleog-NEAR/patch-2</li>
<li>Additional commits viewable in <a
href="https://github.com/rust-fuzz/arbitrary/compare/v1.3.0...v1.3.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=arbitrary&package-manager=cargo&previous-version=1.3.0&new-version=1.3.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [docify](https://github.com/sam0x17/docify) from 0.2.6 to 0.2.7.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/sam0x17/docify/releases">docify's
releases</a>.</em></p>
<blockquote>
<h2>v0.2.7</h2>
<p>updates toml to 0.8</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/sam0x17/docify/commit/a6bb26159613db316c14392215f08479d60093e1"><code>a6bb261</code></a>
bump to v0.2.7</li>
<li><a
href="https://github.com/sam0x17/docify/commit/22b6e0cbedb47cf86dd8e9b394557508a6af20d0"><code>22b6e0c</code></a>
Merge pull request <a
href="https://redirect.github.com/sam0x17/docify/issues/24">#24</a> from
kayabaNerve/main</li>
<li><a
href="https://github.com/sam0x17/docify/commit/19d3cd625d3d4f699d1aeeb6d08a8408b495724b"><code>19d3cd6</code></a>
toml 0.8</li>
<li>See full diff in <a
href="https://github.com/sam0x17/docify/compare/v0.2.6...v0.2.7">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=docify&package-manager=cargo&previous-version=0.2.6&new-version=0.2.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

https://github.com/paritytech/polkadot-sdk/pull/3042/files#r1465115145

---------

Co-authored-by: command-bot <>

Moved from: https://github.com/paritytech/polkadot/pull/6951

closes https://github.com/paritytech/polkadot-sdk/issues/490

- [x] update cumulus

--- 
This PR introduces transactional processing of certain xcm instructions.
For the list of instructions checkout
https://github.com/paritytech/polkadot-sdk/issues/490. The transactional
processing is implemented as an xcm-executor config item. The two
implementations in this PR are `FrameTransactionalProcessor` and `()`.
The `()` implementation does no transactional processing. Each
implementation of the `ProcessTransaction` trait has an
`IS_TRANSACTIONAL` const that tells the XCVM if transactional processing
is actually implemented. If Transactional processing is implemented,
changes to touched registers should also be rolled back to prevent
inconsistencies.


Note for reviewers:
Check out the following safety assumption:
https://github.com/paritytech/polkadot-sdk/pull/1222/files#diff-4effad7d8c1c9de19fd27e18661cbf2128c8718f3b2420a27d2f816e0749ea53R30

---------

Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: command-bot <>

This PR avoids automatically pulling rococo/westend runtime constants
into the runtime by default. Usually, we have testnet runtimes dedicated
to rococo or westend, and therefore, we don't need both dependencies.
Additionally, it prevents pulling rococo/westend-related items into the
`polkadot-fellows` repo, as seen in the Cargo.lock
[here](https://github.com/polkadot-fellows/runtimes/blob/main/Cargo.lock#L14137-L14151)
and
[here](https://github.com/polkadot-fellows/runtimes/blob/main/Cargo.lock#L9756-L9770).

---------

Co-authored-by: Adrian Catangiu <[email protected]>

## Summary

This PR consolidates `pallet-state-trie-migration` as a part of
https://github.com/paritytech/polkadot-sdk/issues/226 /
https://github.com/paritytech/polkadot-sdk/issues/171:

`pallet-state-trie-migration`:
- [x] replace `Currency` with `fungible` traits
- [x] run benchmarks
- [x] refactor to `DefaultConfig`

`pallet_nicks`:
- [x]  remove

others:
- [x] remove `as Fn*` or `asFun*` stuff based on discussion
[here](https://github.com/paritytech/polkadot-sdk/issues/226#issuecomment-1822861445)

---------

Co-authored-by: Richard Melkonian <[email protected]>
Co-authored-by: command-bot <>

As activation can fail, we ensure that we don't miss deactivation of
leaves.

This PR implements phase 1 of:
https://github.com/paritytech/polkadot-sdk/pull/2372#discussion_r1461873933

NOTE: This means that all the current parachains can upgrade their code
to the maximum size for free.

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Radha <[email protected]>

There were several improvements and PRs that didn't apply to all
runtimes, so this PR attempts to align those small differences. In
addition, the PR eliminates unused dependencies across multiple modules.

Relates to PR for `polkadot-fellows`:
https://github.com/polkadot-fellows/runtimes/pull/154

The
[archive](https://github.com/paritytech/json-rpc-interface-spec/blob/main/src/api/archive.md)
API is unstable and subject to change.

This PR enables the `archive` class of the RPC-V2 spec to substrate
based chains.

The `archive` API is enabled for archive nodes: 
- the state of the blocks is in archive mode
- the block's bodies are in archive mode

While at it, this PR extends the `BlocksPrunning` enum with an
`is_archive` helper to check if the pruning mode keeps the block's
bodies for long enough.

Defaults used for the `archive` API:
- a maximum of 5 responses are provided for descendants queries (this is
similar to chainHead)
- a maximum of 8 item queries are accepted at a time

Before stabilizing the API we should look into these defaults and adjust
after collecting some data.

---------

Signed-off-by: Alexandru Vasile <[email protected]>

Found the issue while investigating the recent finality stall on Westend
after upgrading to 1.6.0. Approval distribution aggression is supposed
to trade off bandwidth and re-send assignemnts/approvals until enough
approvals are be received by at least 2/3 validators. This is supposed
to be a catch all mechanism when network connectivity goes south or many
validators reboot at the same time.

This fix ensures that we always resend approvals starting with the first
unfinalized block even in the case when it appears approved from the
node's perspective.

TODO:
- [x] Versi test

---------

Signed-off-by: Andrei Sandu <[email protected]>

This is a rather big change in jsonrpsee, the major things in this bump
are:
- Server backpressure (the subscription impls are modified to deal with
that)
- Allow custom error types / return types (remove jsonrpsee::core::Error
and jsonrpee::core::CallError)
- Bug fixes (graceful shutdown in particular not used by substrate
anyway)
   - Less dependencies for the clients in particular
   - Return type requires Clone in method call responses
   - Moved to tokio channels
   - Async subscription API (not used in this PR)

Major changes in this PR:
- The subscriptions are now bounded and if subscription can't keep up
with the server it is dropped
- CLI: add parameter to configure the jsonrpc server bounded message
buffer (default is 64)
- Add our own subscription helper to deal with the unbounded streams in
substrate

The most important things in this PR to review is the added helpers
functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
much chore.

Regarding the "bounded buffer limit" it may cause the server to handle
the JSON-RPC calls
slower than before.

The message size limit is bounded by "--rpc-response-size" thus "by
default 10MB * 64 = 640MB"
but the subscription message size is not covered by this limit and could
be capped as well.

Hopefully the last release prior to 1.0, sorry in advance for a big PR

Previous attempt: https://github.com/paritytech/substrate/pull/13992

Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
https://github.com/paritytech/polkadot-sdk/issues/627

Bumps [substrate-bip39](https://github.com/paritytech/substrate-bip39)
from 0.4.4 to 0.4.5.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/paritytech/substrate-bip39/commits/v0.4.5">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=substrate-bip39&package-manager=cargo&previous-version=0.4.4&new-version=0.4.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [shlex](https://github.com/comex/rust-shlex) from 1.1.0 to 1.3.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/comex/rust-shlex/blob/master/CHANGELOG.md">shlex's
changelog</a>.</em></p>
<blockquote>
<h1>1.3.0</h1>
<ul>
<li>Full fix for the high-severity security vulnerability <a
href="https://rustsec.org/advisories/RUSTSEC-2024-0006.html">RUSTSEC-2024-0006</a>
a.k.a. <a
href="https://github.com/comex/rust-shlex/security/advisories/GHSA-r7qv-8r2h-pg27">GHSA-r7qv-8r2h-pg27</a>:
<ul>
<li>Deprecates quote APIs in favor of <code>try_</code> equivalents that
complain about nul bytes.</li>
<li>Also adds a builder API, which allows re-enabling nul bytes without
using the deprecated interface, and in the future can allow other things
(as discussed in quoting_warning).</li>
<li>Adds documentation about various security risks that remain,
particularly with interactive shells.</li>
</ul>
</li>
<li>Adds explicit MSRV of 1.46.0.</li>
</ul>
<h1>1.2.1</h1>
<ul>
<li>Partial fix for the high-severity security vulnerability <a
href="https://rustsec.org/advisories/RUSTSEC-2024-0006.html">RUSTSEC-2024-0006</a>
a.k.a. <a
href="https://github.com/comex/rust-shlex/security/advisories/GHSA-r7qv-8r2h-pg27">GHSA-r7qv-8r2h-pg27</a>
without bumping MSRV:
<ul>
<li>The bytes <code>{</code> and <code>\xa0</code> are now escaped by
quoting functions.</li>
</ul>
</li>
</ul>
<h1>1.2.0</h1>
<ul>
<li>Adds <code>bytes</code> module to support operating directly on byte
strings.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/comex/rust-shlex/commits">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=shlex&package-manager=cargo&previous-version=1.1.0&new-version=1.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/paritytech/polkadot-sdk/network/alerts).

</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

If an Executor is required in some pallet's config, a mock might be
provided where `Executor = ()`. `()` already implements `ExecuteXcm`,
but not `XcmAssetTransfers`, which is also related. This PR just fixes
that so you can skip having to create a whole xcm configuration, which
is non-trivial right now.

Step towards https://github.com/paritytech/polkadot-sdk/issues/1975

As reported
https://github.com/paritytech/polkadot-sdk/issues/1975#issuecomment-1774534225
I'd like to encapsulate crypto related stuff in a dedicated folder.

Currently all cryptographic primitive wrappers are all sparsed in
`substrate/core` which contains "misc core" stuff.

To simplify the process, as the first step with this PR I propose to
move the cryptographic hashing there.

The `substrate/crypto` folder was already created to contains `ec-utils`
crate.

Notes:
- rename `sp-core-hashing` to `sp-crypto-hashing`
- rename `sp-core-hashing-proc-macro` to `sp-crypto-hashing-proc-macro`
- As the crates name is changed I took the freedom to restart fresh from
version 0.1.0 for both crates

---------

Co-authored-by: Robert Hambrock <[email protected]>

... it was like that before it was accidentally changed in
https://github.com/paritytech/polkadot-sdk/commit/a84dd0db



Signed-off-by: Alexandru Gheorghe <[email protected]>

Bumps [wasm-instrument](https://github.com/paritytech/wasm-instrument)
from 0.3.0 to 0.4.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/paritytech/wasm-instrument/blob/master/CHANGELOG.md">wasm-instrument's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<p>All notable changes to this project will be documented in this
file.</p>
<p>The format is based on <a
href="https://keepachangelog.com/en/1.0.0/">Keep a Changelog</a>,
and this project adheres to <a
href="https://semver.org/spec/v2.0.0.html">Semantic Versioning</a>.</p>
<p>The semantic versioning guarantees cover the interface to the
substrate runtime which
includes this pallet as a dependency. This module will also add storage
migrations whenever
changes require it. Stability with regard to offchain tooling is
explicitly excluded from
this guarantee: For example adding a new field to an in-storage data
structure will require
changes to frontends to properly display it. However, those changes will
still be regarded
as a minor version bump.</p>
<p>The interface provided to smart contracts will adhere to semver with
one exception: Even
major version bumps will be backwards compatible with regard to already
deployed contracts.
In other words: Upgrading this pallet will not break pre-existing
contracts.</p>
<h2>[Unreleased]</h2>
<h3>New</h3>
<ul>
<li>Add new gas metering method: mutable global + local gas function
<a
href="https://redirect.github.com/paritytech/wasm-instrument/pull/34">#34</a></li>
<li>Account for locals initialization costs
<a
href="https://redirect.github.com/paritytech/wasm-instrument/pull/38">#38</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/paritytech/wasm-instrument/commits/v0.4.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=wasm-instrument&package-manager=cargo&previous-version=0.3.0&new-version=0.4.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

> **Note**
> Automatic rebases have been disabled on this pull request as it has
been open for over 30 days.

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

This PR wraps the `Snapshot`, `SnapshotMetadata` and `DesiredTargets`
storage items in the [EPM
pallet](https://paritytech.github.io/substrate/master/pallet_election_provider_multi_phase/index.html)
in order to keep them in sync throughout the election lifetime and in
order to be killed together.

Prior to this PR, these storage items were mutated in different places
in the pallet;

In addition 2 helper `fns` are introduced for chekcing if all the
wrapped storage items exist or not;

Fixes #413 ;

---------

Co-authored-by: Gonçalo Pestana <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.10 to
0.9.3.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md">memmap2's
changelog</a>.</em></p>
<blockquote>
<h2>[0.9.3] - 2023-12-19</h2>
<h3>Fixed</h3>
<ul>
<li>Build on Android.</li>
</ul>
<h2>[0.9.2] - 2023-12-17</h2>
<h3>Fixed</h3>
<ul>
<li>Build on FreeBSD.</li>
</ul>
<h2>[0.9.1] - 2023-12-16</h2>
<h3>Changed</h3>
<ul>
<li>Added <code>MmapOptions::huge</code> method to support mapping
hugetlb. Linux only.
<a href="https://github.com/ollie-etl"><code>@​ollie-etl</code></a>
<a
href="https://github.com/oliverbunting"><code>@​oliverbunting</code></a></li>
</ul>
<h2>[0.9.0] - 2023-10-03</h2>
<h3>Changed</h3>
<ul>
<li>The <code>Advice</code> struct was split into two enums:
<code>Advice</code> and <code>UncheckedAdvice</code>.<!-- raw HTML
omitted -->
<code>Advice</code> can be passed to safe <code>advise</code> and
<code>advise_range</code> methods.
And <code>UncheckedAdvice</code> can be passed to unsafe
<code>unchecked_advise</code>
and <code>unchecked_advise_range</code> methods.<!-- raw HTML omitted
-->
<a
href="https://github.com/adamreichold"><code>@​adamreichold</code></a></li>
</ul>
<h2>[0.8.0] - 2023-09-25</h2>
<h3>Changed</h3>
<ul>
<li>The <code>Advice</code> type is a struct and not an enum now.
<a
href="https://github.com/adamreichold"><code>@​adamreichold</code></a></li>
</ul>
<h3>Fixed</h3>
<ul>
<li>Some of the <code>Advise</code> variants were unsound and now
require <code>unsafe</code> to be constructed.
<a
href="https://github.com/adamreichold"><code>@​adamreichold</code></a></li>
</ul>
<h2>[0.7.1] - 2023-06-24</h2>
<h3>Fixed</h3>
<ul>
<li>Mapping beyond 4GB offset on 32 bit glibc. Linux-only.
<a href="https://github.com/lvella"><code>@​lvella</code></a></li>
</ul>
<h2>[0.7.0] - 2023-06-08</h2>
<h3>Added</h3>
<ul>
<li><code>Mmap::remap</code>, <code>MmapMut::remap</code> and
<code>MmapRaw::remap</code>. Linux-only.
<a
href="https://github.com/Phantomical"><code>@​Phantomical</code></a></li>
<li><code>Advice::PopulateRead</code> and
<code>Advice::PopulateWrite</code>. Linux-only.
<a
href="https://github.com/Jesse-Bakker"><code>@​Jesse-Bakker</code></a></li>
</ul>
<h3>Changed</h3>
<ul>
<li>libc crate &gt;= 0.2.143 is required now.</li>
</ul>
<h2>[0.6.2] - 2023-05-24</h2>
<h3>Fixed</h3>
<ul>
<li>Alignment for empty files on Windows.
<a href="https://github.com/timvisee"><code>@​timvisee</code></a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/5a49e8ab0bda97770a6ed972633dfb862e17cd38"><code>5a49e8a</code></a>
Version bump.</li>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/1079b61f400542a5a4b946d9f57761eb2ae674a4"><code>1079b61</code></a>
Fix build on Android.</li>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/c82f49b2969807e3ae9f2e835917ddfd2bd1d25d"><code>c82f49b</code></a>
Version bump.</li>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/faafbfd031e283ab6ef637e8c9d6a5d85e84602b"><code>faafbfd</code></a>
Fix formatting.</li>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/428bb9bf061af89c33d6188dfa4abff2ffcd7e31"><code>428bb9b</code></a>
Fix tests.</li>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/55109c68837e80f3c00f4a56a75a6abd49ead7c7"><code>55109c6</code></a>
Disable huge tables support for freebsd, since it doesn't support
them.</li>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/c173463cc3bfae8b5e6f19ae909af89dd11afe39"><code>c173463</code></a>
Version bump.</li>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/e5faf1339d21ecbe4f62a803050aaae415c96a8d"><code>e5faf13</code></a>
Fix madvise tests to not assume 4k pages.</li>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/3c71286cebd43b49e87449490608024e6b05596b"><code>3c71286</code></a>
Add huge pages support.</li>
<li><a
href="https://github.com/RazrFalcon/memmap2-rs/commit/f16835d01ad1d36198718e23273558f1947fc138"><code>f16835d</code></a>
Version bump.</li>
<li>Additional commits viewable in <a
href="https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.10...v0.9.3">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=memmap2&package-manager=cargo&previous-version=0.5.10&new-version=0.9.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Changes `sc-informant` to respect the `--disable-log-color` CLI flag.

Closes: https://github.com/paritytech/polkadot-sdk/issues/2795

---------

Co-authored-by: Michal Kucharczyk <[email protected]>

The goal is to move all system chains on Rococo (+ other testnets?) to
use async backing.

Starting with `asset-hub-rococo` to get feedback, before I do the rest.

## Related

Example: https://github.com/paritytech/polkadot-sdk/pull/1619/
Guide:
https://github.com/w3f/polkadot-wiki/blob/master/docs/maintain/maintain-guides-async-backing.md

---------

Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: Dónal Murray <[email protected]>

Srlabs review is most of the times just blocking the merge, especially
when the changes do not require any audit. This pr removes the
requirement of srlabs to approve these prs.

Clean up all the old syntax.

---------

Co-authored-by: command-bot <>
Co-authored-by: gupnik <[email protected]>
Co-authored-by: Nikhil Gupta <[email protected]>
Co-authored-by: Maksym H <[email protected]>

This PR aims to channel the backpressure of the PVF host's preparation
and execution queues to the candidate validation subsystem consumers.

Related: #708