Submit the outstanding PRs from the old repos(these were already
reviewed and approved before the repo rorg, but not yet submitted):
Main PR: https://github.com/paritytech/substrate/pull/14014
Companion PRs: https://github.com/paritytech/polkadot/pull/7134,
https://github.com/paritytech/cumulus/pull/2489



The changes in the PR:
1. ChainSync currently calls into the block request handler directly.
Instead, move the block request handler behind a trait. This allows new
protocols to be plugged into ChainSync.
2. BuildNetworkParams is changed so that custom relay protocol
implementations can be (optionally) passed in during network creation
time. If custom protocol is not specified, it defaults to the existing
block handler
3. BlockServer and BlockDownloader traits are introduced for the
protocol implementation. The existing block handler has been changed to
implement these traits
4. Other changes:
[X] Make TxHash serializable. This is needed for exchanging the
serialized hash in the relay protocol messages
[X] Clean up types no longer used(OpaqueBlockRequest,
OpaqueBlockResponse)

---------

Co-authored-by: Dmitry Markin <[email protected]>
Co-authored-by: command-bot <>

In follow-up to https://github.com/paritytech/polkadot-sdk/pull/1518



Adding extra tests for inclusion pruning. Primarily focusing on various
cases surrounding candidates included in different forks (with different
relay parents).

All cases fall into a few buckets based on 3 degrees of freedom - number
of candidates, number of blocks (height), number of forks + extra case
for pruning multiple heights at once.

Added small tweak to the original pruning function to disregard stale
candidate duplicates which should keep the same behaviour.

---------

Co-authored-by: Tsvetomir Dimitrov <[email protected]>

In follow-up to https://github.com/paritytech/polkadot-sdk/pull/1432

Some additional unit tests for the inclusion struct used in the scraper.

Bumps the known_good_semver group with 1 update:
[clap](https://github.com/clap-rs/clap).

<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/releases">clap's
releases</a>.</em></p>
<blockquote>
<h2>v4.4.3</h2>
<h2>[4.4.3] - 2023-09-12</h2>
<h3>Documentation</h3>
<ul>
<li><em>(derive)</em> Clarify use of attributes within the tutorial</li>
<li>Split sections in the builder and derive tutorials into separate
modules</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's
changelog</a>.</em></p>
<blockquote>
<h2>[4.4.3] - 2023-09-12</h2>
<h3>Documentation</h3>
<ul>
<li><em>(derive)</em> Clarify use of attributes within the tutorial</li>
<li>Split sections in the builder and derive tutorials into separate
modules</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/clap-rs/clap/commit/e9668b364c5a0d259780a3e6d8352203d6f329f5"><code>e9668b3</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/bc4986e01e4a6c817e4af8dbd91f96c6aae97107"><code>bc4986e</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/3d53641ca1696fdeafb8e1e87def6b27f1a7d79d"><code>3d53641</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5122">#5122</a>
from epage/docs</li>
<li><a
href="https://github.com/clap-rs/clap/commit/32586c7b6335f437e6f01a089e7664f9c88ddc09"><code>32586c7</code></a>
docs(tutorial): Split into separate modules per section</li>
<li><a
href="https://github.com/clap-rs/clap/commit/5f6d4a3dadd436899923f27ea6ac62b26198ad91"><code>5f6d4a3</code></a>
docs(tutorial): Split out into a module</li>
<li><a
href="https://github.com/clap-rs/clap/commit/20987de0bd2b9b2720771b0f0572745d2ee91433"><code>20987de</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5121">#5121</a>
from epage/docs</li>
<li><a
href="https://github.com/clap-rs/clap/commit/9e7404b5996128efcf4d6098b7d4af4eacb04fca"><code>9e7404b</code></a>
docs(tutorial): Attempt to clarify attributes</li>
<li><a
href="https://github.com/clap-rs/clap/commit/db97a2c5be69fa81af80e9cee5d3943bcfc664c4"><code>db97a2c</code></a>
docs(derive): Clarify value attributes are for PossibleValue</li>
<li><a
href="https://github.com/clap-rs/clap/commit/37ba6075a9e3c968a196e73e78e95a1e7c6e9326"><code>37ba607</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/3234c74a886fdb14abc94b34413d8604d2da0e93"><code>3234c74</code></a>
docs: Update changelog</li>
<li>Additional commits viewable in <a
href="https://github.com/clap-rs/clap/compare/v4.4.2...v4.4.3">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=clap&package-manager=cargo&previous-version=4.4.2&new-version=4.4.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores

)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

This PR addresses multiple issues pending:

* [x] Update orchestra to the recent version and test how the node
performs
* [x] Add some useful metrics for outbound network bridge
* [x] Try to send incoming network requests to all subsystems without
blocking on some particular subsystem in that loop
* [x] Fix all incompatibilities between orchestra and polkadot code
(e.g. malus node)

Closes #583

After the separation of PVF worker binaries, dedicated puppet workers
are not needed for tests anymore. The production workers can be used
instead, avoiding some code duplication and decreasing complexity.

The changes also make it possible to further refactor the code to
isolate workers completely.

Bumps the known_good_semver group with 2 updates: [serde_json](https://github.com/serde-rs/json) and [syn](https://github.com/dtolnay/syn).


Updates `serde_json` from 1.0.105 to 1.0.106
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.105...v1.0.106)

Updates `syn` from 2.0.31 to 2.0.32
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.31...2.0.32

)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: known_good_semver
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: known_good_semver
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove redundant book keeping in dispute coordinator.

* Fix docs.

---------

Co-authored-by: eskimor <[email protected]>

* Removing unnecessary log

* Drop unneeded log

* Handling logs conditionally

* Changing unexpected message to warn

* Back to debug

* fmt

Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.3.4 to 0.4.1.
- [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.3.4...hex-literal-v0.4.1

)

---
updated-dependencies:
- dependency-name: hex-literal
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bastian Köcher <[email protected]>

* polkadot: propagate UnpinHandle to ActiveLeafUpdate

Also extract the leaf creation for tests
into a common function.

* dispute-coordinator: try pinned blocks for slashin

* apparently 1.72 is smarter than 1.70

* address nits

* rename fresh_leaf to new_leaf

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.47 to 1.0.48.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.47...1.0.48

)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Liam Aharon <[email protected]>

* Delete stale adoc files

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Convert adoc to md

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add adoc to gitignore

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Delete more random unmaintained files

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Markdown lint

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add markdown linting

- add linter default rules
- adapt rules to current code
- fix the code for linting to pass
- add CI check

fix #1243

* Fix markdown for Substrate
* Fix tooling install
* Fix workflow
* Add documentation
* Remove trailing spaces
* Update .github/.markdownlint.yaml

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
* Fix mangled markdown/lists
* Fix captalization issues on known words

Bumps the known_good_semver group with 1 update: [syn](https://github.com/dtolnay/syn).

- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.29...2.0.31

)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: known_good_semver
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix wrong ref counting

in case of multiple inserts at same height.

* More warn

---------

Co-authored-by: eskimor <[email protected]>

* Import changes from archieved repo

* Revert erroneous changes

* Fix more tests

* Resolve discussions

* Fix MORE tests

* approval-voting: launch_approval better interface (#1355)

---------

Co-authored-by: Javier Viola <[email protected]>
Co-authored-by: ordian <[email protected]>
Co-authored-by: ordian <[email protected]>

Bumps [bs58](https://github.com/Nullus157/bs58-rs) from 0.4.0 to 0.5.0.
- [Changelog](https://github.com/Nullus157/bs58-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Nullus157/bs58-rs/compare/0.4.0...0.5.0

)

---
updated-dependencies:
- dependency-name: bs58
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* refactor and more debug

Signed-off-by: Andrei Sandu <[email protected]>

* review

Signed-off-by: Andrei Sandu <[email protected]>

* fix

Signed-off-by: Andrei Sandu <[email protected]>

* 🤦



Signed-off-by: Andrei Sandu <[email protected]>

---------

Signed-off-by: Andrei Sandu <[email protected]>

Bumps the known_good_semver group with 1 update: [clap](https://github.com/clap-rs/clap).

- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.4.1...v4.4.2

)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: known_good_semver
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Rename `polkadot-parachain` to `polkadot-parachain-primitives`

While doing this it also fixes some last `rustdoc` issues and fixes
another Cargo warning related to `pallet-paged-list`.

* Fix compilation

* ".git/.scripts/commands/fmt/fmt.sh"

* Fix XCM docs

---------

Co-authored-by: command-bot <>

* Fix polkadot-node-core-pvf-prepare-worker build with jemalloc

The jemalloc feature on polkadot-node-core-pvf-prepare-worker depended
on some feature gated code in polkadot-node-core-pvf-common but there
way no way to enable this feature gate.

This commit adds the feature and makes prepare-worker enable it.

* More jemalloc-allocator fixes

* Fix jemalloc-allocator feature dep

* Run `zepter format features`

---------

Co-authored-by: Marcin S <[email protected]>

* move min backing votes const to runtime

also cache it per-session in the backing subsystem

Signed-off-by: alindima <[email protected]>

* add runtime migration

* introduce api versioning for min_backing votes

also enable it for rococo/versi for testing

* also add min_backing_votes runtime calls to statement-distribution

this dependency has been recently introduced by async backing

* remove explicit version runtime API call

this is not needed, as the RuntimeAPISubsystem already takes care
of versioning and will return NotSupported if the version is not
right.

* address review comments

- parametrise backing votes runtime API with session index
- remove RuntimeInfo usage in backing subsystem, as runtime API
caches the min backing votes by session index anyway.
- move the logic for adjusting the configured needed backing votes with the size of the backing group
to a primitives helper.
- move the legacy min backing votes value to a primitives helper.
- mark JoinMultiple error as fatal, since the Canceled (non-multiple) counterpart is also fatal.
- make backing subsystem handle fatal errors for new leaves update.
- add HostConfiguration consistency check for zeroed backing votes threshold
- add cumulus accompanying change

* fix cumulus test compilation

* fix tests

* more small fixes

* fix merge

* bump runtime api version for westend and rollback version for rococo

---------

Signed-off-by: alindima <[email protected]>
Co-authored-by: Javier Viola <[email protected]>

Bumps [zstd](https://github.com/gyscos/zstd-rs) from 0.11.2+zstd.1.5.2 to 0.12.4.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/commits

)

---
updated-dependencies:
- dependency-name: zstd
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Directly use tokio::test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove old code

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Delete substrate-test-utils-test-crate

Also not needed anymore.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Rename squatted crates

This commit adds the staging- prefix to squatted crates so we can go forward and publish them to crates.io.

Using the staging- prefix is a temp fix until we decide on replacement names.
https://forum.parity.io/t/renaming-squated-crates-in-substrate-polkadot-cumulus/1964/6

* Fix test after crate renames

* Update Lockfile

* Add missing Cumulus licenses

* Typo

* Add missing Substrate licenses

* Single job checking the sub-repos in steps

* Remove dates

* Remove dates

* Add missing (C)

* Update FRAME UI tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update more UI tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

Bumps the known_good_semver group with 2 updates: [serde](https://github.com/serde-rs/serde) and [clap](https://github.com/clap-rs/clap).


Updates `serde` from 1.0.186 to 1.0.188
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.186...v1.0.188)

Updates `clap` from 4.4.0 to 4.4.1
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.4.0...v4.4.1

)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: known_good_semver
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: known_good_semver
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix `test-rustdoc`

* ".git/.scripts/commands/fmt/fmt.sh"

---------

Co-authored-by: command-bot <>

* Fix build profiles

Closes https://github.com/paritytech/polkadot-sdk/issues/1155



Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Manually set version to 1.0.0

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use workspace repo

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* 'Authors and Edition from workspace

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Manually fix conflicting ?

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove duplicates

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Autofix feature propagation

zepter lint propagate-feature --feature try-runtime --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="try-runtime:frame-try-runtime"
zepter lint propagate-feature --feature runtime-benchmarks --left-side-feature-missing=ignore --workspace --fix --feature-enables-dep="runtime-benchmarks:frame-benchmarking"
zepter lint propagate-feature --feature std --left-side-feature-missing=ignore --workspace --fix
zepter f f

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump zepter

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add some duplicates

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Add some duplicates"

This reverts commit c6ce6272

.

* Remove default enabled features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump Zepter

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump in correct location 🤦



Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* DNM: Add some mistakes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* DNM: Add some mistakes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "DNM: Add some mistakes"

This reverts commit d469b3f0.

* Revert "DNM: Add some mistakes"

This reverts commit d892a73a

.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* deps: replace lru with schnellru

* bring the peace to the galaxy

* add Rotko Networks to relay chain bootnodes

* add backed up key to westend

* Add CI for monorepo

* fix frame tests

* Format features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* add note for skipping tests and disable test-linux-stable-all

* Fix tests and compile issues (#1152)

* Fix feature dependant import

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump test timeout

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove feature gate

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add resolver 2

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove old lockfile

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Format features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix check-dependency-rules

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* rm test-runtime

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Actually fix script

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* enable cargo-check-each-crate-macos

* Run check-each-crate on 6 machines (#1163)

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>