Adds a `NodeFeatures` bitfield value to the runtime `HostConfiguration`,
with the purpose of coordinating the enabling of node-side features,
such as: https://github.com/paritytech/polkadot-sdk/issues/628 and
https://github.com/paritytech/polkadot-sdk/issues/598.
These are features that require all validators enable them at the same
time, assuming all/most nodes have upgraded their node versions.

This PR doesn't add any feature yet. These are coming in future PRs.

Also adds a runtime API for querying the state of the client features
and an extrinsic for setting/unsetting a feature by its index in the bitfield.

Note: originally part of:
https://github.com/paritytech/polkadot-sdk/pull/1644, but posted as
standalone to be reused by other PRs until the initial PR is merged

Co-authored-by: Marcin S <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

The staging- rename commit was missing from the last PR for some reason.

Missing descriptions (47):  

- [x] `cumulus/client/collator/Cargo.toml`
- [x] `cumulus/client/relay-chain-inprocess-interface/Cargo.toml`
- [x] `cumulus/client/cli/Cargo.toml`
- [x] `cumulus/client/service/Cargo.toml`
- [x] `cumulus/client/relay-chain-rpc-interface/Cargo.toml`
- [x] `cumulus/client/relay-chain-interface/Cargo.toml`
- [x] `cumulus/client/relay-chain-minimal-node/Cargo.toml`
- [x] `cumulus/parachains/pallets/parachain-info/Cargo.toml`
- [x] `cumulus/parachains/pallets/ping/Cargo.toml`
- [x] `cumulus/primitives/utility/Cargo.toml`
- [x] `cumulus/primitives/aura/Cargo.toml`
- [x] `cumulus/primitives/core/Cargo.toml`
- [x] `cumulus/primitives/parachain-inherent/Cargo.toml`
- [x] `cumulus/test/relay-sproof-builder/Cargo.toml`
- [x] `cumulus/pallets/xcmp-queue/Cargo.toml`
- [x] `cumulus/pallets/dmp-queue/Cargo.toml`
- [x] `cumulus/pallets/xcm/Cargo.toml`
- [x] `polkadot/erasure-coding/Cargo.toml`
- [x] `polkadot/statement-table/Cargo.toml`
- [x] `polkadot/primitives/Cargo.toml`
- [x] `polkadot/rpc/Cargo.toml`
- [x] `polkadot/node/service/Cargo.toml`
- [x] `polkadot/node/core/parachains-inherent/Cargo.toml`
- [x] `polkadot/node/core/approval-voting/Cargo.toml`
- [x] `polkadot/node/core/dispute-coordinator/Cargo.toml`
- [x] `polkadot/node/core/av-store/Cargo.toml`
- [x] `polkadot/node/core/chain-api/Cargo.toml`
- [x] `polkadot/node/core/prospective-parachains/Cargo.toml`
- [x] `polkadot/node/core/backing/Cargo.toml`
- [x] `polkadot/node/core/provisioner/Cargo.toml`
- [x] `polkadot/node/core/runtime-api/Cargo.toml`
- [x] `polkadot/node/core/bitfield-signing/Cargo.toml`
- [x] `polkadot/node/network/dispute-distribution/Cargo.toml`
- [x] `polkadot/node/network/bridge/Cargo.toml`
- [x] `polkadot/node/network/collator-protocol/Cargo.toml`
- [x] `polkadot/node/network/approval-distribution/Cargo.toml`
- [x] `polkadot/node/network/availability-distribution/Cargo.toml`
- [x] `polkadot/node/network/bitfield-distribution/Cargo.toml`
- [x] `polkadot/node/network/gossip-support/Cargo.toml`
- [x] `polkadot/node/network/availability-recovery/Cargo.toml`
- [x] `polkadot/node/collation-generation/Cargo.toml`
- [x] `polkadot/node/overseer/Cargo.toml`
- [x] `polkadot/runtime/parachains/Cargo.toml`
- [x] `polkadot/runtime/common/slot_range_helper/Cargo.toml`
- [x] `polkadot/runtime/metrics/Cargo.toml`
- [x] `polkadot/xcm/pallet-xcm-benchmarks/Cargo.toml`
- [x] `polkadot/utils/generate-bags/Cargo.toml`
- [x]  `substrate/bin/minimal/runtime/Cargo.toml`

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Signed-off-by: alindima <[email protected]>
Co-authored-by: ordian <[email protected]>
Co-authored-by: Tsvetomir Dimitrov <[email protected]>
Co-authored-by: Marcin S <[email protected]>
Co-authored-by: alindima <[email protected]>
Co-authored-by: Sebastian Kunert <[email protected]>
Co-authored-by: Dmitry Markin <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

**_PR migrated from https://github.com/paritytech/polkadot/pull/6782_** 

This PR will upgrade the network protocol to version 3 -> VStaging which
will later be renamed to V3. This version introduces a new kind of
assignment certificate that will be used for tranche0 assignments.
Instead of issuing/importing one tranche0 assignment per candidate,
there will be just one certificate per relay chain block per validator.
However, we will not be sending out the new assignment certificates,
yet. So everything should work exactly as before. Once the majority of
the validators have been upgraded to the new protocol version we will
enable the new certificates (starting at a specific relay chain block)
with a new client update.

There are still a few things that need to be done:

- [x] Use bitfield instead of Vec<CandidateIndex>:
https://github.com/paritytech/polkadot/pull/6802


  - [x] Fix existing approval-distribution and approval-voting tests
  - [x] Fix bitfield-distribution and statement-distribution tests
  - [x] Fix network bridge tests
  - [x] Implement todos in the code
  - [x] Add tests to cover new code
  - [x] Update metrics
  - [x] Remove the approval distribution aggression levels: TBD PR
  - [x] Parachains DB migration 
  - [x] Test network protocol upgrade on Versi
  - [x] Versi Load test
  - [x] Add Zombienet test
  - [x] Documentation updates
- [x] Fix for sending DistributeAssignment for each candidate claimed by
a v2 assignment (warning: Importing locally an already known assignment)
 - [x]  Fix AcceptedDuplicate
 - [x] Fix DB migration so that we can still keep old data.
 - [x] Final Versi burn in

---------

Signed-off-by: Andrei Sandu <[email protected]>
Signed-off-by: Alexandru Gheorghe <[email protected]>
Co-authored-by: Alexandru Gheorghe <[email protected]>

Co-authored-by: Marcin S <[email protected]>

While investigating some db migrations that make the node startup fail,
I noticed that the node wasn't exiting and that the log file were
growing exponentially, until my whole system was freezing and that makes
it really hard to actually find why it was failing in the first place.

E.g:
```
 ls -lh /tmp/zombie-01a04c2a2c0265d85f6440cf01c0f44a_-51319-uyggzuD4wEpV/bob.log
 32,6G oct 27 11:16 /tmp/zombie-01a04c2a2c0265d85f6440cf01c0f44a_-51319-uyggzuD4wEpV/bob.log
```

This was happening because the following errors were being printed
continously without the subsystem main loop exiting:

From dispute-coordinator:
```
WARN tokio-runtime-worker parachain::dispute-coordinator: error=Subsystem(Generated(Context("Signal channel is terminated and empty.")))
```

From availability recovery:
```
Erasure task channel closed. Node shutting down ?
```

Signed-off-by: Alexandru Gheorghe <[email protected]>

Fixes #768.

in-progress PR adding new tests and solving bugs

---------

Co-authored-by: Bradley Olson <[email protected]>
Co-authored-by: eskimor <[email protected]>
Co-authored-by: eskimor <[email protected]>
Co-authored-by: Andrei Sandu <[email protected]>

closes #695

Could potentially be helpful to preserving caches when applicable, as
discussed in #685

kusama address: FvpsvV1GQAAbwqX6oyRjemgdKV11QU5bXsMg9xsonD1FLGK

closes #622 

Pros:
* simpler interface, just functions:
`create_runtime_from_artifact_bytes()` and `execute_artifact()`

Cons:
* extra overhead of constructing executor semantics each time

I could make it a combination of
* `create_runtime_config(params)` (such that we could clone the
constructed semantics)
* `create_runtime(blob, config)`
* `execute_artifact(blob, config, params)`

Not sure if it's worth it though.

---------

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Marcin S <[email protected]>

# Description

In a couple of cases, there were links pointing to the w3f github pages
domain. In other instances, there were links pointing to the old
polkadot repo's github pages. Both of these are now pointing to the
relevant links in
https://paritytech.github.io/polkadot-sdk/book/index.html.

These changes were made specifically because the w3f github pages
returns a 404, and while fixing the links, the old polkadot repo links
were touched up as well even if they do redirect properly.

This shouldn't affect anything as these are documentation link changes
only.

Exposes disabled validators list via a runtime API.

---------

Co-authored-by: ordian <[email protected]>
Co-authored-by: ordian <[email protected]>

Co-authored-by: command-bot <>

Follow-up to #1543.

- Async-backing related primitives are stable `primitives::v6`
- Async-backing API is now part of `api_version(7)`
- It's enabled on Rococo and Westend runtimes

---------

Signed-off-by: Andrei Sandu <[email protected]>
Co-authored-by: Andrei Sandu <[email protected]>

It seems the old strategy have been depracted more than one year. 
So maybe it's time to clean up old strategy for wasm executor.


---
polkadot address: 15ouFh2SHpGbHtDPsJ6cXQfes9Cx1gEFnJJsJVqPGzBSTudr

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Koute <[email protected]>

In follow-up to https://github.com/paritytech/polkadot-sdk/pull/1518



Adding extra tests for inclusion pruning. Primarily focusing on various
cases surrounding candidates included in different forks (with different
relay parents).

All cases fall into a few buckets based on 3 degrees of freedom - number
of candidates, number of blocks (height), number of forks + extra case
for pruning multiple heights at once.

Added small tweak to the original pruning function to disregard stale
candidate duplicates which should keep the same behaviour.

---------

Co-authored-by: Tsvetomir Dimitrov <[email protected]>

In follow-up to https://github.com/paritytech/polkadot-sdk/pull/1432

Some additional unit tests for the inclusion struct used in the scraper.

Closes #583

After the separation of PVF worker binaries, dedicated puppet workers
are not needed for tests anymore. The production workers can be used
instead, avoiding some code duplication and decreasing complexity.

The changes also make it possible to further refactor the code to
isolate workers completely.

* Remove redundant book keeping in dispute coordinator.

* Fix docs.

---------

Co-authored-by: eskimor <[email protected]>

Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.3.4 to 0.4.1.
- [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.3.4...hex-literal-v0.4.1

)

---
updated-dependencies:
- dependency-name: hex-literal
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bastian Köcher <[email protected]>

* polkadot: propagate UnpinHandle to ActiveLeafUpdate

Also extract the leaf creation for tests
into a common function.

* dispute-coordinator: try pinned blocks for slashin

* apparently 1.72 is smarter than 1.70

* address nits

* rename fresh_leaf to new_leaf

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.47 to 1.0.48.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.47...1.0.48

)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Liam Aharon <[email protected]>

* Fix wrong ref counting

in case of multiple inserts at same height.

* More warn

---------

Co-authored-by: eskimor <[email protected]>