* runtimes: use updated BeefyApi

* update lockfile for substrate

Basically updates the docs for the paras module.

This commit hooks up the API provided by #4457 to the runtime API
subsystem. In a following PR this API will be consumed by the PVF
pre-checking subsystem.

Co-authored-by: Chris Sosnin <[email protected]>

Co-authored-by: Chris Sosnin <[email protected]>

* parachains: Fix configuration module

Closes #4529
Closes #4533

I figured that trying to avoid updates does not really worth it to keep.
This is because we seem to not update the configuration often and when
we do we approach this carefully. Thus possibility of a redundant update
is really negligable. At the same time, if such a redundant update does
happen then the effects of that are really small: just some wasted
storage interactions.

On the other hand, making it work was a little bit annoying. With the
proper fix for the pending updates this would be even more annoying
since now we would have to add combinatorically more cases to test this.

So I figured that I will just scrap that and simplify the code.

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::configuration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_configuration.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::configuration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_configuration.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::configuration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_configuration.rs

* review fixes

Co-authored-by: Parity Bot <[email protected]>

Bumps [bounded-vec](https://github.com/ergoplatform/bounded-vec) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/ergoplatform/bounded-vec/releases)
- [Changelog](https://github.com/ergoplatform/bounded-vec/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/ergoplatform/bounded-vec/compare/v0.4.0...v0.5.0

)

---
updated-dependencies:
- dependency-name: bounded-vec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Substrate companion for change slot duration fetching

https://github.com/paritytech/substrate/pull/10509

* Fix bridges

* update lockfile for substrate

Co-authored-by: parity-processbot <>

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.18 to 0.3.19.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.18...0.3.19

)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.0...0.7.1

)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add existential deposit

* cargo +nightly fmt

* add EXISTENTIAL_DEPOSIT to runtimes

* remove custom kube file in creds

* typo in image name echo

Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.15 to 0.14.16.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.15...v0.14.16

)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Create a README for XCMv2 detailing notable changes

* Typo

* Appease spellcheck and cargo fmt

* Add XCM pallet item to be aware of

* cargo fmt

* Remove all XCM README.md

* enable disputes, for all known chains but polkadot

* chore: fmt

* don't propagate disputes either

* review

* remove disputes feature

* remove superfluous line

* Update node/service/src/lib.rs

Co-authored-by: Andronik Ordian <[email protected]>

* fixup

* allow being a dummy

* rialto

* add an enum, to make things work better

* overseer

* fix test

* comments

* move condition out

* excess arg

Co-authored-by: Andronik Ordian <[email protected]>

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.131 to 1.0.132.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.131...v1.0.132

)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [nix](https://github.com/nix-rust/nix) from 0.23.0 to 0.23.1.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/compare/v0.23.0...v0.23.1

)

---
updated-dependencies:
- dependency-name: nix
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fetch ancestors of the activated leaf

* Replace param type

* Increase step size

* Request block numbers for ancestors

* Store activated leaves in lru cache

* Fix doc-comment

* Rework LRU usage

* Typos & formatting

* Handle errors better

* Introduce a size limit for the ancestry

* Return fatal error when fetching finalized block

* Update tests

* Add tests for ordering provider

* Better naming

* fix zombienet test, new version

* zombienet add debug

* debug zombienet

Co-authored-by: Javier Viola <[email protected]>

* test new version of zombienet

* use default chain spec command

* use commit ref to get test files

* change test spec for zombienet new versionzombienet_tests/parachains/0001-parachains-smoke-test.toml

* changes for new version of zombienet

* use paritytech zombienet version

Simply extract hardcoded log target into a const.

* pvf-precheck: Integrate PVF pre-checking into paras module

Closes #4009

This is the most of the runtime-side change needed for #3211.

Here is how it works.

The PVF pre-checking can be triggered either by an upgrade or by
onboarding (i.e. calling `schedule_para_initialize`). The PVF
pre-checking process is identified by the PVF code hash that is being
voted on. If there is already PVF pre-checking process running, then no
new PVF pre-checking process will be started. Instead, we just subscribe
to the existing one.

If there is no PVF pre-checking process running but the PVF code hash
was already saved in the storage, that necessarily means (I invite the
reviewers to double-check this invariant) that the PVF already passed
pre-checking. This is equivalent to instant approving of the PVF.

The pre-checking process can be concluded either by obtaining a
supermajority or if it expires.

Each validator checks the list of PVFs available for voting. The vote is
binary, i.e. accept or reject a given PVF. As soon as the supermajority
of votes are collected for one of the sides of the vote, the voting is
concluded in that direction and the effects of the voting are enacted.

Only validators from the active set can participate in the vote. The set
of active validators can change each session. That's why we reset the
votes each session. A voting that observed a certain number of sessions
will be rejected.

The effects of the PVF accepting depend on the operations requested it:

1. All onboardings subscribed to the approved PVF pre-checking process will
get scheduled and after passing 2 session boundaries they will be onboarded.
2. All upgrades subscribed to the approved PVF pre-checking process will
get scheduled very similarly to the existing process. Upgrades with
pre-checking are really the same process that is just delayed by the
time required for pre-checking voting. In case of instant approval the
mechanism is exactly the same. This is important from parachains
compatibility standpoint since following the delayed upgrade requires
the parachain to implement
https://github.com/paritytech/cumulus/pull/517

.

In case, PVF pre-checking process was concluded with rejection, then all
the requesting operations get cancelled. For onboarding it means it gets
without movement: the lifecycle of such parachain is terminated on the
`Onboarding` state and after rejection the lifecycle is none. That in
turn means that the caller can attempt registering the parachain once
more. For upgrading it means that the upgrade process is aborted: that
flashes go-ahead signal with `Abort` flag.

Rejection leads to removing the allegedly bad validation code from the
chain storage. Among other things, this implies that the operation can
be re-requested. That allows for retrying an operation in case there was
some bug. At the same time it does not look as a DoS vector due to the
caching performed by the nodes.

PVF pre-checking can be enabled and disabled. Initially, according to
the changes in #4420, this mechanism is disabled. Triggering the PVF
pre-checking when it is disabled just means that we insta approve the
requesting operation. This should lead to the behavior being unchanged.

Follow-ups:

- expose runtime APIs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

* Review fixes

Co-authored-by: Parity Bot <[email protected]>

Co-authored-by: Andronik Ordian <[email protected]>

* Add runtime metrics provider

Signed-off-by: Andrei Sandu <[email protected]>

* Runner changes

Signed-off-by: Andrei Sandu <[email protected]>

* Some sample metrics in paras_inherent

Signed-off-by: Andrei Sandu <[email protected]>

* update cargo toml

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* bug

Signed-off-by: Andrei Sandu <[email protected]>

* more fmt after merge

Signed-off-by: Andrei Sandu <[email protected]>

* Refactor metric prefix override

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* remove bug comment

Signed-off-by: Andrei Sandu <[email protected]>

* Add runtime metric primitives

Signed-off-by: Andrei Sandu <[email protected]>

* Impl trace event parsing

Signed-off-by: Andrei Sandu <[email protected]>

* Update metrics

Signed-off-by: Andrei Sandu <[email protected]>

* cargo lock

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* Fix target check

Signed-off-by: Andrei Sandu <[email protected]>

* Runtime metrics primitives

Signed-off-by: Andrei Sandu <[email protected]>

* Review feedback

Signed-off-by: Andrei Sandu <[email protected]>

* Runtime metrics crate

Signed-off-by: Andrei Sandu <[email protected]>

* Node side runtime metric changes

Signed-off-by: Andrei Sandu <[email protected]>

* use runtime CounterVec instead of macro

Signed-off-by: Andrei Sandu <[email protected]>

* fmt nice

Signed-off-by: Andrei Sandu <[email protected]>

* remove dead code

Signed-off-by: Andrei Sandu <[email protected]>

* base58 decoding

Signed-off-by: Andrei Sandu <[email protected]>

* base58 encoding

Signed-off-by: Andrei Sandu <[email protected]>

* fix warn

Signed-off-by: Andrei Sandu <[email protected]>

* typo

Signed-off-by: Andrei Sandu <[email protected]>

* Review feedback

Signed-off-by: Andrei Sandu <[email protected]>

* Finish label support

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* please compile

Signed-off-by: Andrei Sandu <[email protected]>

* add feature gate

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* Comment cargo toml

Signed-off-by: Andrei Sandu <[email protected]>

* Fix cargo toml description

Signed-off-by: Andrei Sandu <[email protected]>

* Update doc.

Signed-off-by: Andrei Sandu <[email protected]>

* switch to `runtime-metrics` feature

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* cargo toml

Signed-off-by: Andrei Sandu <[email protected]>

* fix tests

Signed-off-by: Andrei Sandu <[email protected]>

* fixes

Signed-off-by: Andrei Sandu <[email protected]>

* better ux

Signed-off-by: Andrei Sandu <[email protected]>

* from_utf8_unchecked is safe

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* Add Counter and refactor

Signed-off-by: Andrei Sandu <[email protected]>

* Fixes

Signed-off-by: Andrei Sandu <[email protected]>

* review fixes

Signed-off-by: Andrei Sandu <[email protected]>

* more fixes

Signed-off-by: Andrei Sandu <[email protected]>

* add integration test

Signed-off-by: Andrei Sandu <[email protected]>

* dev deps

Signed-off-by: Andrei Sandu <[email protected]>

* gitlab script update

Signed-off-by: Andrei Sandu <[email protected]>

* review fixes

Signed-off-by: Andrei Sandu <[email protected]>

* fix merge damage

Signed-off-by: Andrei Sandu <[email protected]>

* Run tests twice

Signed-off-by: Andrei Sandu <[email protected]>

* small fix

Signed-off-by: Andrei Sandu <[email protected]>

* typo

Signed-off-by: Andrei Sandu <[email protected]>

* cargo lock

Signed-off-by: Andrei Sandu <[email protected]>

* tests

Signed-off-by: Andrei Sandu <[email protected]>

* spellcheck happy ?

Signed-off-by: Andrei Sandu <[email protected]>

* more fixes

Signed-off-by: Andrei Sandu <[email protected]>

* Guard tracing init

Signed-off-by: Andrei Sandu <[email protected]>

* missing copyright

Signed-off-by: Andrei Sandu <[email protected]>

* update lockfile for substrate

Co-authored-by: parity-processbot <>

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.14.0...tokio-1.15.0

)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [mick-jaeger](https://github.com/tomaka/mick-jaeger) from 0.1.6 to 0.1.7.
- [Release notes](https://github.com/tomaka/mick-jaeger/releases)
- [Commits](https://github.com/tomaka/mick-jaeger/commits

)

---
updated-dependencies:
- dependency-name: mick-jaeger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Versi chainspec based on Rococo.

* Fix.

* Fix compilation without features.

* Spelling

* Another typo.

* Support versi in native version

* 🤦



* Regenerated genesis for versi.

* Update cli/src/command.rs

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.33 to 1.0.34.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](https://github.com/dtolnay/proc-macro2/compare/1.0.33...1.0.34

)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* make compiler happy

* node side compile warning quiet

* ffs

* malus: remove unused keystore

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.109 to 0.2.112.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.109...0.2.112

)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.72...v1.0.73

)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

We wanted to change niceness to accomodate the fact that some of the
preparation tasks are low priority. For example, when a node sees that
there is a new para was onboarded the node may start preparing right
away. Since all other activities are more important, such as network I/O
or validation of the backed candidates and preparation of the
immediatelly needed PVFs.

However, it turned out that this approach does not work: generally
non-root processes can only decrease niceness and they cannot increase
it to the previous value, as was assumed by the code.

Apart from that, https://github.com/paritytech/polkadot/pull/4123
assumes all PVFs are prepared in the same way. Specifically, that if a
PVF preparation failed before, then PVF pre-checking will also report
that it was failed, even though it could happen that preparation failed
due to being low-priority. In order to avoid such cases, we decided to
simplify the whole preparation model. Preparation under low priority
does not work well with that.

Closes https://github.com/paritytech/polkadot/issues/4520