* Companion for Substrate#14511

https://github.com/paritytech/substrate/pull/14511

* Make benchmarking build without any native runtime

* Fix warning

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* Implement runtime apis for fake runtime

These runtime api implementations are only used to make the compiler
think that we have implemented all required runtime apis. They will not
be called as we switch the executor to `WasmExecutor`. In the near
future we will not require these fake implementations anymore after
Substrate has shifted away from this compile time requirement.

This brings us the advantage that the `polkadot-service` doesn't need to
depend on the runtimes for getting the `RuntimeApi` type.

It also removes around 1min of build time on my machine ;)

* Fix warning

* FMT

* ".git/.scripts/commands/fmt/fmt.sh"

* Use more descriptive id

* Fix warnings

* Adapt path

* Fix 🙈

---------

Co-authored-by: command-bot <>

* Companion for Substrate#14474

This removes the requirement to pass the native dispatch.

* Adapt

* Better

* Remove patch

* ...

* Fix

* Update Substrate

We consider BEEFY mature enough to run by default on all nodes
for test networks (Rococo/Wococo/Versi).

Right now, most nodes are not running it since it's opt-in using
--beefy flag. Switch to an opt-out model for test networks.

Replace --beefy flag from CLI with --no-beefy and have BEEFY
client start by default on test networks.

Signed-off-by: acatangiu <[email protected]>

* PVF: Refactor workers into separate crates, remove host dependency

* Fix compile error

* Remove some leftover code

* Fix compile errors

* Update Cargo.lock

* Remove worker main.rs files

I accidentally copied these from the other PR. This PR isn't intended to
introduce standalone workers yet.

* Address review comments

* cargo fmt

* Update a couple of comments

* Update log targets

* upgrade js-sys to 0.3.61

* update wasm-bindgen-futures

* update once_cell

* update either

* update futures

* update pyroscope to 0.5.3

* use the new pyroscope API

* add missing imports

* readme is incorrect

* add shutdown

* fix shutdown

* remove mut

* remove mut 2

* redo updates

* update zeroize

* update lockfile for {"substrate"}

* update lockfile for {"substrate", "cumulus"}

* redo updates

* restart CI

---------

Co-authored-by: parity-processbot <>

* Move PVF workers into separate crate

* Fix indentation

* Fix compilation errors

* Fix more compilation errors

* Rename `worker.rs` files, make host interface to worker more clear

* Fix more compilation errors

* Fix more compilation errors

* Add link to issue

* Address review comments

* Update comment

* Happy New Year!

* Remove year entierly

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Remove years from copyright notice in the entire repo

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Check spawned worker version vs node version before PVF preparation

* Address discussions

* Propagate errors and shutdown preparation and execution pipelines properly

* Add logs; Fix execution worker checks

* Revert "Propagate errors and shutdown preparation and execution pipelines properly"

This reverts commit b96cc3160ff58db5ff001d8ca0bfea9bd4bdd0f2.

* Don't try to shut down; report the condition and exit worker

* Get rid of `VersionMismatch` preparation error

* Merge master

* Add docs; Fix tests

* Update Cargo.lock

* Kill again, but only the main node process

* Move unsafe code to a common safe function

* Fix libc dependency error on MacOS

* pvf spawning: Add some logging, add a small integration test

* Minor fixes

* Restart CI

---------

Co-authored-by: Marcin S <[email protected]>

* Adjust command

* Move imports

* Chain-specific blocktime

* update lockfile for {"substrate"}

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix bench

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Export constants from node service

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use constants in CLI

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Piotr Mikołajczyk <[email protected]>
Co-authored-by: parity-processbot <>

This feature should only be activated by the polkadot binary. Otherwise
parachains may accidentally activate this feature.

* Storage monitor added to polkadot node

Related to: https://github.com/paritytech/substrate/issues/12399

* Cargo.lock updated

* Cargo.lock update

* Revert "Cargo.lock update"

This reverts commit 04dfe5fe65ea7d1476d4632ad647f2c1c669ab1a.

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* missing_docs fixes

* Cargo.lock updated

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <[email protected]>

* Warn validators with slow hardware

* update

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix printing logic

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fix

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* add deps

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* BlockId removal: refactor: HeaderBackend::header

It changes the arguments of:
- `HeaderBackend::header`,
- `Client::header`

methods from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* missed fixes

* BlockId removal: refactor: HeaderBackend::expect_header

It changes the arguments of `HeaderBackend::expect_header` method from: `BlockId<Block>` to: `Block::Hash`

* update lockfile for {"substrate"}

* misspell fixed

Co-authored-by: parity-processbot <>

* update to reflect latest try-runtime stuff

* update to latest version

* fix

* fix miner

* update

* update

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Begin removing `parity-util-mem`; remove `collect_memory_stats`

* Update some dependencies that were using `parity-util-mem`

* Remove `trie-memory-tracker` feature

* Update Cargo.lock

* Update `kvdb-shared-tests`

* Add back jemalloc

* Add missing license header

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
Co-authored-by: Andronik <[email protected]>

* rust 1.64 enables workspace properties

* add edition, repository and authors.

* of course, update the version in one place.

Co-authored-by: Andronik <[email protected]>

* Add clippy config and remove .cargo from gitignore

* first fixes

* Clippyfied

* Add clippy CI job

* comment out rusty-cachier

* minor

* fix ci

* remove DAG from check-dependent-project

* add DAG to clippy

Co-authored-by: alvicsam <[email protected]>

* westend: update transaction version

* polkadot: update transaction version

* kusama: update transaction version

* Bump spec_version to 9330

* bump versions to 0.9.33

* Remove the `wasmtime` feature flag

* Update `substrate` to the newest `master`

* Update `substrate` to the newest `master`

* Bump spec_version to 9310

* bump transaction_version (0.9.31) (#6171)

* Bump transaction_version for polkadot

* Bump transaction_version for kusama

* Bump transaction_version for rococo

* Bump transaction_version for westend

* Bump transaction_version for polkadot

* Bump transaction_version for kusama

* Bump transaction_version for rococo

* Bump transaction_version for westend

* Bump crate versions (0.9.31)

* Move clap to 4.0.9

* Remove "deprecated" feature flag

* Convert to default_value_t

* update lockfile for {"substrate"}

* Add group(skip) to avoid naming conflict

* More group(skip)

Co-authored-by: parity-processbot <>

* Expose the full Cli through malus

Signed-off-by: Andrei Sandu <[email protected]>

* fix lonely test

Signed-off-by: Andrei Sandu <[email protected]>

Signed-off-by: Andrei Sandu <[email protected]>

* Add `DisputeState` to `DisputeCoordinatorMessage::RecentDisputes`

The new signature of the message is:
```
RecentDisputes(oneshot::Sender<Vec<(SessionIndex, CandidateHash, DisputeStatus)>>),
```

As part of the change also add `DispiteStatus` to
`polkadot_node_primitives`.

* Move dummy_signature() in primitives/test-helpers

* Enable staging runtime api on Rococo

* Implementation

* Move disputes to separate module
* Vote prioritisation
* Duplicates handling
* Double vote handling
* Unit tests
* Logs and metrics
* Code review feedback
* Fix ACTIVE/INACTIVE separation and update partition names
* Add `fn dispute_is_inactive` to node primitives and refactor `fn get_active_with_status()` logic
* Keep the 'old' logic if the staging api is not enabled
* Fix some comments in tests
* Add warning message if there are any inactive_unknown_onchain disputes
* Add file headers and remove `use super::*;` usage outside tests
* Adding doc comments
* Fix test methods names

* Fix staging api usage

* Fix `get_disputes` runtime function implementation

* Fix compilation error

* Fix arithmetic operations in tests

* Use smaller test data

* Rename `RuntimeApiRequest::StagingDisputes` to `RuntimeApiRequest::Disputes`

* Remove `staging-client` feature flag

* fmt

* Remove `vstaging` feature flag

* Some comments regarding the staging api

* Rename dispute selection modules in provisioner
with_staging_api -> prioritized_selection
without_staging_api -> random_selection

* Comments for staging api

* Comments

* Additional logging

* Code review feedback

process_selected_disputes -> into_multi_dispute_statement_set
typo
In trait VoteType: vote_value -> is_valid

* Code review feedback

* Fix metrics

* get_disputes -> disputes

* Get time only once during partitioning

* Fix partitioning

* Comments

* Reduce the number of hardcoded api versions

* Code review feedback

* Unused import

* Comments

* More precise log messages

* Code review feedback

* Code review feedback

* Code review feedback - remove `trait VoteType`

* Code review feedback

* Trace log for DisputeCoordinatorMessage::QueryCandidateVotes counter in vote_selection

* Enable correct features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Feature gate function

The expose_db function is currently feature gated with runtime-benchmarks.
After the related Substrate MR, this feature gate will actually *work*
as intended instead of doing nothing.
BUT then we also have to pass through the correct feature, hence this MR.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

* Fix bench features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fix more features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>

* Bump transaction_version for westend

* Bump transaction_version for rococo

* Bump transaction_version for kusama

* Bump transaction_version for polkadot

* Bump spec_version to 9290

* Bump crate versions

* Reflect benchmarking fn signature change

* fmt

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Bump crate versions

* Bump spec_version to 9280 for kusama

* Bump spec_version to 9280 for polkadot

* Bump spec_version to 9280 for rococo

* Bump spec_version to 9280 for westend

* update Cargo.lock

Co-authored-by: parity-processbot <>

* Bump crate versions

* Update cargo.lock

* add malus_finality_delay flag

* cargo update -p sp-io

* no warnings when malus is disabled

* subpar workaround for feature unification problem

* remove malus_finality_delay from regular cli

* document finality_delay param, rename cli arg

* Cleanup light client leftovers

* Remove light clent leftovers in test-parachains

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* allow re-use and avoid compiling kusama parachain code

* fixup removed trailing ;

* make it compat with rustfmt +nightly

* Expose 'benchmark extrinsic' command

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Split benchmarking into own mod

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Simplify code

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fixup

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Spell

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert Cargo.lock updates

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

* Fix brittle test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump spec version to 9270

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Sleep in test to make it work in CI...

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Disable failing test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Bump spec version to 9270"

This reverts commit c1c385d7a4dc849e7f4d4723740aec66c2b7be09.

* Delete brittle test, see #5788

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Disable failing test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Delete more failing tests...

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Co-authored-by: parity-processbot <>

* Bump spec_version to 9260

* Version bump to v0.9.26

* kusama: bump spec_version to 9250

* polkadot: bump spec_version to 9250

* rococo: bump spec_version to 9250

* westend: bump spec_version to 9250

* bump version to 0.9.25

* bump transaction version (polkadot & kusama) (#5690)

* kusama: bump transaction_version to 12

* polkadot: bump transaction_version to 13

* bump version to 0.9.23

* bump spec_version to 9230

* bump versions to 0.9.22

* bump spec_version to 9220