* Upgrade `kvdb-*`, `trie-db` and `memory-db`

The updates of `trie-db` and `memory-db` are important, as they fix the
non-deterministic build of Polkadot/Substrate.

* Change `trie-db` version

* Update test-utils/runtime/Cargo.toml

Co-authored-by: Andronik Ordian <[email protected]>

* Update primitives/trie/Cargo.toml

Co-authored-by: Andronik Ordian <[email protected]>

* Update `Cargo.lock` and `trie-bench`

* Fix UI tests

* Switch to fixed version of memory-db

Co-authored-by: Andronik Ordian <[email protected]>

* Use async/await in build_network_future

* Address concerns

* Fix test

* slots: create primitives crate for consensus slots

* offences: add method to check if an offence is unknown

* babe: initial equivocation reporting implementation

* babe: organize imports

* babe: working equivocation reporting

* babe: add slot number to equivocation proof

* session: move duplicate traits to session primitives

* babe: move equivocation stuff to its own file

* offences: fix test

* session: don't have primitives depend on frame_support

* babe: use opaque type for key owner proof

* babe: cleanup client equivocation reporting

* babe: cleanup equivocation code in pallet

* babe: allow sending signed equivocation reports

* node: fix compilation

* fix test compilation

* babe: return bool on check_equivocation_proof

* babe: add test for equivocation reporting

* babe: add more tests

* babe: add test for validate unsigned

* babe: take slot number in generate_key_ownership_proof API

* babe: add benchmark for equivocation proof checking

* session: add benchmark for membership proof checking

* offences: fix babe benchmark

* babe: add weights based on benchmark results

* babe: adjust weights after benchmarking on reference hardware

* babe: reorder checks in check_and_report_equivocation

* primitives: use generic Header in testing runtime

* frame: remove unused imports

* Remove warning

Co-authored-by: Bastian Köcher <[email protected]>

* Move `create_inherents` into the block-builder

This moves the `create_inherents` call into the block-builder. This has
the advantage that `create_inherents` will be able to reuse the same
context that will be used when applying the extrinsics and we also save
one call to `on_initialize`. To make sure that `create_inherents` does
not modify any state, we execute it in a transaction that is
rolled-back after doing the runtime call.

* Feedback and build fix

* Update primitives/runtime/src/lib.rs

Co-authored-by: Sergei Shulepov <[email protected]>

* Update client/block-builder/src/lib.rs

Co-authored-by: Sergei Shulepov <[email protected]>

* Update SubstrateCli to return String

* Add default implementation for executable_name()

* Use display instead of PathBuf

* Get file_name in default impl of executable_name

* Remove String::from and use .into()

* Use default impl for executable_name()

* Use .as_str() and remove useless .to_string()

* Update only sp-io when running companion build

* Remove unneeded update of sp-io in CI

Co-authored-by: Cecile Tonglet <[email protected]>

* Initial commit

Forked at: 4919c808
Parent branch: origin/master

* Drop the tokio runtime before the task_manager

The tokio runtime must be dropped before the task_manager. Otherwise the objects the task_manager
keep alive are dropped before the tasks are finished.

* :)

* Slight tidy

* Remove ServiceBuilderCommand

* Remove whitespace

* Keep task manager alive for check_block/import_blocks

* Pass task_manager to run_until_exit

* Make task_manager in run_until_exit and make subcommands async

* Change the async_run fn to return a future and task manager

* async_run should take a result fn

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Fix spaces in export_raw_state

Co-authored-by: Bastian Köcher <[email protected]>

* Restrict `Protected` to some heap types.

* Comment abut Protected usage.

* Remove Protected from crypto, use secrecy crate for existing uses.

* use a parse function

* fix error convert

* Rename and move secretY string function.

* std result

* Fix mocking multiple http calls in the same function call

Fixes an issue where a function call would perform more than one http request and wait for each to complete before proceeding. The `RequestId` comes from the length of the `requests` collection in the `OffchainState` and if a request is completed before the next one starts it will be removed and the "next expected" will be off by one. This PR tries to fix that by using a request counter that tracks how many requests have been performed so that we can `remove()` items from the `expected_requests` at the right index.

I suspect that this is a sub-optimal soluton and perhaps requests and their mocks should live side by side in the same collection, e.g. in a tuple of `(PendingRequest, Option<ExpectedRequest>)`.

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <[email protected]>

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <[email protected]>

* Panic on overflow

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Use a Deque and push/pop expected requests

* fix test

Co-authored-by: Bernhard Schuster <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* grandpa: replace Result<(), ()> with Option<()>

* grandpa: replace &Option<T> with Option<&T>

* grandpa: cleanup local id and keystore usages

* grandpa: return bool on check_message_signature

* grandpa: fix erroneous log message on startup

* grandpa: fix test

* Start

* Remove debug println

* Add tests

* WIP

* Making progress

* Almost ready

* Get service tests compiling

* Fix node screenshot

* Line widths

* Fix node cli tests

* Fix node cli warning

* ChainComponents -> ServiceComponents, fix tests

* make spawn_handle public

* Remove spawnnamed impl for taskmanager

* Move the keep alive stuff to the task manager

* Move the telemetry, base path, rpc keep_alive to the service builder

* Make the task manager keep alive an internal detail

* Rewrite the browser start_client future

* Remove run_node etc

* Revert my personal changes to browser-demo/build.sh

* use |config|

* Add a runtime_version function to SubstrateCli

* Reexport role and runtime version from sc cli

* Update Cargo.lock

* runtime_version -> native_runtime_version

* Pass chain spec to native_runtime_version for polkadot

* Fix line widths

* Traitify ServiceComponents Client

* Fix tx-pool returning the same transaction multiple times

This fixes a bug that lead to returning the same transaction multiple
times when iterating the `ready` iterator. Internally the transaction
was kept in the `best` list and could be duplicated in that list be
re-inserting it again. This `best` list is using a `TransactionRef`
which internally uses a `insertion_id`. This `insertion_id` could lead
to the same transaction being inserted multiple times into the `best`
list.

* Update client/transaction-pool/src/testing/pool.rs

Co-authored-by: Nikolay Volf <[email protected]>

Co-authored-by: Nikolay Volf <[email protected]>

* Update to libp2p-0.20.0

* Update to `libp2p-0.20.1`.

Co-authored-by: Pierre Krieger <[email protected]>

* Increase the limit for the maximum size of the telemetry name

* Fix test

* Fix an extra semi-colon yielding a wrong error

* Update client/cli/src/commands/run_cmd.rs

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Martin Pugh <[email protected]>

* feat: Include genesis hash in system.connected message for telemetry

* chore: Spread call arguments into multiline

* chore: Removed commented code

* initial mockup

* add and wipe

* track writes

* start to add to pipeline

* return all reads/writes

* Log reads and writes from bench db

* causes panic

* Allow multiple commits

* commit before ending benchmark

* doesn't work???

* fix

* Update lib.rs

* switch to struct for `BenchmarkResults`

* add to output

* fix test

* line width

* @Kianenigma

 review

* Add Whitelist to DB Tracking in Benchmarks Pipeline (#6405)

* hardcoded whitelist

* Add whitelist to pipeline

* Remove whitelist pipeline from CLI, add to runtime

* clean-up unused db initialized whitelist

* Add regression analysis to DB Tracking (#6475)

* Add selector

* add tests

* debug formatter for easy formula

* Update client/db/src/bench.rs

Co-authored-by: Kian Paimani <[email protected]>

Co-authored-by: arkpar <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>

* Initial commit

Forked at: 0c42ceda


No parent branch.

* Ensure the listen addresses are consistent with the transport

* Update client/network/src/error.rs

* Update client/network/src/service.rs

* Better implementation

* Fix bad previous impl

* add boot_nodes

* reserved nodes

* test boot nodes

* reserved nodes tests

* add public_addresses and make specific error type

* Update client/network/src/error.rs

Co-authored-by: Pierre Krieger <[email protected]>

Co-authored-by: Pierre Krieger <[email protected]>

* client/network/service: Add primary dimension to connection metrics

Two nodes can be interconnected via one or more connections. The first
of those connections is called the primary connection.

This commit adds another dimension to the
`sub_libp2p_connections_{closed,opened}_total` metrics to differentiate
primary and non-primary connections being opened / closed.

By intuition more than one connection between two nodes is rare.
Tracking the fact whether a connection is primary or not will help prove
or disprove this intuition.

* .maintain/monitoring: Ensure to sum over all connections_closed variants

* client/network/service: Rename is_primary to is_first

* client/network/service: Split by metric name with two additional metrics

* Revert ".maintain/monitoring: Ensure to sum over all connections_closed variants"

This reverts commit 2d2f93e4.

* client/network/service: Remove labels from distinct metrics

* Fix browser informant

* Fix documentation

* Add an informant_output_format function to the cli config

* Wrap informant output format in an option

* Revert batch verifier

* Remove wasm-timer from primitives io cargo lock

* Drop informant_output_format function

* derive debug for output format

* Initial commit

Forked at: d735e4d0


No parent branch.

* Make sc_service::Configuration derive Debug

* Replace task_executor fn's input by proper TaskExecutor type (cleaner)

* impl From<Fn> for TaskExecutor

* Update client/cli/src/runner.rs

* Add some doc, examples and tests

* Replace Deref by fn spawn as suggested

Co-authored-by: Bastian Köcher <[email protected]>

* Fix typo in offchain's docs

* Use Self keyword in AsyncApi::new()

* Move httpclient to be part of OffchainWorkers to optimize block import

* Fix compilation errors for tests

* Add wrapper struct for HyperClient

* Use lazy_static share SharedClient amongst OffchainWorkers. Remove the need to raise the fd limit

* Revert "Use lazy_static share SharedClient amongst OffchainWorkers. Remove the need to raise the fd limit"

This reverts commit 7af97498.

* Add lazy_static for tests

* Avoid panic on dropping a `Receiver`.

* CI

* service: add spawner for essential tasks

* node: spawn block authoring and grandpa voter as blocking tasks

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

We need to check that the public key of an authority exists in our
keystore before we can successfully claim a plain secondary slot.

* Validate encoding of extrinsics passed to runtime

* Bump codec version explicitly

In order to tell whether an address is the local nodes address the
authority discovery module previously compared the Multihash within the
`p2p` Multiaddr protocol.

rust-libp2p recently switched to a new PeerId representation (see [1]).
Multihashes of the same PeerId in the new and the old format don't
equal.

Instead of comparing the Multihashes, this patch ensures the module
compares the PeerIds

[1] https://github.com/libp2p/rust-libp2p/issues/555

Before we only updated the chain info of sync when we have imported
something using the import queue. However, if you import your own
blocks, this is not done using the import queue and so sync is not
updated. If we don't do this, it can lead to sync switching to "major
sync" mode because sync is not informed about new blocks. This
especially happens on Cumulus, where a collator is selected multiple
times to include its block into the relay chain and thus, sync switches
to major sync mode while the node is still building blocks.

* Initial commit

Forked at: d735e4d0
No parent branch.

* Errors if slot_duration is zero

* Errors if slot_duration is zero

* Revert "Errors if slot_duration is zero"

This reverts commit a9e9820e

.

* Update client/consensus/slots/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

This implements tracking of the last seen blocks in informant display
to prevent printing the import message twice. In Cumulus we first import
blocks as part of the block building with `new_best == false` and set
the best block after we know which one was included by the relay chain.
This leads to printing the import messages two times. This pr solves the
problem by track the latest seen blocks to not print the message twice.

* Introduce trait

* Implement VRFSigner in keystore

* Use vrf_sign from keystore

* Convert output to VRFInOut

* Simplify conversion

* vrf_sign secondary slot using keystore

* Fix RPC call to claim_slot

* Use Public instead of Pair

* Check primary threshold in signer

* Fix interface to return error

* Move vrf_sign to BareCryptoStore

* Fix authorship_works test

* Fix BABE logic leaks

* Acquire a read lock once

* Also fix RPC acquiring the read lock once

* Implement a generic way to construct VRF Transcript

* Use make_transcript_data to call sr25519_vrf_sign

* Make sure VRFTranscriptData is serializable

* Cleanup

* Move VRF to it's own module

* Implement & test VRF signing in testing module

* Remove leftover

* Fix feature requirements

* Revert removing vec macro

* Drop keystore pointer to prevent deadlock

* Nitpicks

* Add test to make sure make_transcript works

* Fix mismatch in VRF transcript

* Add a test to verify transcripts match in babe

* Return VRFOutput and VRFProof from keystore