* Add derive_more to sp_core

* Convert Vec to Signature

* Use sign_with in AURA and BABE

* Signing errors

* Update slots to return consensus result

* Fix use

* Clone public key

* Match block_params

* WIP

* Use to_public_crypto_pair

* Pass public key only to block import params

* Address PR review

* Fix consensus RPC

* Fix babe tests

* adjust uses

* Fix line widths

* Fist benchmark barely working

* Debug checkpoint

* add rest of benchmarks

* Add to runtime

* Fix build

* Update frame/elections-phragmen/src/benchmarking.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Update frame/elections-phragmen/src/benchmarking.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* major imp

* Make them run on release

* Help finish phragmen benchmarks (#5886)

* update caller, account, and member/runner-up creation

* remove stuff

* ocd

* make it work with real run

* relax the numbers a bit

* New and improved version

* Make elections-phragmen weighable and secure. (#5949)

* Make elections-phragmen weighable.

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Fix all tests

* Fix everything

* Add note

Co-authored-by: Alexander Popiak <[email protected]>

* Doc update

* Fix some complexity params

* Once more ready to benchmark

* ready for bench

* final tunes

* Update frame/elections-phragmen/src/lib.rs

* Fix fix

* Update frame/elections-phragmen/src/lib.rs

* Update frame/elections-phragmen/src/benchmarking.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update frame/elections-phragmen/src/benchmarking.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update to latest weights

* Some fixes

* Fix dual voter read from @thiolliere

* Remove todos

* review from @shawntabrizi



* Fix bench tests.

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Alexander Popiak <[email protected]>

* Update wasmtime (#5822)

* update wasmtime

* update tests

* Update client/executor/wasmtime/src/host.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* wip

* use master-candidate

* update with patches

* update versions

Co-authored-by: Bastian Köcher <[email protected]>

* Bump version

* Added parallel execution test

Co-authored-by: Nikolay Volf <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

This reverts commit c4285c95.

We found some bugs that we first need to fix.

* session: runtime api for generating session membership proofs

* grandpa: add runtime api for creating equivocation report txs

* grandpa: submit signed equivocation report transactions

* grandpa: use proper equivocation report type

* grandpa: report equivocations

* grandpa: validate equivocation proof

* grandpa: update to finality-grandpa 0.9.1

* grandpa: fix encoding of session membership proof

* grandpa: initialize set id session mapping for genesis session

* grandpa: fix bug in set_id session validation

* fix compilation

* cleanup from merge conflicts

* cleanup crate tomls

* grandpa: refactor equivocation handling to separate trait

* node-template: fix compilation

* fix test compilation

* bump finality-grandpa to v0.10.2

* rpc: fix runtime version test

* CHERRY-PICK #4200: Add documentation to SubmitSignedTransaction and actually make it work

Squashed commit of the following:

commit dc8d71c3
Author: Tomasz Drwięga <[email protected]>
Date:   Tue Dec 3 16:29:33 2019 +0100

    Split the method to avoid confusing type error message.

commit 0c4c0378
Author: Tomasz Drwięga <[email protected]>
Date:   Tue Dec 3 16:19:55 2019 +0100

    Make accounts optional, fix logic.

commit d715f645
Author: Tomasz Drwięga <[email protected]>
Date:   Tue Dec 3 10:06:20 2019 +0100

    Remove warning.

commit 3f38218a
Merge: f85b8903 368318c9
Author: Tomasz Drwięga <[email protected]>
Date:   Tue Dec 3 07:08:05 2019 +0100

    Merge branch 'master' into td-signed-transactions

commit f85b8903
Merge: f8c95403 d8d5da2c
Author: Tomasz Drwięga <[email protected]>
Date:   Mon Dec 2 13:57:25 2019 +0100

    Merge branch 'master' into td-signed-transactions

commit f8c95403
Author: Tomasz Drwięga <[email protected]>
Date:   Mon Nov 25 17:34:52 2019 +0100

    Forgotten import.

commit a645b90d
Author: Tomasz Drwięga <[email protected]>
Date:   Mon Nov 25 17:32:10 2019 +0100

    Fix naming and bounds.

commit bc28c605
Author: Tomasz Drwięga <[email protected]>
Date:   Mon Nov 25 17:01:05 2019 +0100

    Add documentation to signed transactions and actually make them work.

* grandpa: skip block initialization on report submission method

* primitives: allow transaction pool access by default for offchain calls

* grandpa: unused parameters

* grandpa: remove unused method

* grandpa: enable equivocation reporting

* grandpa: add workaround for parameter encoding

* grandpa: fix localized_payload calls in tests

* fix submit_report_equivocation_extrinsic in runtimes

* node: fix submit transaction test compilation

* node: bump spec_version

* rpc: fix api version test

* grandpa: allow custom equivocation offence type

* grandpa: add test for authorities::next_change_height

* grandpa: cleanup report_equivocation function

* node: move reporting app crypto to node-primitives

* grandpa: move equivocation traits to own module

* grandpa: rename app-crypto crate import

* grandpa: export equivocation types

* node: bump spec_version

* grandpa: rename EquivocationReport to EquivocationProof

* grandpa: add missing docs to primitives

* grandpa: add missing docs to equivocation

* node: fix compilation

* grandpa: add missing docs to pallet

* node: bump spec_version

* fix whitespace

* grandpa: return error on offence reporting

* grandpa: expose session and validator count in proofs through traits

* grandpa: use strong key in module KeyOwnerProofSystem

* grandpa: move key ownership proof to grandpa runtime api

* grandpa: remove unnecessary cloning when checking equivocation proof

* grandpa: make report_equivocation a method in Environment

* support: implement KeyOwnerProofSystem for ()

* grandpa: move KeyOwnerProofSystem to module trait

* test-utils: fix runtime compilation

* grandpa: fix test compilation

* grandpa: fix test compilation after merge

* grandpa: simplify transaction submission types

* grandpa: validate equivocation report in signed extension

* client: fix test

* node: use ValidateEquivocationReport signed extension

* grandpa: expose key ownership proof under opaque type

* grandpa: better docs on key ownership proofs

* grandpa: add note about signed extension

* grandpa: add ValidateEquivocationReport::new

* grandpa: remove skip_initialize_block from runtime api

* grandpa: use new offchain transaction submission API

* grandpa: take set_id in generate_key_ownership_proof

* grandpa: update to finality-grandpa v0.12.2

* grandpa: cleanup usages of AuthoritySet::current

* grandpa: fix test

* grandpa: add mocking utilities for equivocation reporting

* grandpa: add test for equivocation reporting

* grandpa: move SetIdSession initialization

* grandpa: add more tests

* node: enable historical session manager

* node: bump spec_version

* node: use strong key types in KeyOwnerProofSystem definitions

* grandpa: export GrandpaEquivocationOffence type

* client/network-gossip/bridge: Use bounded channel

Instead of returning an unbounded channel on
`GossipEngine::messages_for` return a bounded channel. For now the
channel length is determined by the amount of past messages cached in
the `ConsensusGossip`.

With a bounded channel, one can't just fire-and-forget style send into
it, but has to first check whether the channel is ready. Thus this
commit restructures `GossipEngine::poll` and introduces a
`ForwardingState` into `GossipEngine`.

* client/network-gossip/bridge: Add quickcheck for different size channels

* really rough mock runtime

* start to work on offences

* Make sure to start the session.

* Update to latest master.

* Add verify.

* Fix on_initialize benchmark.

* Add grandpa offence.

* Add Babe offence benchmarking.

* Enable babe test.

* Address review grumbles.

* Address review grumbles.

* Address review grumbles part 1/2

* use currency trait

* features

* Check events explicitly.

* Auto-impl tuple converter.

* Removed dead code.

* add test feature flag

* dont use std

Co-authored-by: Shawn Tabrizi <[email protected]>

* Start improving `storage_append`

* Fix some stuff

* Fix compilation

* Update docs and add new test

* More tests

* Test kill + append

* Remove tx factory files.

* Remove unused imports.

* Revert cargo lock.

* grandpa: wire up basic RPC call

* grandpa: make it compile against GRANDPA with expose round state

* grandpa: use shared voter state to expose RPC endpoint

* grandpa: restructure into nested structs

* grandpa: return background rounds too

* grandpa: return error when endpoint not ready

* grandpa: collect grandpa rpc deps

* grandpa: decide to use concrete AuthorityId in finality-grandpa-rpc

* grandpa: remove unncessary type annotation

* grandpa: move error code to const

* grandpa: remove unnecessary WIP comment

* grandpa: remove Id type parameter for SharedVoterState

* grandpa: update tests to add shared_voter_state in parameters

* grandpa: remove old deprecated test

* grandpa: fix getting the correct set_id

* grandpa: make SharedVoterState a struct

* grandpa: wrap shared_voter_state in rpc_setup

* grandpa: replace spaces with tabs

* grandpa: limit RwLock write attempt to 1 sec

* grandpa: add missing doc comments and remove some pub

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <[email protected]>
Co-Authored-By: Hernando Castano <[email protected]>

* grandpa: update function name call after change in finality-grandpa

* grandpa: group pub use and only export voter::report

* grandpa: add missing docs

* grandpa: extract out structs used for json serialization

* grandpa: stick to u32 for fields intended for js

* grandpa: move Error type to its own file

* grandpa: group pub use better

* Apply code review suggestion

Co-authored-by: Bastian Köcher <[email protected]>

* grandpa: use correct version of finality-granpda in rpc crate

* grandpa: add back basic rpc unit test

* grandpa: replace SharedVoterState::new() with empty()

* node: cleanup grandpa::SharedVoterState usage in macro

* grandpa: remove VoterState error variant

* grandpa: enable missing futures compat feature

* grandpa: fix typo in error variant

* grandpa: remove test_utils

* grandpa: allow mocking rpc handler components

* grandpa: rename serialized to report in rpc module

* grandpa: add proper test for RPC

* grandpa: update to finality-grandpa v0.12.1

Co-authored-by: André Silva <[email protected]>
Co-authored-by: Demi Obenour <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Hernando Castano <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* vrf: remove Raw* types

* babe: remove Raw* types

* pallet-babe: switch representation of RawVRFOutput to Randomness

* pallet-babe: calculate inout within the pallet

* Remove make_transcript duplication

* Bump spec version

* Fix frame tests

* and_then -> map

* Always enable u64_backend

* Fix nostd compile

* fix import: should not use std

* Remove unused definition of RawVRFOutput

* Remove unused import of RuntimeDebug

Co-authored-by: Gavin Wood <[email protected]>

* Expose that `BasicQueue` expects blocking spawn

Up to now `BasicQueue` expected a closure that to spawn a `Future`.
This was expected to be a closure that spawns a blocking future.
However, this wasn't documented anywhere. This pr introduces a new trait
`SpawnBlocking` that exposes this requirement to the outside.

* Feedback

* update wasmtime

* update tests

* Update client/executor/wasmtime/src/host.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* wip

* use master-candidate

* update with patches

* update versions

Co-authored-by: Bastian Köcher <[email protected]>

* grandpa: update to v0.12.0

* grandpa: fix tests

* grandpa: better validation of authority set invariants

* grandpa: avoid duplicating invalid authority list check

* grandpa: add missing doc

* grandpa: better validation of expect proofs

* grandpa: fix test compilation

* grandpa: fix tests

* grandpa: add test for AuthoritySet invariants

* grandpa: bump finality-grandpa to v0.12.0

* Frame System Benchmarking

* Add to substrate node, avoid divide by zero errors in analysis

* reduce features

* some fixes

* copy pasta

* parity-db bench

* use arkady suggestion

Co-Authored-By: Pierre Krieger <[email protected]>

* Remove sp-runtime dependency from sp-phragmen

* Use regular derive(Debug) instead of RuntimeDebug

* Drop client from sc-network and sc-client-db, move LongestChain to sc-client-api

* move leaves, cht, in_mem to sc-client-api, drop client from sc-finality-grandpa

* drop sc-service from sc-rpc

* drop sc-service from sc-consensus-aura

* drop sc-client from manual-seal and babe

* drop sc-client from utils/frame/rpc/system and utils/frame/benchmarking-cli

* drop sc-client from bin/node and bin/node-template

* drop sc-client

* fix tests

* remove check -p sc-client from gitlab.yml

* fix warnings

* fixes ui test

* fix light client tests

* adds associated Client type to AbstractService

* adds UsageProvider to Client

* fixed ui test, again

* tried and failed to get node-cli to compile for wasm

* thanks to tomaka for helping me get node-cli to compile for wasmm

* ui test pls pas 🙏🏾



* all tests passing 🪄

* no_run documentation code

* rm -f documentation code

* ClientProvider

* fix mega trait

* move LongestChain to sc-consensus, use adds minimal bounds to AbstractService::Client

* adds license to sc-consensus

Co-authored-by: Benjamin Kampmann <[email protected]>

* Master.into()

* Remove debug stuff

* Better license

* Migrate away from SimpleDispatchInfo

* Fix test

* Revert "Migrate away from SimpleDispatchInfo"

This reverts commit dbdd27fa.

* Move to offchain randomness

* Fix tests

* Fix tests more

* use append api to deposit events

* use optimized append

* one more optimization

* fix bug

* fix issues

* address review

* Optimize pinning

* Ref counting state backend

* Style

Co-Authored-By: Wei Tang <[email protected]>

* Update Cargo.lock

* Handle empty node

Co-authored-by: Wei Tang <[email protected]>

Co-Authored-By: Kian Paimani <[email protected]>
Co-Authored-By: Tomasz Drwięga <[email protected]>

Co-Authored-By: Tomasz Drwięga <[email protected]>
Co-Authored-By: Sergei Pepyakin <[email protected]>

* network: Only insert global addresses into the DHT.

Currently every address reported via libp2p-identify is inserted into
the DHT which thus contains a multitude of unreachable addresses such
as from 127.0.0.0/8 or 10.0.0.0/8.

Issue #5099 suggested a dedicated service over UDP to gauge the
reachability of an address, which would however incur extra I/O costs
and be of limited use.

As an alternative and simpler tactic, this PR only allows global IP
addresses to be inserted into the DHT unless an explicit command-line
flag `--allow-non-global-addresses-in-dht` is given or a node is
started with `--dev`. This opt-in behaviour is meant to allow
site-local networks to still make use of a DHT.

* Enable non-global in more test setups.

* Replace command-line option with different name.

* Another test fix.

* client/network-gossip: Move sink IO outside of state_machine

`ConsensusGossip` is supposed to be a deterministic state machine.
`GossipEngine` wrapping `ConsensusGossip` should handle IO operations.

This commit moves the `message_sink` IO operations to `GossipEngine`.
More specifically on incoming messages a `GossipEngine` calls
`ConsensusGossip::on_incoming` to validate and register the messages.
`ConsensusGossip` returns the valid messages which are then forwarded by
`GossipEngine` to the upper layer via the `message_sinks`.

* client/network-gossip: Adjust and extend tests

* Update client/network-gossip/src/bridge.rs

Co-authored-by: Benjamin Kampmann <[email protected]>

Co-Authored-By: thiolliere <[email protected]>

Co-Authored-By: Pierre Krieger <[email protected]>