It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).

This touches a few important things:
 - the startup of the async task with tokei:
    This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
 - increased the version of structopt and tokei
 - no more use of structopt internal's API
 - less use of generics

Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want

Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.

Patch d4fbb897 reintegrated the periodic neighbor packet worker, by
moving its logic into a `Future` implementation on `NetworkBridge` which
needs to be polled by its upper layer.

Polling by the upper layer was implemented within the `Future`
implementation of the `VoterWork` struct but not within the `Future`
implementation of the `ObserverWork` struct. This patch adds polling of
the `NetworkBridge` to the latter.

* Add `get_global` for `Sandbox`

This pr adds `get_global` to retrieve a `global` variable from an
instantiated sandbox wasm blob.

* Bump `spec_version`

* Update primitives/wasm-interface/src/lib.rs

Co-Authored-By: Sergei Pepyakin <[email protected]>

* `get_global` -> `get_global_val`

Co-authored-by: Sergei Pepyakin <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

* adding license fields to all crates

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

A light client can not be a validator, so we should not set `authority
== true` in dev mode.

* Rpc api for offchain storage

* Replace Vec<u8> to Bytes, replace Mutex to RwLock

* Remove pub

* Modify copyright year

* Display error message when checking has_api.

* Cargo.lock.

* Make sure to destroy state in case of error.

* Update client/src/call_executor.rs

* Update client/src/call_executor.rs

Co-authored-by: Bastian Köcher <[email protected]>

* WASM runtime switch to import memory

Up to now runtimes have exported their memory. To unify it with
sandboxing, this pr switches runtimes to import memory as well.

From a functional perspective, exporting/importing memory makes no
difference to the runtime.

To provide backwards compatibility, WASM exported memory is still supported.

* Revert debug stuff

* Revert some stuff

* Fixes for libp2p 0.15

* More work

* Update libp2p

* Update to libp2p 0.15

* transaction pool changes

* service & network changes

* address review

* reduce future pool

* shuffle tests

* update tests

* inc_nonce -> increment_nonce

* instant/manual seal

unbounded queues are evil

Apply suggestions from code review

Co-Authored-By: Robert Habermeier <[email protected]>

add fork tests, docs, remove todos

moar docs

Update client/consensus/manual-seal/src/rpc.rs

Co-Authored-By: Robert Habermeier <[email protected]>

remove unbound generic, parameter, docs, deps, code style changes

Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <[email protected]>

code style chnges

remove unused deps, remove dep renames, check if block is empty before importing, use ? for error propagation

fix tests

log errors for instant seal

use debug

code style changes, updated copyright dates

use txpool::Pool instead of BasicPool, code style changes

fixed tests

* fix tests

* requested changes from review

* check inherents len

* rebase

* Quick and dirty impl.

* Clean up

* Check the signatures.

* Fix tests.

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>

* Ignore non function members.

Co-authored-by: Bastian Köcher <[email protected]>

* Introduces `author_hasKey` and `author_hasSessionKeys` rpc endpoints

Both endpoints can be used to check if a key is present in the keystore.

- `hasKey` works on with an individual public key and key type. It
checks if a private key for the given combination exists in the
keystore.
- `hasSessionKeys` works with the full encoded session key blob stored
on-chain in `nextKeys`. This requires that the given blob can be decoded
by the runtime. It will return `true`, iff all public keys of the
session key exist in the storage.

Fixes: https://github.com/paritytech/substrate/issues/4696



* Update client/rpc-api/src/author/error.rs

Co-Authored-By: Nikolay Volf <[email protected]>

* Indentation

Co-authored-by: Nikolay Volf <[email protected]>

* async/await in tx graph

* review notes

* remove unused typedef

* Switch GrandPa to new futures

* Work on making tests work

* until_imported tests working again

* Work on switching tests to stable futures

* Modifications

* Re-add test as #[ignore]

* Don't ignore

* Add manual unpins

* Remove Header import

* Return concrete Sink type

* Switch to crates.io finality-grandpa version

* Remove use statement that slipped in

* Fix some nitpicks

* Remove unpin from i

* Fixed typo

* Move futures01 to dev-deps

* Fix nitpicks

* Update client/finality-grandpa/src/communication/mod.rs

Co-Authored-By: André Silva <[email protected]>

* nitpicking

Co-authored-by: Pierre Krieger <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Reduction.

* Reformation.

* add locked timer stuff

* fix issues and introduce full pool

* arrange together

* fix benches

* fix new_light

* Add revalidation test case

* review fixes

* review fixes

* use just ready future

* address review

* Refactor and test spec block rules

* address review

* Update client/src/block_rules.rs

Co-Authored-By: André Silva <[email protected]>

Co-authored-by: André Silva <[email protected]>

Max Inden authored Jan 23, 2020 and asynchronous rob committed Jan 23, 2020

* clinet/finality-grandpa: Make round_communication use bounded channel

`round_communication` returns a `Sink` and a `Stream` for outgoing and
incoming messages. The messages send into the `Sink` are forwarded down
to the network as well as send back into the `Stream` to ensure the node
processes its own messages.

So far, to send messages into the `Sink` back into the `Stream`, an
unbounded channel was used. This patch updates `round_communication` and
`OutgoingMessages` to use a bounded channel.

This is part of a greater effort to reduce the number of owners of
components within `finality-grandpa` and `network` as well as to reduce
the amount of unbounded channels. For details see d4fbb897 and
f0c18520.

* client/finality-grandpa: Import futures03::future::ready at the top

* client/finality-grandpa: Make tests use compat of future 03

* client/finality-grandpa: Do not import ready into scope

Instead of importing futures03::future::ready into the scope, only
import futures::future03 into scope and call ready as furure03::ready.

* consensus, pow: intermediate separation and fail

* Fix compiles

* Update primitives/consensus/common/src/block_import.rs

Co-Authored-By: Robert Habermeier <[email protected]>

* Update primitives/consensus/common/src/block_import.rs

Co-Authored-By: Robert Habermeier <[email protected]>

* Document what None means for `fork_choice` in block import params

Co-authored-by: Robert Habermeier <[email protected]>

Bastian Köcher authored Jan 22, 2020 and Sergey Pepyakin committed Jan 22, 2020

This moves the wasm-allocator (`FreeingBumpHeapAllocator`) into its own
crate `sp-allocator`. This new crate can theoretically provide multiple
different allocators. Besides moving the allocator, this pr also makes
`FreeingBumpHeapAllocator` compile on `no_std`.

* client/authority-discovery/Cargo.toml: Update dependency

* client/authority-discovery: Pass packet payload and addresses as slice

Starting with Bytes 0.5 `Vec<T>` does not implement `Buf`, but `&[T]`
does.

* Pass an executor through the Configuration

* Make tasks_executor mandatory

* Fix tests

* Make debug builds more usable

This pr makes debug builds more usable in terms of `cargo run -- --dev`.

1. `--dev` activates `--execution native`, iff `--execution` is not
given or no sub `--execution-*` is given.
2. It was probably a mistake to compile WASM in debug for a debug build.
So, we now build the WASM binary always as `release` (if not requested
differently by the user). So, we trade compilation time for a better
debug experience.

* Make sure we only overwrite default values

* Make it work

* Apply suggestion

* deprecate chain_status field of network handshake

* Update client/network/src/protocol/message.rs

remove unneeded whitespace.

Co-Authored-By: Pierre Krieger <[email protected]>

Co-authored-by: Pierre Krieger <[email protected]>

* client/finality-grandpa: Reintegrate gossip validator report stream

The `finality-grandpa` `GossipValidator` is called by the `GossipEngine`
in a synchronous fashion on each gossip message. Its main task is to
decide whether to gossip the given message on, or whether to drop it.

In addition it also updates the reputation of a node's peers based on
the incoming gossip messages. To do so it needs to be able to report the
reputation change which it does through an unbounded channel (in order
to stay synchronous).

Previously the receiving side of this channel would be handled by a new
task, polling the channel and forwarding the changes to a clone of the
`GossipEngine` that it would own.

Instead the receiver of the above mentioned channel is now being polled
by the `NetworkBridge` within its `Future::poll` implementation.
Reputation changes are reported through the already existing
`GossipEngine` instance within `NetworkBridge`.

For details on the overall goal, see d4fbb897.

* client/finality-grandpa: Remove exit future from test NetworkBridges