* instant/manual seal

unbounded queues are evil

Apply suggestions from code review

Co-Authored-By: Robert Habermeier <[email protected]>

add fork tests, docs, remove todos

moar docs

Update client/consensus/manual-seal/src/rpc.rs

Co-Authored-By: Robert Habermeier <[email protected]>

remove unbound generic, parameter, docs, deps, code style changes

Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <[email protected]>

code style chnges

remove unused deps, remove dep renames, check if block is empty before importing, use ? for error propagation

fix tests

log errors for instant seal

use debug

code style changes, updated copyright dates

use txpool::Pool instead of BasicPool, code style changes

fixed tests

* fix tests

* requested changes from review

* check inherents len

* rebase

* Quick and dirty impl.

* Clean up

* Check the signatures.

* Fix tests.

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>

* Ignore non function members.

Co-authored-by: Bastian Köcher <[email protected]>

* Introduces `author_hasKey` and `author_hasSessionKeys` rpc endpoints

Both endpoints can be used to check if a key is present in the keystore.

- `hasKey` works on with an individual public key and key type. It
checks if a private key for the given combination exists in the
keystore.
- `hasSessionKeys` works with the full encoded session key blob stored
on-chain in `nextKeys`. This requires that the given blob can be decoded
by the runtime. It will return `true`, iff all public keys of the
session key exist in the storage.

Fixes: https://github.com/paritytech/substrate/issues/4696

* Update client/rpc-api/src/author/error.rs

Co-Authored-By: Nikolay Volf <[email protected]>

* Indentation

Co-authored-by: Nikolay Volf <[email protected]>

* async/await in tx graph

* review notes

* remove unused typedef

* Switch GrandPa to new futures

* Work on making tests work

* until_imported tests working again

* Work on switching tests to stable futures

* Modifications

* Re-add test as #[ignore]

* Don't ignore

* Add manual unpins

* Remove Header import

* Return concrete Sink type

* Switch to crates.io finality-grandpa version

* Remove use statement that slipped in

* Fix some nitpicks

* Remove unpin from i

* Fixed typo

* Move futures01 to dev-deps

* Fix nitpicks

* Update client/finality-grandpa/src/communication/mod.rs

Co-Authored-By: André Silva <[email protected]>

* nitpicking

Co-authored-by: Pierre Krieger <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Reduction.

* Reformation.

* add locked timer stuff

* fix issues and introduce full pool

* arrange together

* fix benches

* fix new_light

* Add revalidation test case

* review fixes

* review fixes

* use just ready future

* address review

* Refactor and test spec block rules

* address review

* Update client/src/block_rules.rs

Co-Authored-By: André Silva <[email protected]>

Co-authored-by: André Silva <[email protected]>

Max Inden authored Jan 23, 2020 and asynchronous rob committed Jan 23, 2020

* clinet/finality-grandpa: Make round_communication use bounded channel

`round_communication` returns a `Sink` and a `Stream` for outgoing and
incoming messages. The messages send into the `Sink` are forwarded down
to the network as well as send back into the `Stream` to ensure the node
processes its own messages.

So far, to send messages into the `Sink` back into the `Stream`, an
unbounded channel was used. This patch updates `round_communication` and
`OutgoingMessages` to use a bounded channel.

This is part of a greater effort to reduce the number of owners of
components within `finality-grandpa` and `network` as well as to reduce
the amount of unbounded channels. For details see d9837d7d and
5f80929d.

* client/finality-grandpa: Import futures03::future::ready at the top

* client/finality-grandpa: Make tests use compat of future 03

* client/finality-grandpa: Do not import ready into scope

Instead of importing futures03::future::ready into the scope, only
import futures::future03 into scope and call ready as furure03::ready.

* consensus, pow: intermediate separation and fail

* Fix compiles

* Update primitives/consensus/common/src/block_import.rs

Co-Authored-By: Robert Habermeier <[email protected]>

* Update primitives/consensus/common/src/block_import.rs

Co-Authored-By: Robert Habermeier <[email protected]>

* Document what None means for `fork_choice` in block import params

Co-authored-by: Robert Habermeier <[email protected]>

Bastian Köcher authored Jan 22, 2020 and Sergey Pepyakin committed Jan 22, 2020

This moves the wasm-allocator (`FreeingBumpHeapAllocator`) into its own
crate `sp-allocator`. This new crate can theoretically provide multiple
different allocators. Besides moving the allocator, this pr also makes
`FreeingBumpHeapAllocator` compile on `no_std`.

* client/authority-discovery/Cargo.toml: Update dependency

* client/authority-discovery: Pass packet payload and addresses as slice

Starting with Bytes 0.5 `Vec<T>` does not implement `Buf`, but `&[T]`
does.

* Pass an executor through the Configuration

* Make tasks_executor mandatory

* Fix tests

* Make debug builds more usable

This pr makes debug builds more usable in terms of `cargo run -- --dev`.

1. `--dev` activates `--execution native`, iff `--execution` is not
given or no sub `--execution-*` is given.
2. It was probably a mistake to compile WASM in debug for a debug build.
So, we now build the WASM binary always as `release` (if not requested
differently by the user). So, we trade compilation time for a better
debug experience.

* Make sure we only overwrite default values

* Make it work

* Apply suggestion

* deprecate chain_status field of network handshake

* Update client/network/src/protocol/message.rs

remove unneeded whitespace.

Co-Authored-By: Pierre Krieger <[email protected]>

Co-authored-by: Pierre Krieger <[email protected]>

* client/finality-grandpa: Reintegrate gossip validator report stream

The `finality-grandpa` `GossipValidator` is called by the `GossipEngine`
in a synchronous fashion on each gossip message. Its main task is to
decide whether to gossip the given message on, or whether to drop it.

In addition it also updates the reputation of a node's peers based on
the incoming gossip messages. To do so it needs to be able to report the
reputation change which it does through an unbounded channel (in order
to stay synchronous).

Previously the receiving side of this channel would be handled by a new
task, polling the channel and forwarding the changes to a clone of the
`GossipEngine` that it would own.

Instead the receiver of the above mentioned channel is now being polled
by the `NetworkBridge` within its `Future::poll` implementation.
Reputation changes are reported through the already existing
`GossipEngine` instance within `NetworkBridge`.

For details on the overall goal, see d9837d7d.

* client/finality-grandpa: Remove exit future from test NetworkBridges

The `NeighborPacketWorker` within `client/finality-grandpa` does two
things:

1. It receives neighbor packets from components within
`client/finality-grandpa`, sends them down to the `GossipEngine` in
order for neighboring nodes to receive.

2. It periodically sends out the most recent neighbor packet to the
`GossipEngine`.

In order to send out packets it had a clone to a `GossipEgine` within
an atomic reference counter and a mutex. The `NeighborPacketWorker` was
then spawned onto its own asynchronous task.

Instead of running in its own task, this patch reintegrates the
`NeighborPacketWorker` into the main `client/finality-grandpa` task not
requiring the `NeighborPacketWorker` to own a clone of the
`GossipEngine`.

The greater picture

This is a tiny change within a greater refactoring. The overall goal is
to **simplify** how finality-grandpa interacts with the network and to
**reduce** the amount of **unbounded channels** within the logic.

Why no unbounded channels: Bounding channels is needed for backpressure
and proper scheduling. With unbounded channels there is no way of
telling the producer side to slow down for the consumer side to catch
up.  Rephrased, there is no way for the scheduler to know when to favour
the consumer task over the producer task on a crowded channel and the
other way round for an empty channel.

Reducing the amount of shared ownership simplifies the logic and enables
one to use async-await syntax-suggar, given that one does not need to
hold a lock across poll invocations. Using async-await enables one to
use bounded channels without complex logic.

* Add typedefs for storage types

* Fix after merge

* DigestItem::ChangesTrieSignal

* introduce changes_trie::State

* introduce config activation block

* ChangesTrieSignal::as_new_configuration

* moved well_known_cache_keys to client

* extracted DbChangesTrieStorage to separate file

* change meaning of none in blockchain cache

* changes trie config (FULL) cache draft

* eliminating const ChangesTrieConfiguration

* delay pruning

* continue elimination

* do not prune CT config from cache

* removed redundant code

* fix some TODOs

* introduce ConfigurationRange

* use Configuration range in build

* build skewed digest

* remove debug print

* extracted surface iterator

* key_changes works with skewed digests

* fix client build

* add test for NeverPrune

* fix TODO

* fixed some TODOs

* more tests

* fixing TODOs

* fixed compilation

* update runtime version

* git rid of large tuple

* too long lines

* config_activation_block -> zero

* obsolete TODO

* removed unjustified expect

* update TODOs with issue number

* new CT pruning algorithm

fixed cache + multiple blocks finalization

track CT configuraiton on light clients

support CT configuration change revert

revert CT config test

new CT pruning algorithm

fixed cache + multiple blocks finalization

track CT configuraiton on light clients

support CT configuration change revert

revert CT config test

* BlockIdOrHeader isn't really required

* removed debug leftovers + some docs

* more docs

* more post-merge fixes

* more post-merge fixes

* revertes some unnecessary changes

* reverted unnecessary changes

* fix compilation + unnecessary changes

* (restart CI)

* fix cache update when finalizing multiple blocks

* fixed tests

* collect_extrinsics -> set_collect_extrinsics

* restore lost test

* do not calculate block number twice

* Update primitives/blockchain/src/error.rs

Co-Authored-By: cheme <[email protected]>

* map_err -> unwrap_or

* document get_at Result

* delete abandoned file

* added weight for set_changes_trie_config

* prefer_configs -> fail_if_disabled

* Update client/api/src/backend.rs

Co-Authored-By: cheme <[email protected]>

* Update client/db/src/changes_tries_storage.rs

Co-Authored-By: cheme <[email protected]>

* CommitOperation+merge -> CommitOperations

* fixed test compilation

* merged two different CTRange structs

* lost file

* uggrade db from v0 to v1 (init CT cache + add column)

* fix after merge

Co-authored-by: cheme <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

André Silva authored Jan 16, 2020 and asynchronous rob committed Jan 16, 2020

* grandpa: generic voting rule for backing off from best block

* grandpa: fix tests

* Implement runtime version checks in `set_code`

Check that the new runtime code given to `set_code` fullfills some
requirements:

- `spec_name` matches
- `spec_version` does not decreases
- `impl_version` does not decreases
- Either `spec_version` and `impl_version` increase

* Make tests almost work

* Some fixes after master merge

* Fix tests

* Add missed file

* Make depedency check happy?

* Remove leftover `sc-executor`

* AHHHHH

* Reset debug stuff

* Remove some 'static

* More 'static

* Some docs

* Update `Cargo.lock`

* Rename basic-authority to basic-authorship

* Revert weird tab.

* Fix rename.

* Expose a method that allows converting RunCmd to Configuration

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Make Proposer instantiation potentially async.

* fix node-service test

* fix basic-authority doc-test

* only block once on futures in test

* use async/await

* reset a couple of versions down from v0.2.0

* Unstablise browser-utils (very new) and grafana-data-source (going to be replaced with prometheus sometime)

* unmark a bunch more client crates for stabilization

* fix versions in Cargo.tomls

* Downgrade network-test

* Set frame-metadata version to `10.0.0`

* Hide and documented storage generator as internal api

* Downgrade `sp-externalities`

* Downgrade `sc-cli`

* Downgrade sc-executor et al to 0.8.0

* Downgrade sp-state-machine to 0.8.0

* Downgrade pallet-contracts et al to 0.8.0

* Downgrade sp-sandbox to 0.8.0

* downgrade pallet-evm to 0.8.0

* Downgrade pallet-staking to 0.8.0

2.0 should be implementation of lazy payout alongside all the fixes of
current module

* Downgrade node-transaction-factory to 0.8.0

* frame pallets are 2.0.0

Co-authored-by: Ashley <[email protected]>
Co-authored-by: Pierre Krieger <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Sergei Pepyakin <[email protected]>
Co-authored-by: Wei Tang <[email protected]>
Co-authored-by: thiolliere <[email protected]>
Co-authored-by: Marcio Diaz <[email protected]>

* network: add logging for extra requests

* fixed tests compilation

Co-authored-by: Svyatoslav Nikolsky <[email protected]>

* Add warning if offchain workers version is not supported.

* Support only v2.

* Make it a warning.

* Switch service to futures03

* Fix tests

* Fix service test and cli

* Re-add Executor trait to SpawnTaskHandle

* Fix node-service

* Update babe

* Fix browser node

* Update aura

* Revert back to tokio-executor to fix runtime panic

* Add todo item

* Fix service tests again

* Timeout test futures

* Fix tests

* nits

* Fix service test

* Remove zstd patch

* Re-add futures01 to aura and babe tests as a dev-dep

* Change failing test to tee

* Fix node

* Upgrade tokio

* fix society

* Start switching grandpa to stable futures

* Revert "Start switching grandpa to stable futures"

This reverts commit 9c1976346237637effc07c13f7d0403daf5e71cf.

* Fix utils

* Revert substrate service test

* Revert gitlab

Co-authored-by: thiolliere <[email protected]>