* primitives/runtime: initial changes on supporting multiple Justifications

* primitives/runtime: make Justifications strongly typed

* Encode/decode Justifications

* primitives/runtime: add Justification type

* backend: apply_finality and finalize_block takes a single Justification

* manual-seal: create engine id and let rpc take encoded justification

* backend: skeleton functions for appending justifications

* backend: initial implementation append_justification

Initial implementation of append_justification on the Backend trait, and also remove unused skeleton
functions for append_justificaton on Finaziler trait.
k

* backend: guard against duplicate consensus engine id

* client/db: add check for block finality

* client/api: add append_justification to in_mem db

* client/light: add no-op append_justification

* network: fix decode call for Justification

* network: only send a single Justification in BlockData

* network: minor comment update

* protocol: update field names to distinguish single justification

* client: further field renames to plural

* client: update function names to plural justifications

* client/db: upgrade existing database for new format

* network: remove dependency on grandpa crate

* db: fix check for finalized block

* grandpa: check for multiple grandpa justifications hwne importing

* backend: update Finalizer trait to take multiple Justifications

* db: remove debugging statements in migration code

* manual-seal: update note about engine id

* db: fix check for finalized block

* client: update variable name to reflect it is now plural

* grandpa: fix incorrect empty Justications in test

* primitives: make Justifications opaque to avoid being empty

* network: fix detecting empty Justification

* runtime: doc strings for Justifications functions

* runtime: add into_justifications

* primitives: check for duplicates in when adding to Justifications

* network/test: use real grandpa engine id in test

* client: fix reviewer comments

* primitives: rename Justifications::push to append

* backend: revert changes to Finalizer trait

* backend: revert mark_finalized

* backend: revert changes to finalize_block

* backend: revert finalized_blocks

* db: add a quick early return for performance

* client: minor reviewer comments

* service/test: use local ConsensusEngineId

* network: add link to issue for sending multiple Justifications

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <[email protected]>

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <[email protected]>

* network: tweaks to review suggestions

* network: revert change to BlockData for backwards compatibility

* Apply suggestion from code review

Co-authored-by: Pierre Krieger <[email protected]>

* Apply suggestions from code review

Co-authored-by: André Silva <[email protected]>

* primitives: update doc comment for Justifications

* client/db/upgrade: avoid grandpa crate dependency

* consensus: revert to single Justification for import_justification

* primitives: improve justifications docs

* style cleanups

* use and_then

* client: rename JUSTIFICATIONS db column

* network: revert to using FRNK in network-test

Co-authored-by: Pierre Krieger <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: André Silva <[email protected]>

polkadot companion: paritytech/polkadot#2535

* Ensure we spawn the block import worker as an essential task

This pr ensures that we spawn the block import worker as an essential
task. This is quite important as we need to bring down the node when the
block import is done. Besides that it adds some debug output to the
block import worker.

* Don't be stupid :D

* transaction-pool: drop unpropagable txs if local node cant author blocks

* fix test compilation

* transaction-pool: remove unnecessary static bound on CanAuthor

Co-authored-by: Tomasz Drwięga <[email protected]>

* rpc-api: add translation for PoolError::Unactionable

* transaction-pool: add test for rejecting unactionable transactions

* basic-authorship: fix doc test

* transaction-pool: fix benchmark compilation

* transaction-pool: rename CanAuthor to IsValidator

* transaction-pool: nit in error message

Co-authored-by: Tomasz Drwięga <[email protected]>

* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL

This prevents nodes from executing the same block 2 times.

* grandpa: remove light-client specific block import

* consensus, network: remove finality proofs

* Initial commit

Forked at: 601e2fa1
Parent branch: origin/master

* WIP

Forked at: 601e2fa1
Parent branch: origin/master

* WIP

Forked at: 601e2fa1
Parent branch: origin/master

* WIP

Forked at: 601e2fa1
Parent branch: origin/master

* WIP

Forked at: 601e2fa1
Parent branch: origin/master

* WIP

Forked at: 601e2fa1
Parent branch: origin/master

* CLEANUP

Forked at: 601e2fa1
Parent branch: origin/master

* Add notes to original source code

* CLEANUP

Forked at: 601e2fa1
Parent branch: origin/master

* CLEANUP

Forked at: 601e2fa1
Parent branch: origin/master

* WIP

Forked at: 601e2fa1
Parent branch: origin/master

* WIP

Forked at: 601e2fa1
Parent branch: origin/master

* WIP

Forked at: 601e2fa1
Parent branch: origin/master

* CLEANUP

Forked at: 601e2fa1
Parent branch: origin/master

* WIP

Forked at: 601e2fa1
Parent branch: origin/master

* Some doc

* Test with trybuild

* Revert "Test with trybuild" (issue with trybuild atm)

This reverts commit 9055ec2206808ba3ddce6e3d87eb358907fa5e42.

https://github.com/dtolnay/trybuild/issues/53

* Apply suggestions

* Rename derive to proc-macro

* Remove "prefix" feature from informant

* Blocking task should use SpawnHandle::spawn_blocking

* Improve doc as suggested

* Fixes

Forked at: 601e2fa1


Parent branch: origin/master

* Apply suggestion

* Update client/cli/proc-macro/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* More suggestions

* CLEANUP

Forked at: 601e2fa1
Parent branch: origin/master

* Improve error message

* CLEANUP

Forked at: 601e2fa1
Parent branch: origin/master

* Fix async issue

* CLEANUP

Forked at: 601e2fa1
Parent branch: origin/master

* CLEANUP

Forked at: 601e2fa1


Parent branch: origin/master

* Add test

* fix doc test

* Update client/cli/src/logging.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Apply suggestions

* Suggestions

* Clarify doc

* WIP

Forked at: 601e2fa1


Parent branch: origin/master

Co-authored-by: Bastian Köcher <[email protected]>

* manual seal is now consensus agnostic

* pr grumbles

* Revalidate transactions only on latest best block

We should revalidate transactions only on the latest best block and not
on any arbitrary block. The revalidation before failed when there were
multiple blocks on the height given to the revalidation function, but no
block was imported as best block.

* Update test-utils/runtime/transaction-pool/src/lib.rs

Co-authored-by: Jaco Greeff <[email protected]>

* Fix tests

* Only process best blocks in the transaction pool

Co-authored-by: Jaco Greeff <[email protected]>

* Remove any implementation of `Spawn` or `Executor` from our task executors

* Fix compilation

* Rename `SpawnBlockingExecutor`

* Update primitives/core/src/traits.rs

Co-authored-by: Kian Paimani <[email protected]>

* Fix tests

Co-authored-by: Kian Paimani <[email protected]>

* Simplify a few chain components creation APIs related to the service

* Fix basic-authorship doc tests

* Remove DefaultQueue

* Update client/service/src/builder.rs

Co-authored-by: André Silva <[email protected]>

* Move ExecutionExtensions comment around

* Remove unused BlakeTwo256

Co-authored-by: André Silva <[email protected]>

* Seperate out the complexity in ServiceBuilder::build_common into seperate functions

* Fix line widths

* Move some functions to their respective crates

The `tree_route` generated by the import notification is only from the
old best block to the new best parent. This means, it does not contain
the new best block in `enacted()`. We need to prune the transactions of
the new best block "manually" to fix this bug.

Besides that, this pr also changed the `id` parameter of the `NewBlock`
chain event to `hash`. The hash of a block is unique in contrast to the
block number. (Block id can either be number or hash)

* Make tx pool aware of retracted fork blocks

* Make it compile

* Update client/transaction-pool/src/lib.rs

Co-authored-by: Nikolay Volf <[email protected]>

* Fix doc test

* Simplify the implementation

* Send tree route as arc to prevent heavy clones

* Switch to use `ExtrinsicHash` to make it more clear

* Fix benchmark

Co-authored-by: Nikolay Volf <[email protected]>

* Add basic authorship metrics

* fixes

* no arc

* move to crate

* Update client/proposer-metrics/Cargo.toml

Co-authored-by: Max Inden <[email protected]>

* remove prefix

* use HistogramTimer

* Update client/proposer-metrics/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/proposer-metrics/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/proposer-metrics/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Max Inden <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* WIP

* WIP

* Expose that `BasicQueue` expects blocking spawn

Up to now `BasicQueue` expected a closure that to spawn a `Future`.
This was expected to be a closure that spawns a blocking future.
However, this wasn't documented anywhere. This pr introduces a new trait
`SpawnBlocking` that exposes this requirement to the outside.

* Feedback

* replace replace with take

* Update client/api/src/cht.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/api/src/cht.rs

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

Co-Authored-By: Pierre Krieger <[email protected]>

* make new contructor

* add metrics to txpool

* fix review

* fix doc comment

* change to counters

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <[email protected]>

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <[email protected]>

* Update client/transaction-pool/src/metrics.rs

Co-Authored-By: Max Inden <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Max Inden <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Max Inden <[email protected]>

* use dedicated wrapper

Co-authored-by: Max Inden <[email protected]>

* manual-seal uses Finalizer trait for finalization instead of Backend

* fix tests

* use Transaction type

* refactor import_queue

* ugh

* line-width

* Update client/consensus/manual-seal/src/lib.rs

Co-Authored-By: Joshy Orndorff <[email protected]>

* fix tests

* update docs

* Update client/consensus/manual-seal/src/seal_new_block.rs

Co-Authored-By: André Silva <[email protected]>

* Don't auto-finalize on verification

* Explicity don't finalize on import.

Co-authored-by: Joshy Orndorff <[email protected]>
Co-authored-by: Joshy Orndorff <[email protected]>
Co-authored-by: André Silva <[email protected]>

* WiP

* Support source in the runtime API.

* Finish implementation in txpool.

* Fix warning.

* Fix tests.

* Apply suggestions from code review

Co-Authored-By: Kian Paimani <[email protected]>
Co-Authored-By: Nikolay Volf <[email protected]>

* Extra changes.

* Fix test and benches.

* fix test

* Fix test & benches again.

* Fix tests.

* Update bumpalo

* Fix doc test.

* Fix doctest.

* Fix doctest.

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Nikolay Volf <[email protected]>

* make sure return ready iterator once state is updated

* update sc_basic_authorship tests

* update node tests

* fix manual seal

* actually fix service test

* add tests

* Update client/basic-authorship/src/basic_authorship.rs

Co-Authored-By: Tomasz Drwięga <[email protected]>

* helper function

* review suggestions

* warning and continue

* add debug log

* use futures::chennel::oneshot

* use declaration bound

* no option for updated_at

* no allocation

* ready_at / ready

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* removes use of sc-client from sc-basic-authorship

* refactor use of ProposerFactory

* correct dep path

* Revalidation queeue.

* add docs and license

* move test

* refactor worker to async/await

* address review

* fix warnings

* update Cargo.lock

* move background task to service

* use tomusdrw loop

* naming

* return From::from

* add doc comment

* add more doc comments

* fix merge bug

* add doc comment for test function

* Update client/transaction-pool/src/testing/pool.rs

Co-Authored-By: Tomasz Drwięga <[email protected]>

* more review fixes

* refactor to allow service keep background tasks from isntantiated subsystems

* use const delay

* fix fallout

* remove fallout

* remove already moved test

* fix doc test

* add valid_at helper

Co-authored-by: Tomasz Drwięga <[email protected]>

* Refactor BlockImportParams to be non_exhaustive

* Fix cargo check compile

* adds finalization support to sc-transaction-pool using MaintainedTransactionPool for finalization events

* adds TransactionStatus::Retracted, notify watchers of retracted blocks, finalized now finalizes, transactions for current finalized -> last finalized block

* adds last_finalized to ChainApi, use generic BlockT for ChainEvent

* fix tests

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <[email protected]>

* tests

* fix tests, docs, lazily dedupe pruned hashes

* fix tests, Cargo.lock

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <[email protected]>

* remove tree_route, last_finalized from ChainApi, add block hash to Finalization and Retracted events

* prune finality watchers

* fix tests

* remove HeaderBackend bound from FullChainApi

* code style nits, terminate stream in finality_timeout

Co-authored-by: Tomasz Drwięga <[email protected]>

* Cleanup transaction pool deps

* Fix it properly

* Fix doc test

* shuffle tests

* update tests

* inc_nonce -> increment_nonce

* instant/manual seal

unbounded queues are evil

Apply suggestions from code review

Co-Authored-By: Robert Habermeier <[email protected]>

add fork tests, docs, remove todos

moar docs

Update client/consensus/manual-seal/src/rpc.rs

Co-Authored-By: Robert Habermeier <[email protected]>

remove unbound generic, parameter, docs, deps, code style changes

Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <[email protected]>

code style chnges

remove unused deps, remove dep renames, check if block is empty before importing, use ? for error propagation

fix tests

log errors for instant seal

use debug

code style changes, updated copyright dates

use txpool::Pool instead of BasicPool, code style changes

fixed tests

* fix tests

* requested changes from review

* check inherents len

* rebase