* A patch

* Fix

* Better approach.

* Fix grumbles

* client/finality-grandpa: Instrument until-imported queue

The `UntilImported` queue takes as input finality grandpa messages that
depend on blocks that are not yet imported and holds them back until
those blocks are imported.

This commit adds a basic metric, the amount of messages waiting in the
queue, to the module. For now this metric is only available for the
global `UntilImported` queue awaiting blocks for commit and catch-up
messages.

* client/finality-grandpa/src/until_imported: Update metric help text

Co-Authored-By: Ashley <[email protected]>

Co-authored-by: Ashley <[email protected]>

* Increase limit on light client response size

* Address review

* Add deposit/withdraw events

* Remove balances trait

* Don't populate runtime events in genesis

* typo

* Change to block zero

* Fix vesting tests

* Update frame/system/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update frame/system/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Add test

* Fix test

* Fix contract tests

* Fix phragmen tests

* Fix Generic Assets Tests

* Fix offences tests

* Fix im-online

* fix recovery

* Fix utility tests

* Shorter

* Use ext

Co-authored-by: Bastian Köcher <[email protected]>

* Batch benchmarks together with `*` notation.

* Fix short structopt conflict

* Return error if `batches` is empty

* Move fast benchmarks macro into `frame_benchmarking` (#5445)

* Move macro into `frame_benchmarking`

* Update docs

* Extra line

* Return error if `batches` is empty

Co-authored-by: Shawn Tabrizi <[email protected]>

* Implements mocking of runtime apis

This pr adds support for easily mock runtime api implementations for
tests by using the `mock_impl_runtime_apis!` macro. The syntax is
similar to `impl_runtime_apis!`. The mocked implementation automatically
implements `ApiExt`, `ApiErrorExt` and `Core` as these are required by
the runtime api traits, but not required in tests or only a subset of them.

* Fix warnings

* Update primitives/api/proc-macro/src/utils.rs

Co-Authored-By: Nikolay Volf <[email protected]>

* Review feedback

Co-authored-by: Nikolay Volf <[email protected]>

* imporoved intervallier version

* remove extra event

* use version 0.3.1

* fix warning

* Added option to disable default block announce

* Added on_block_imported on NetworkService

* Revert "Added on_block_imported on NetworkService"

This reverts commit ba360cad96e0cb041d7047af30df2a35eb112449.

* Do not announce block if set to not announce block

* Revert fix

* Moving default announce block to NetworkConfig

* WIP

Forked at: ad90ab7e
Parent branch: origin/master

* WIP

Forked at: ad90ab7e
Parent branch: origin/master

* Removing boolean in favor of explicit call

* Fixing tests

* WIP

Forked at: ad90ab7e
Parent branch: origin/master

* WIP

Forked at: ad90ab7e
Parent branch: origin/master

* increase spec_version

* increase spec_version

* Fixed test

* Fixing test

* Renamed should_announce_imported_blocks to announce_imported_blocks

* Updated assert_cmd

* add post-build-test stage

* check status

* fix busybox shell

* renaming for consistency

* Show network id/version in subkey

- show network id/version when creating
- show network id/version when inspecting

* Update bin/utils/subkey/src/main.rs

use `unwrap_or_default` for more clean syntax

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

The library `sysinfo` exposes process memory as kibibytes and not bytes,
thus the value needs to be multiplied by 1024 to comply with the metric
name and the Prometheus base units [1].

[1] https://prometheus.io/docs/practices/naming/#base-units

* Fix tried to send handshake twice

* Fix wrong boolean

* Change to debug

* Add KEY_KIND_ID to the public trait

This change is being introduced for the purpose of identifying a public
key with it's identifier and algorithm "kind".

* Use `sign_with` as implemented in BareCryptoStore

* Implement `sign_with` in sc_keystore

* Fix inconsistencies, use *_KIND_ID in sp_core testing

* Rename KeyKindId to CryptoTypeId

* Remove pair-returning functions from BareCryptoStore trait

* Define CryptoTypeId in app-crypto macros

* Add functions to get keys supported by keystore

* Fix sign_with signature to include CryptoTypePublicPair

* Add `sign_with_any` and `sign_with_all`

* Use keystore.sign_with in auth_discovery

* Rename get_supported_keys -> supported_keys

* Added headers to function docstrings

* Use chain instead of extending a temp vector

* Fixed some code formatting

* Restrict size of CryptoTypeId

This is to be able to use Encode/Decode derives and the overcome having
the size being unknown at compile-time.

* Implement sign_with in the trait itself

* Remove whitespace

* Use key_type also as a CryptoTypeId in app_crypto macros

* Rename `get_keys` to `keys` in BareCryptoStore

* Remove usage of key_pair funcs in tests

* Adjust docstring for *_CYPTO_ID constants

* Fix failures

* Simplify mapping on keys

* Remove one let

* Fixed typo

* PR feedback

* remove whitespace

* Zip keys and signatures

* Use into_iter & remove cloned

* Pass index to MissingSignature

* Use typed errors instead of strings for BareCryptoStore

* Implement Debug for trait error

* Use hashsets for better performance for supported_keys

* Make sure keys are inserted into the keystore

* Make sign_with_all return type consistent with `sign_with`

* Rename Error to BareCryptoStoreError

* Rename CRYPT_TYPE_ID -> CRYPTO_ID

* Remove unnecessary CRYPTO_ID declaration in Public trait

* Convert pub key to CryptoTypePublicPair

* Fix use

* Fix code style

* Implement From on CryptoTypePublicPair in app_crypto macros

* Change CryptoTypePublicPair to a struct

* Implement Display on CryptoTypePublicPair

* Pass CryptoTypePublicPair to MissingSignature error

* Adjust docs according to function signature

* Unify keys implementation

* Fix RPC author tests

* Fix stackoverflow

* Tabify spaces

* Pass KeyTypeId to error for easier debugging

* Fix asserts

* Use ToHex to format public key

* Use constants from sp_core

* Rename testing KeyTypeId constants

* Please compiler

* Restore KeyTypeId names

apparently, they're not only used in tests

* Use BareCryptoStoreError instead of String

* Document return value

* Fix borrow check

* Convert to hashset internally

* WIP - iter_keys

* Return raw_public_keys

* Address PR feedback

* Address PR Feedback

* Fix hexdisplay import error

* Update primitives/core/src/traits.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Make transactions and block announces use notifications

* Add documentation

* Add docs.rs metadata to all cargo.toml files

* Remove docs.rs metada in substrate's cargo.toml

The signal handler will be reset to the default system one. This means
the program will be exited instead of panicking. This is required to
support piping in the console.

* Extend PerThing + Saturating

* Add saturating_pow to Saturating
* Add saturating_truncating_mul to PerThing (rounding-down mul)
* Add saturating_reciprocal_mul to PerThing (divide x by perthing)
* Provide default methods where possible

* Restore const functions

* Fix test

* Update primitives/arithmetic/src/per_things.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Add comment and test verifying no overflow

* Formatting

* Fix possible overflow and change type constraint

* Use overflow pruning for all mul

* Formatting and comments

* Improve comments and names

* Comments in `rational_mul_correction` explain overflow aversion.

* Test rational_mul_correction

* Formatting

* Docs and formatting

* Add new trait methods to Perthing type impl

* Fix signature

* saturating_pow for Delegations

* Add missing trait method to impl

Co-authored-by: Kian Paimani <[email protected]>

* Provide easy conversions from common key types to `AccountId32`

* Fix spaces instead of tabs

* hide metadata function for non generic event

* add doc for decl_storage

* Enable `wasmtime` only for `x86_64` in substrate node

When using the new feature of cargo `cargo build -Z features=itarget`,
the `wasmtime` feature will not bleed into the build for other
architectures.

* Use new cargo feature in CI

* Update CONTRIBUTING.adoc

* Update docs/CONTRIBUTING.adoc

Co-Authored-By: Bastian Köcher <[email protected]>

* Update docs/CONTRIBUTING.adoc

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

This removes the requirement to import `PerThing` to use a type that
implements the trait.

* adds state_queryStorageAt

* make at param for query_storage_at optional

* Update client/rpc/src/state/state_full.rs

Co-Authored-By: Tomasz Drwięga <[email protected]>

* adds query_storage_at to StateBackend

Co-authored-by: Tomasz Drwięga <[email protected]>

* Initial skeleton for offchain phragmen

* Basic compact encoding decoding for results

* add compact files

* Bring back Self::ensure_storage_upgraded();

* Make staking use compact stuff.

* First seemingly working version of reduce, full of todos

* Everything phragmen related works again.

* Signing made easier, still issues.

* Signing from offchain compile fine 😎

* make compact work with staked asssignment

* Evaluation basics are in place.

* Move reduce into crate. Document stuff

* move reduce into no_std

* Add files

* Remove other std deps. Runtime compiles

* Seemingly it is al stable; cycle implemented but not integrated.

* Add fuzzing code.

* Cleanup reduce a bit more.

* a metric ton of tests for staking; wip 🔨

* Implement a lot more of the tests.

* wip getting the unsigned stuff to work

* A bit gleanup for unsigned debug

* Clean and finalize compact code.

* Document reduce.

* Still problems with signing

* We officaly duct taped the transaction submission stuff. 🤓

* Deadlock with keys again

* Runtime builds

* Unsigned test works 🙌

* Some cleanups

* Make all the tests compile and stuff

* Minor cleanup

* fix more merge stuff

* Most tests work again.

* a very nasty bug in reduce

* Fix all integrations

* Fix more todos

* Revamp everything and everything

* Remove bogus test

* Some review grumbles.

* Some fixes

* Fix doc test

* loop for submission

* Fix cli, keyring etc.

* some cleanup

* Fix staking tests again

* fix per-things; bring patches from benchmarking

* better score prediction

* Add fuzzer, more patches.

* Some fixes

* More docs

* Remove unused generics

* Remove max-nominator footgun

* Better fuzzer

* Disable it ❌



* Bump.

* Another round of self-review

* Refactor a lot

* More major fixes in perThing

* Add new fuzz file

* Update lock

* fix fuzzing code.

* Fix nominator retain test

* Add slashing check

* Update frame/staking/src/tests.rs

Co-Authored-By: Joshy Orndorff <[email protected]>

* Some formatting nits

* Review comments.

* Fix cargo file

* Almost all tests work again

* Update frame/staking/src/tests.rs

Co-Authored-By: thiolliere <[email protected]>

* Fix review comments

* More review stuff

* Some nits

* Fix new staking / session / babe relation

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <[email protected]>

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <[email protected]>

* Update primitives/phragmen/compact/src/lib.rs

Co-Authored-By: thiolliere <[email protected]>

* Some doc updates to slashing

* Fix derive

* Remove imports

* Remove unimplemented tests

* nits

* Remove dbg

* Better fuzzing params

* Remove unused pref map

* Deferred Slashing/Offence for offchain Phragmen  (#5151)

* Some boilerplate

* Add test

* One more test

* Review comments

* Fix build

* review comments

* fix more

* fix build

* Some cleanups and self-reviews

* More minor self reviews

* Final nits

* Some merge fixes.

* opt comment

* Fix build

* Fix build again.

* Update frame/staking/fuzz/fuzz_targets/submit_solution.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Update frame/staking/src/slashing.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Update frame/staking/src/offchain_election.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Fix review comments

* fix test

* === 🔑

 Revamp without staking key.

* final round of changes.

* Fix cargo-deny

* Update frame/staking/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

Co-authored-by: Joshy Orndorff <[email protected]>
Co-authored-by: thiolliere <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

* client/finality-grandpa/src/until_imported: Refactor schedule_wait

Previously `BlockUntilImported::schedule_wait` took two closures, one to
report ready items and one to report items to await. None of the implementors of
`BlockUntilImported` call both closures. From a symantic perspective it
would as well not make sense to both await something and state that it
is ready.

Instead with this commit `BlockUntilImported::schedule_wait` simply
returns whether the given item needs waiting or is ready to be passed
on.

This reduces complexity by:

- Removing side effects through the two closures.

- Reducing borrowing given that `UntilImported` `ready` and `pending`
don't need to be borrowed from within the two closures.

- Removes the need for trait bounds for the two closures.

* client/finality-grandpa/src/until_imported: Fix comments

Co-Authored-By: André Silva <[email protected]>

Co-authored-by: André Silva <[email protected]>

* Improve sc-network's documentation for network protocols

* Add note about protocol id

* Apply suggestions from code review

Co-Authored-By: Max Inden <[email protected]>

Co-authored-by: Max Inden <[email protected]>