This PR refactors the metrics measuring and Prometheus exposing entity in sc-service into its own submodule and extends the parameters it exposes by:

- system load average (over one, five and 15min)
- the TCP connection state of the process (lsof), refs #5304
- number of tokio threads
- number of known forks
- counter for items in each unbounded queue (with internal unbounded channels)
- number of file descriptors opened by this process (*nix only at this point)
- number of system threads (*nix only at this point)

refs #4679

Co-authored-by: Max Inden <[email protected]>
Co-authored-by: Ashley <[email protected]>

* Simple Payouts

* explicit test for out of order claiming

* Add `payout_all` benchmark

* Fix merge

* add docs

* change event to controller

* Fix timestamp test warnings

* Revert "change event to controller"

This reverts commit 5d4a9783

.

* Update Reward event doc

* Add "non-production" test

* add unlock chunk to test

* fix merge

* End payout early if no reward points

* payout_validator -> payout_stakers

* bring back payout nominator/validator, but limit their use to before migration era

* Add test for before migration

* New payout works for the era that we migrate

* Fix logic, check that migration era works

* Migrate Era tests (copypasta)

* Move comment

* Add mock back to external functions

* Fixes based on review from gui

* Update Cargo.lock

* Update Cargo.lock

* small docs update

Co-authored-by: joepetrowski <[email protected]>

* EnsureOrigin is a frame abstraction - it should be in frame_support

* Fixes

* support: add lateness trait

* babe: implement the lateness trait

* babe: add docs about lateness entry lifetime

* babe: don't use option for lateness storage entry

* Split the Roles bitfield in three

* Forgot to include some changes

* Fix cli test

* More test fixes

* Oh God, merging master broke other tests

* Didn't run the doctests

* Address review

* I'm trying to fix the build blindly because it's taking a good hour to compile on my machine

* Address some review

* Also update the peerset's API to make sense

* Fix peerset tests

* Fix browser node

* client: distinguish between local and network authority

Co-authored-by: André Silva <[email protected]>

* test (ci): prepopulate caches with debug

* test (ci): error in path

* optimization (ci): CARGO_HOME cache prepopulation

* Make Dispatchable return the actual weight consumed

Add PostInfo associated type to Dispatchable and have frame implement
Dispatchable { type PostInfo = PostDispatchInfo } where PostDispatchInfo
contains the actual weight consumed.

* Fix whitespace issues in docs

* custom benchmark runner

* add license preamble

* delete old benchmarks

* update toml

* imporove macro

* tabs

* add size

* size fixes

* add docs for cli

* Run benchmarks on block 1

* Put example benchmarks behind feature flag

* add feature to cargo.toml

* typo

* Add `frame_system` trait bound

* Update instance benchmarks too

* ci: fix echo sed failure

Co-authored-by: gabriel <[email protected]>

* Better logging for notifications and buffer size increase

* Address review

* Improve warning about notifications queue and remove spurious triggers

* Update CONTRIBUTING.adoc

* Update docs/CONTRIBUTING.adoc

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

* Add a sub_libp2p_notifications_queues_size Prometheus metric

* Fix network tests

* Address review

* More emojies and spaces

* Sync emojis

* More emoji

This pr ensures that we don't panic because of a deadline that is
already reached in basic authorship. This is mainly useful for debug
builds, as release builds normally don't have debug assertions enabled.

* Registering state from overlay.

* fix

* fix2

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Switch to the new protocol

* Oops, forgot to remove light_dispatch.rs

* Fix tests

* Address review

This was removed in https://github.com/paritytech/substrate/pull/5291
but is still too early to do it as it requires nodes still on Polkadot
v0.7.26 to resync from scratch.

* A few missing emojies

* P2p message emojis

* Add a bit of colour

* format

* Introduce a couple of spaces

* Some spaces

* check_polkadot.sh: use the branch matching substrate's branch if possible

* convert to grep

* update comment

* Initial draft of the logic

* Build and tests

* Make work with new initialize infratructure.

* Update frame/scheduler/src/lib.rs

Co-Authored-By: Marcio Diaz <[email protected]>

* Update frame/scheduler/src/lib.rs

Co-Authored-By: Marcio Diaz <[email protected]>

* Update frame/scheduler/src/lib.rs

Co-Authored-By: Marcio Diaz <[email protected]>

* Update frame/scheduler/src/lib.rs

Co-Authored-By: Marcio Diaz <[email protected]>

* Fix test

* Update frame/scheduler/src/lib.rs

Co-Authored-By: Marcio Diaz <[email protected]>

* Rejig interface to make it more useful for democracy.

* Try to get democraxy module to make use of scheduler.

* Make democracy use scheduler.

* Use actual max weight for enactent

* Remove TODO

* Fix runtime build

* Minor cleanup

* Fix scheduler.

* Fix benchmarks

* Fix

* Fix

* Fix

* More bench fixes

* Fix

* Fix.

* Add more bench constants.

* Fix cancel_queued bench.

* Fix test comment.

* Update frame/scheduler/src/lib.rs

Co-Authored-By: Marcio Diaz <[email protected]>

Co-authored-by: Marcio Diaz <[email protected]>

* Reset by removing pos-genesis changes

* CLI option for DB cache size

* Update Cargo.lock

Co-authored-by: Shawn Tabrizi <[email protected]>

* Move tests

* Move around feature cfgs

* Start benchmarks

* as_multi bench

* finish

* fix tests

* Update frame/utility/src/benchmarking.rs

Co-Authored-By: Marcio Diaz <[email protected]>

* Update Cargo.toml

* fix test

Co-authored-by: Marcio Diaz <[email protected]>

* A patch

* Fix

* Better approach.

* Fix grumbles

* client/finality-grandpa: Instrument until-imported queue

The `UntilImported` queue takes as input finality grandpa messages that
depend on blocks that are not yet imported and holds them back until
those blocks are imported.

This commit adds a basic metric, the amount of messages waiting in the
queue, to the module. For now this metric is only available for the
global `UntilImported` queue awaiting blocks for commit and catch-up
messages.

* client/finality-grandpa/src/until_imported: Update metric help text

Co-Authored-By: Ashley <[email protected]>

Co-authored-by: Ashley <[email protected]>

* Increase limit on light client response size

* Address review

* Add deposit/withdraw events

* Remove balances trait

* Don't populate runtime events in genesis

* typo

* Change to block zero

* Fix vesting tests

* Update frame/system/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update frame/system/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Add test

* Fix test

* Fix contract tests

* Fix phragmen tests

* Fix Generic Assets Tests

* Fix offences tests

* Fix im-online

* fix recovery

* Fix utility tests

* Shorter

* Use ext

Co-authored-by: Bastian Köcher <[email protected]>

* Batch benchmarks together with `*` notation.

* Fix short structopt conflict

* Return error if `batches` is empty

* Move fast benchmarks macro into `frame_benchmarking` (#5445)

* Move macro into `frame_benchmarking`

* Update docs

* Extra line

* Return error if `batches` is empty

Co-authored-by: Shawn Tabrizi <[email protected]>