Remove `txpool` as an export of `sc_transaction_pool`, exporting the used components instead. (#9217)

* Remove `txpool` as an export of `sc_transaction_pool`, exporting the used components instead.

* Fix tests

* Use MAX associated const

* Add transaction pool docs.

* Extra docs.

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <[email protected]>

* Expand on some review comments.

* Update README.md

Fixed typos / spellings

Co-authored-by: Pierre Krieger <[email protected]>
Co-authored-by: Squirrel <[email protected]>

* Less slices

Co-authored-by: Bastian Köcher <[email protected]>

* Add dummy Debug instance to AuthorityDiscoveryService.

* Update client/authority-discovery/src/service.rs

More idiomatic print

Co-authored-by: Pierre Krieger <[email protected]>

Co-authored-by: Pierre Krieger <[email protected]>

* State sync

* Importing state fixes

* Bugfixes

* Sync with proof

* Status reporting

* Unsafe sync mode

* Sync test

* Cleanup

* Apply suggestions from code review

Co-authored-by: cheme <[email protected]>
Co-authored-by: Pierre Krieger <[email protected]>

* set_genesis_storage

* Extract keys from range proof

* Detect iter completion

* Download and import bodies with fast sync

* Replaced meta updates tuple with a struct

* Fixed reverting finalized state

* Reverted timeout

* Typo

* Doc

* Doc

* Fixed light client test

* Fixed error handling

* Tweaks

* More UpdateMeta changes

* Rename convert_transaction

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Code review suggestions

* Fixed count handling

Co-authored-by: cheme <[email protected]>
Co-authored-by: Pierre Krieger <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* grandpa: don't use block_on in Environment::report_equivocation

* grandpa: add issue number to todo

* make SelectChain async

* make JustificationImport async

* aura, babe: proposal slot lenience must take into account proposal portion

* slots: add support for max_block_proposal_slot_portion

* fix compilation

* slots: add tests

* aura: fix comment

Co-authored-by: Bastian Köcher <[email protected]>

* slots: log the actual proposing duration after lenience is applied

Co-authored-by: Bastian Köcher <[email protected]>

* babe: add comments to block weight and expose block_weight function

* babe: expose function for block weight key

* grandpa: make gossip more conservative (and fair)

* grandpa: make round commit timer dependent on gossip_duration

* grandpa: add gossip tests

* grandpa: reduce variance in tests

* Make it possible to override maximum payload of RPC

* Finish it.

* remove todo.

* Update client/cli/src/commands/run_cmd.rs

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Incorporate suggestions

* Thread rpc_max_payload from configuration to trace_block

* Try obey line gitlab/check_line_width.sh

* update state rpc tests

* Improve readbility

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Zeke Mostov <[email protected]>
Co-authored-by: David <[email protected]>

This instructs the Aura runtime api to skip initialize block, when
requesting the authorities. This is important, as we don't want to use
the new authorities that should be used from the next block on.
Besides that, it removes the caching stuff. The cache is not available
on full nodes anyway. In the future we should store the authorities
probably in the aux store.

* grandpa: cleanup sync bounds

* grandpa: cleanup imports

* remove cargo patch

* remove prefixed content with limit.

* test match

* factor comment and factor ext limit removal.

* fix benchmark

Co-authored-by: Shawn Tabrizi <[email protected]>

* Test restoring zeroed data

* Change to u64

* Decommit instance memory after a runtime call on Linux

* Update documentation for the test

* Remove unfinished comment

* Use saturating_sub.

Also update the doc comment.

* Precise RSS tracking in the test

Instead of tracking RSS for the whole process we just look at the particular mapping that is associated with the linear memory of the runtime instance

* Remove unused import

* Fix unused imports

* Fix the unused imports error for good

* Rollback an accidental change to benches

* Fix the test

* Remove now unneeded code

* Add function to test whether function is exported in wasm blob

* Address Feedback

* Update based on feedback

* Transaction pool: Remove futures-diagnose and thread pool

This pr removes `futures-diagnose` as this isn't used anymore. Besides
that the pr also removes the thread pool that was used to validate the
transactions in the background. Instead of this thread pool we now spawn
two separate long running tasks that we use to validate the
transactions. All tasks of the transaction pool are now also spawned as
essential tasks. This means, if any of these tasks is stopping, the node
will stop as well.

* Update client/transaction-pool/src/api.rs

* Migrate ProfilingLayer to tracing registry API

* Remove the `current_span` field from `BlockSubscriber`.

* Bump the `tracing-subscriber` version

* Fix Gitlab CI

(We send network information to prometheus)

* Remove TelemetryWorker::with_transport
Make logging more useful

* Re-instate TelemetryWorker::with_transport

* Fix typo, don't spam

* Add optional `rpc-http-threads` cli arg

* Update `http::ServerBuilder`threads

* Periodically call alloc_slots on all slots

* Add test

* validation extension in sp_io

* need paths

* arc impl

* missing host function in executor

* io to pkdot

* decode function.

* encode primitive.

* trailing tab

* multiple patch

* fix child trie logic

* restore master versionning

* bench compact proof size

* trie-db 22.3 is needed

* line width

* split line

* fixes for bench (additional root may not be needed as original issue was
with empty proof).

* revert compact from block size calculation.

* New error type for compression.

* Adding test (incomplete (failing)).
Also lacking real proof checking (no good primitives in sp-trie crate).

* There is currently no proof recording utility in sp_trie, removing
test.

* small test of child root in proof without a child proof.

* remove empty test.

* remove non compact proof size

* Missing revert.

* proof method to encode decode.

* consensus: add trait to control justification sync process

* network: implement JustificationSyncLink for NetworkService

* slots: handle justification sync in slot worker

* babe: fix slot worker instantiation

* aura: fix slot worker instantiation

* pow: handle justification sync in miner

* babe: fix tests

* aura: fix tests

* node: fix compilation

* node-template: fix compilation

* consensus: rename justification sync link parameter

* aura: fix test compilation

* consensus: slots: move JustificationSyncLink out of on_slot

* Transaction storage runtime module

* WIP: Tests

* Tests, benchmarks  and docs

* Made check_proof mandatory

* Typo

* Renamed a crate

* Apply suggestions from code review

Co-authored-by: Kian Paimani <[email protected]>

* Added weight for on_finalize

* Fixed counter mutations

* Reorganized tests

* Fixed build

* Update for the new inherent API

* Reworked for the new inherents API

* Apply suggestions from code review

Co-authored-by: cheme <[email protected]>
Co-authored-by: Alexander Popiak <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Store transactions in a Vec

* Added FeeDestination

* Get rid of constants

* Fixed node runtime build

* Fixed benches

* Update frame/transaction-storage/src/lib.rs

Co-authored-by: cheme <[email protected]>

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: cheme <[email protected]>
Co-authored-by: Alexander Popiak <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Transaction pool: Ensure that we prune transactions properly

There was a bug in the transaction pool that we didn't pruned
transactions properly because we called `prune_known`, instead of `prune`.

This bug was introduced by:
https://github.com/paritytech/substrate/pull/4629

This is required to have stale extrinsics being removed properly, so
that they don't fill up the tx pool.

* Fix compilation

* Fix benches

* ...

`ready_at` returns when we have processed the requested block. However,
on startup we already have processed the best block and there
are no transactions in the pool on startup anyway. So, we can set `updated_at`
to the best block on startup.

Besides that `ready_at` now returns early when there are no ready nor
any future transactions in the pool.

This reverts commit 84e40238.

* Remove futures-diagnose

* Use `SpawnTaskHandle`s for spawning tasks in the tx pool

* Box the spawner

* Fix tests

* Use the testing task executor

* Spelling corrections

* As this might break let's do as a separate PR