Before this pr the transaction pool validated each transaction, even if
the transaction was already known to the pool. This pr changes the
behavior to first check if we are already aware of a transaction and
thus, to only validate them if we don't know them yet. However, there is
still the possibility that a given transaction is validated multiple
times. This can happen if the transaction is added the first time, but
is not yet validated and added to the validated pool.

Besides that, this pr fixes the wrong metrics of gossiped transactions
in the network. It also moves some metrics to the transaction pool api,
to better track when a transaction actually is scheduled for validation.

* Start

* Cleanup `construct_runtime!`

* Add tests

* Fix after merge

* Update the docs

* initial mockup

* add and wipe

* track writes

* start to add to pipeline

* return all reads/writes

* Log reads and writes from bench db

* causes panic

* Allow multiple commits

* commit before ending benchmark

* doesn't work???

* fix

* Update lib.rs

* switch to struct for `BenchmarkResults`

* add to output

* fix test

* line width

* @Kianenigma

 review

* Add Whitelist to DB Tracking in Benchmarks Pipeline (#6405)

* hardcoded whitelist

* Add whitelist to pipeline

* Remove whitelist pipeline from CLI, add to runtime

* clean-up unused db initialized whitelist

* Add regression analysis to DB Tracking (#6475)

* Add selector

* add tests

* debug formatter for easy formula

* initial idea

* use all benchmarks

* broken

* working without trait

* Make work for multiple pallets

* Fix merge issues

* writer appends to file

* implement () for balances weight trait

* update name of trait

* Weights to WeightInfo

* auto trait writer

* Heap pages are configurable

* clean out runtime changes

* more clean up

* Fix string generation

* Update comments

* Update bin/node/runtime/src/lib.rs

Co-authored-by: arkpar <[email protected]>

* slots: create primitives crate for consensus slots

* offences: add method to check if an offence is unknown

* babe: initial equivocation reporting implementation

* babe: organize imports

* babe: working equivocation reporting

* babe: add slot number to equivocation proof

* session: move duplicate traits to session primitives

* babe: move equivocation stuff to its own file

* offences: fix test

* session: don't have primitives depend on frame_support

* babe: use opaque type for key owner proof

* babe: cleanup client equivocation reporting

* babe: cleanup equivocation code in pallet

* babe: allow sending signed equivocation reports

* node: fix compilation

* fix test compilation

* babe: return bool on check_equivocation_proof

* babe: add test for equivocation reporting

* babe: add more tests

* babe: add test for validate unsigned

* babe: take slot number in generate_key_ownership_proof API

* babe: add benchmark for equivocation proof checking

* session: add benchmark for membership proof checking

* offences: fix babe benchmark

* babe: add weights based on benchmark results

* babe: adjust weights after benchmarking on reference hardware

* babe: reorder checks in check_and_report_equivocation

* pallet-evm: return Ok(()) when EVM execution fails

* Bump spec version

* Init test module

* Add fail_call_return_ok test

* Fix tests and use full match pattern

Co-authored-by: Gav Wood <[email protected]>

* Initial commit

Forked at: 7e3cf944
Parent branch: origin/master

* Remove polkadot companion detection from branch name

Even though it was nice it was also error prone as there were no indication whatsoever on the PR
that a polkadot companion branch exists.

* primitives: use generic Header in testing runtime

* frame: remove unused imports

* Remove warning

Co-authored-by: Bastian Köcher <[email protected]>

* Initial commit

Forked at: facc6741
Parent branch: origin/master

* Remove polkadot companion detection from branch name

Even though it was nice it was also error prone as there were no indication whatsoever on the PR
that a polkadot companion branch exists.

* Initial commit

Forked at: 8ef1ac0e


Parent branch: origin/master

* Add send_transaction to RpcHandlers

* Extension trait for RpcHandlers

* Revert "Add send_transaction to RpcHandlers"

This reverts commit 03c89e13d404bae3f3123387dd50f026061bca82.

* Add an extension trait for BlockchainEvents

* Update test-utils/client/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update test-utils/client/src/lib.rs

* fix

* deps fix

Co-authored-by: Bastian Köcher <[email protected]>

* Move `create_inherents` into the block-builder

This moves the `create_inherents` call into the block-builder. This has
the advantage that `create_inherents` will be able to reuse the same
context that will be used when applying the extrinsics and we also save
one call to `on_initialize`. To make sure that `create_inherents` does
not modify any state, we execute it in a transaction that is
rolled-back after doing the runtime call.

* Feedback and build fix

* Update primitives/runtime/src/lib.rs

Co-authored-by: Sergei Shulepov <[email protected]>

* Update client/block-builder/src/lib.rs

Co-authored-by: Sergei Shulepov <[email protected]>

* allow specify schedule dispatch origin

* fix tests

* use caller origin for scheduled

* fix tests

* line width

* check origin for cancel

* line width

* fix some issues for benchmarking

* fix doc test

* another way to constraint origin

* fix build issues

* fix cancel

* line width

* fix benchmarks

* bump version

* enable runtime upgrade

* add migration code and test

* Update frame/scheduler/src/lib.rs

Co-authored-by: Gavin Wood <[email protected]>

* expose migration method

* add notes

* bump version

* remove on_runtime_upgrade

* fix test

Co-authored-by: Gavin Wood <[email protected]>

* Update SubstrateCli to return String

* Add default implementation for executable_name()

* Use display instead of PathBuf

* Get file_name in default impl of executable_name

* Remove String::from and use .into()

* Use default impl for executable_name()

* Use .as_str() and remove useless .to_string()

* Update only sp-io when running companion build

* Remove unneeded update of sp-io in CI

Co-authored-by: Cecile Tonglet <[email protected]>

* Initial commit

Forked at: 4919c808
Parent branch: origin/master

* Drop the tokio runtime before the task_manager

The tokio runtime must be dropped before the task_manager. Otherwise the objects the task_manager
keep alive are dropped before the tasks are finished.

* :)

* Slight tidy

* Remove ServiceBuilderCommand

* Remove whitespace

* Keep task manager alive for check_block/import_blocks

* Pass task_manager to run_until_exit

* Make task_manager in run_until_exit and make subcommands async

* Change the async_run fn to return a future and task manager

* async_run should take a result fn

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Fix spaces in export_raw_state

Co-authored-by: Bastian Köcher <[email protected]>

* Restrict `Protected` to some heap types.

* Comment abut Protected usage.

* Remove Protected from crypto, use secrecy crate for existing uses.

* use a parse function

* fix error convert

* Rename and move secretY string function.

* std result

* add burnin-label-notification.yml

* fix burnin-label-notification.yml

* fix burnin-label-notification.yml

* fix burnin-label-notification.yml

* fix burnin-label-notification.yml

* Update .github/workflows/burnin-label-notification.yml

Co-authored-by: Benjamin Kampmann <[email protected]>

Co-authored-by: Benjamin Kampmann <[email protected]>
Co-authored-by: Gav Wood <[email protected]>

statements in github actions cannot use ", must use '

Co-authored-by: Gav Wood <[email protected]>

The transaction queue size alert has been firing with a constant 10
transactions in the queue. While maybe problematic those 10 transactions
don't need to be the same across scrape intervals.

Instead of alerting with a size above 10, alert based on two things:

1. Monotonically increasing queue size

2. Upper limit queue size reached

* Fix mocking multiple http calls in the same function call

Fixes an issue where a function call would perform more than one http request and wait for each to complete before proceeding. The `RequestId` comes from the length of the `requests` collection in the `OffchainState` and if a request is completed before the next one starts it will be removed and the "next expected" will be off by one. This PR tries to fix that by using a request counter that tracks how many requests have been performed so that we can `remove()` items from the `expected_requests` at the right index.

I suspect that this is a sub-optimal soluton and perhaps requests and their mocks should live side by side in the same collection, e.g. in a tuple of `(PendingRequest, Option<ExpectedRequest>)`.

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <[email protected]>

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bernhard Schuster <[email protected]>

* Panic on overflow

* Update primitives/core/src/offchain/testing.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Use a Deque and push/pop expected requests

* fix test

Co-authored-by: Bernhard Schuster <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* debug (ci): ci config [skip ci]

* debug (ci): fix runtime benchmarks

* fix identity benchmarks

* fix utility benchmarks

* Revert "debug (ci): ci config [skip ci]"

This reverts commit 081b175b

.

* change (ci): touch ci config to run CI

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Gav Wood <[email protected]>

* Update to make cargo-deny happy

* Remove cargo deny from CI

* change (ci): run cargo deny only on tags and schedules

Co-authored-by: Denis P <[email protected]>

* grandpa: replace Result<(), ()> with Option<()>

* grandpa: replace &Option<T> with Option<&T>

* grandpa: cleanup local id and keystore usages

* grandpa: return bool on check_message_signature

* grandpa: fix erroneous log message on startup

* grandpa: fix test

* Start

* Remove debug println

* Add tests

* proposer benchmark

* update cargo.lock

* WIP

* Making progress

* Almost ready

* Get service tests compiling

* Fix node screenshot

* Line widths

* Fix node cli tests

* Fix node cli warning

* ChainComponents -> ServiceComponents, fix tests

* make spawn_handle public

* Remove spawnnamed impl for taskmanager

* Move the keep alive stuff to the task manager

* Move the telemetry, base path, rpc keep_alive to the service builder

* Make the task manager keep alive an internal detail

* Rewrite the browser start_client future

* Remove run_node etc

* Revert my personal changes to browser-demo/build.sh

* use |config|

* Add a runtime_version function to SubstrateCli

* Reexport role and runtime version from sc cli

* Update Cargo.lock

* runtime_version -> native_runtime_version

* Pass chain spec to native_runtime_version for polkadot

* Fix line widths

* Traitify ServiceComponents Client

* Fix tx-pool returning the same transaction multiple times

This fixes a bug that lead to returning the same transaction multiple
times when iterating the `ready` iterator. Internally the transaction
was kept in the `best` list and could be duplicated in that list be
re-inserting it again. This `best` list is using a `TransactionRef`
which internally uses a `insertion_id`. This `insertion_id` could lead
to the same transaction being inserted multiple times into the `best`
list.

* Update client/transaction-pool/src/testing/pool.rs

Co-authored-by: Nikolay Volf <[email protected]>

Co-authored-by: Nikolay Volf <[email protected]>

* Update to libp2p-0.20.0

* Update to `libp2p-0.20.1`.

Co-authored-by: Pierre Krieger <[email protected]>

* Allow `retract_tip` on `tip_new`

* initial migration code

* test migration

* make pub

* bump spec

* Increase the limit for the maximum size of the telemetry name

* Fix test

* implement Contains for pallet-membership

* bump version