* WIP

* WIP

* Test

* bug fix

* WIP

* Revert "WIP"

This reverts commit 4e51e9adfdf0dc7cf37b562b60a0e83ca1d0b00d.

* doc

* Improve comment on why all spans are preserved

* Added missing suggestion from previous PR

* Use BoxFuture

* Move TelemetrySpan creation to sc-cli, need to test...

* Test code

* Adapt user code

* Revert "Test code"

This reverts commit 333806b2fe1626efaa2691f9f44d0b4dd979bc36.

* Update client/service/src/task_manager/mod.rs

Co-authored-by: David <[email protected]>

* Better & simpler solution

Co-authored-by: David <[email protected]>

* Doc fixes for sc-telemetry

* Fix flag to disable log reloading

* Forgot to reverse the conditions

* Apply suggestion

* Rename pattern to directives

* Rename GlobalLoggerBuilder to LoggerBuilder

* Return instead of expect

* Use transparent outside the enum

* Update client/tracing/src/logging/directives.rs

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

* Introduce sc_peerset::DropReason

* Fix peerset tests

* Clarify and expand docs.

* clarify that a pallet can verify an inherent without providing one.

* Clarify what calls `is_inherent_required`.

* caution and link to issue

* typo

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

* Introduce a `Slot` type

Instead of having some type definition that only was used in half of the
code or directly using `u64`, this adds a new unit type wrapper `Slot`.
This makes it especially easy for the outside api to know what type is
expected/returned.

* Change epoch duratioC

* rename all instances of slot number to slot

* Make the constructor private

Co-authored-by: André Silva <[email protected]>

* Add stateless verification helper function.

* Split MMR primitives.

* Add RuntimeAPI

* RuntimeAPI with OpaqueLeaves

* Bump spec_version,.

* Move primitives back to frame.

* Fix OpaqueLeaf encoding.

* Add block number to frame_system implementation of LeafDataProvider.

* Relax leaf codec requirements and fix OpaqueLeaf

* Add root to debug line.

* Apply suggestions from code review

Co-authored-by: Hernando Castano <[email protected]>

* Typo.

Co-authored-by: Hernando Castano <[email protected]>

* Fix state cache for cumulus

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

Remove checks that involve cross checking authorities in the runtime against what we have stored in
the AuthoritySetChanges.

* Rewrite the async code in `BasicQueue`

This is some smaller change to rewrite the async code in `BasicQueue`. I
require this for some other pr I'm working on ;)

* Update primitives/consensus/common/src/import_queue/basic_queue.rs

Co-authored-by: André Silva <[email protected]>

* Update primitives/consensus/common/src/import_queue/basic_queue.rs

Co-authored-by: André Silva <[email protected]>

* Update primitives/consensus/common/src/import_queue/basic_queue.rs

Co-authored-by: André Silva <[email protected]>

* Hmm :D

Co-authored-by: André Silva <[email protected]>

* Converting pallet-template to Framev2 macro's

* Add newline

* Convert all indents to tabs

* Update bin/node-template/pallets/template/src/lib.rs

* Update bin/node-template/pallets/template/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* client/network: Report reputation changes via response

When handling a request by a remote peer in a request response handler,
one might want to in- or de-crease the reputation of the peer. E.g. one
might want to decrease the reputation slightly for each request, given
that it forces the local node to do work, or one might want to issue a
larger reputation change due to a malformed request by the remote peer.

Instead of having to pass a peerset handle to each request response
handler, this commit suggests to allow handlers to isssue reputation
changes via the provided `pending_response` `oneshot` channel.

A reputation change issued by a request response handler via the
`pending_response` channel is received by the
`RequestResponsesBehaviour` which passes the reputation change up as an
event to eventually be send to a peerset via a peerset handle.

* client/network/req-resp: Use Vec::new instead of None::<Vec<_>>

* client/network: Rename Response to OutgoingResponse

Given that a request-response request is not called `Request` but
`InomingRequest`, rename a request-response response to
`OutgoingResponse`.

* client/finality-grandpa-warp: Send empty rep change via response

* Fix weight syntax in comments

* Mention to add `IsType` bound

* Link to subsee

* Fix link

* Update frame/support/procedural/src/pallet/parse/call.rs

Co-authored-by: David <[email protected]>

* Apply review suggestion from @dvdplm

, make StorageInstance doc link

* fix ui test

Co-authored-by: David <[email protected]>
Co-authored-by: thiolliere <[email protected]>

* WIP converting balances tests to construct_runtime

* Converting balances tests_local to construct_runtime

* Fix up system and balances Events

* Use static Call instance in tests

* Migrate indices to construct_runtime

* Migrate babe test to construct_runtime

* Update frame/indices/src/mock.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update frame/babe/src/mock.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update frame/babe/src/mock.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Remove redundant import

Co-authored-by: Guillaume Thiolliere <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* client/network/req-resp: Add unit test for request id collision

* client/network/req-resp: Prevent request id collision

`RequestId` is a monotonically increasing integer, starting at
`1`. A `RequestId` is unique for a single `RequestResponse`
behaviour, but not across multiple `RequestResponse` behaviours. Thus
when handling `RequestId` in the context of multiple
`RequestResponse` behaviours, one needs to couple the protocol name
with the `RequestId` to get a unique request identifier.

This commit ensures that pending requests (`pending_requests`) and
pending responses (`pending_response_arrival_time`) are tracked both by
their protocol name and `RequestId`.

* client/network/req-resp: Remove unused import

* client/network/req-resp: Introduce ProtocolRequestId struct

* client/network/req-resp: Update test doc comment

Treat `RequestId` as an opaque type.

* client/network/req-resp: Improve expect proof

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: David <[email protected]>

* doc improvment

* additional fixes

* another fix

* better code suggestion

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Apply suggestions from code review

Co-authored-by: Alexander Popiak <[email protected]>

* Apply suggestions from code review

Co-authored-by: Alexander Popiak <[email protected]>

* apply suggestion

* apply suggestion

* Update frame/support/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* apply suggestion

* better guideline on reexport

* Update frame/support/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update frame/support/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update frame/support/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update frame/support/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update frame/support/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* apopiak suggestion

* clearer check suggestion

* Update frame/support/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update frame/support/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

* Update frame/support/src/lib.rs

Co-authored-by: Alexander Popiak <[email protected]>

Co-authored-by: David <[email protected]>
Co-authored-by: Alexander Popiak <[email protected]>

Found this very minor typo when browsing the docs.

* Moving offchain change set to state machine overlay change set,
preparing use of change set internally.

* Make change set generic over key and value, and use it for offchain
indexing.

* test ui change

* remaining delta

* generating with standard method

* Remove 'drain_committed' function, and documentation.

* Default constructor for enabling offchain indexing.

* Remove offchain change specific iterators.

* remove pub accessor

* keep previous hierarchy, just expose iterator instead.

* Update primitives/state-machine/src/overlayed_changes/mod.rs

Co-authored-by: Tomasz Drwięga <[email protected]>

* fix line break

* missing renamings

* fix import

* fix new state-machine tests.

* Don't expose InnerValue type.

* Add test similar to set_storage.

* Remove conditional offchain storage (hard to instantiate correctly).

* fix

* offchain as children cannot fail if top doesn't

Co-authored-by: Addie Wagenknecht <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>

* grandpa: persist block number for last block of authority set

* grandpa: fix authority_set_changes field in tests

* grandpa: fix date on copyright notice

* grandpa-rpc: implement cleaner api for prove finality rpc

* grandpa-rpc: replace the old prove_finality with the new one

* grandpa: undo accidental whitespace change

* grandpa-rpc: start work on redo of the finality_proof RPC API

* grandpa: manual impl of Decode for AuthoritySet

* grandpa: add comment about appending changes for forced changes

* grandpa: flip order in set changes, tidy up some comments

* grandpa: update some of the doc comments

* grandpa: store authority set changes when applying forced changes

* grandpa: simplify finality_proof.rs

* grandpa: move checks and extend tests in finality_proof

* grandpa: address first set of review comments

* grandpa: check that set changes have well-defined start

* grandpa: rework prove_finality and assocated tests

* grandpa: make AuthoritySetChanges tuple struct

* grandpa: add assertions for tracking auth set changes

* grandpa: remove StorageAndProofProvider trait

* grandpa: return more informative results for unexpected input to RPC

* grandpa: tiny tweak to error msg

* grandpa: fix tests

* grandpa: add error specific to finality_proof

* grandpa: fix review comments

* grandpa: proper migration to new AuthoritySet

* grandpa: fix long lines

* grandpa: fix unused warning after merge

Co-authored-by: André Silva <[email protected]>

* Made a start

* So the proof between authority set is phragmen one, this is crazy big,
or is there some signing of the result : that is the storage key, damn?

* ok getting from header digest seems doable.

* for testing

* get set id from storage directly (should use runtime to handler change).

* move test to init

* correct auth key

* fix iteration

* Correct proof content

* actually update block number.

* actually check last justif against its header

* justification relation to new authorities through header hash check is
needed here. This assumes the hash from header is calculated.

* Few changes

* Connected up cheme's branch

* Clean up

* Move things around a bit so that adding the grandpa warp sync request response protocol happens in the node code

* Nits

* Changes to comments

* Cheme changes

* Remove todos and test compile.

* Rename _authority_ related proof function to _warp_sync_ .

* Update client/grandpa-warp-sync/src/lib.rs

quick fix

* Put the warp sync request response protocol behind a feature flag because we dont' need it on a light client.

* Update client/grandpa-warp-sync/src/lib.rs

Quick fix

* Update Cargo.lock

* Adding test, comment on limitation related to 'delay', this could
be implemented but with a cost.

* Set between a delay override last fragment.

* Check for pending authority set change at start.

* adjust index

* custom cache is not a good idea.

* Use a simple cache instead.

* restore broken indentation

* Address crate rename

* Merge conflict badly resolved, sorry

Co-authored-by: cheme <[email protected]>
Co-authored-by: Pierre Krieger <[email protected]>

* Make offchain indexing work

This fixes some bugs with offchain indexing to make it actually working ;)

* Fix tests

* Fix browser build

* Update client/db/src/offchain.rs

Co-authored-by: cheme <[email protected]>

* Remove seperation between prefix and key

Co-authored-by: cheme <[email protected]>

- Fixing Link
- Clarify that no-force-push applies to Pull Requests, too.

* Attempt to remove the where bounds.

* Fix further and further.

* Format better.

* Update primitives/npos-elections/src/lib.rs

* fix build

* remove unused

Sentry nodes are deprecated. Thus there is no need for
`.maintain/sentry-node` to spin up a sentry node test environment.
Instead this commit rewrites the setup to contain two full-connected
validators and one light client.

With the steps below one can now spin up a local test network with
two validators, one light-client, Prometheus and Grafana.

- cargo build --release
- sudo docker-compose -f .maintain/local-docker-test-network/docker-compose.yml up

* Fix elections-phragmen and proxy issue

* remove TODO

* Update bond to be per-vote

* Update frame/elections-phragmen/src/lib.rs

* Fix benchmakrs

* Fix weight as well.

* Add license

* Make weight interpreted wasm! 🤦🏻

‍♂️

* Remove a bunch of TODOs

* Add migration

* Better storage version.

* Functionify.

* Fix deposit scheme.

* remove legacy bond.

* Master.into()

* better logging.

* Fix benchmarking test

* Fix confused deposit collection.

* Add fine

* Better name for storage item

* Fix name again.

* remove unused

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_elections_phragmen

* new weight fns

* Fix build

* Fix line width

* fix benchmakrs

* fix warning

* cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_elections_phragmen

* Tune the stake again

* cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_elections_phragmen

* All tests work again.

* A large number of fixes.

* more fixes.

* Fix node build

* Some fixes to benchmarks

* Fix some warnings.

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_elections_phragmen --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/elections-phragmen/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_elections_phragmen --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/elections-phragmen/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* a batch of review comments.

* Fix a test.

* Fix some more tests.

* do migration with pallet version???

* Final touches.

* Remove unused storage.

* another rounds of changes and fixes.

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* Review grumbles.

* Fix a bit more.

* Fix build

* Experimental: independent migration.

* WIP: isolated migration logics

* clean up.

* make migration struct private and move migration to own file

* add doc

* fix StorageInstance new syntax

* Update frame/elections-phragmen/src/migrations_3_0_0.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

* another round of self-review.

* bit better formatting

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_elections_phragmen --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/elections-phragmen/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Fix tests.

* Round of self-review

* Clean migrations

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_elections_phragmen --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/elections-phragmen/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Revert unwanted change to construct-runtime

Co-authored-by: Gavin Wood <[email protected]>
Co-authored-by: Guillaume Thiolliere <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Allow validators to block and kick their nominator set.

* migration

* Test

* Better migration

* Fixes

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Update frame/staking/src/lib.rs

Co-authored-by: Shawn Tabrizi <[email protected]>

Co-authored-by: Parity Benchmarking Bot <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* PRINT_PALLET_UPGRADE=1 cargo check -p frame-system

* Copy attributes, imports, mods and type defs

* Copy Config trait

* Annotate constants

* Tabify

* Migrate hooks

* Upgrade template rename interface to hooks

* Migrate pallet call

* Migrate Event

* Migrate Error

* Migrate Origin

* Remove optional validate_unsigned

* Remove remaining TODO_MAYBE_WHERE_CLAUSE

* Overwrite original lib.rs with migrated lib2.rs.

* Add required Event IsType constraint

* Add disable supertrait check

* Fix leftover Trait trait

* Add missing pallet prefix for weight attributes

* Add missing Error type parameter

* Add missing Hooks type parameter

* Private call visibility, restore original helper types and helpers etc

* Fix hooks type parameter

* Rename RawEvent to Event

* Add missing storage type annotations

* Remove unused imports

* Add GenesisConfig helpers for compat

* Fix unused import warnings

* Update frame/support/procedural/src/storage/print_pallet_upgrade.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Fix test errors and warnings

* Fix remaining errors and warnings

* Apply review suggestion: fix formatting

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: annotate BlockLength as constant

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: add triling comma

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: add triling comma

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: add trailing comma

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: fix storage type indentation

* Apply review suggestion: remove redundant Origin type alias

* Add missing codec derives for BlockLength

* Restore module docs

* Module -> Pallet renamel

* Revert "Update frame/support/procedural/src/storage/print_pallet_upgrade.rs"

This reverts commit d2a2d5b6

* Apply review suggestion: merge crate imports

Co-authored-by: Alexander Theißen <[email protected]>

* Revert "Upgrade template rename interface to hooks"

This reverts commit 306f0239

* Single line import

* Refactor generated genesis build

* Import sp_io::storage

* Revert previous, fully qualify sp_io::storage

* Fix ui tests

* Fix errors after merge, missing changes

* Set UpgradedToDualRefCount to true in genesis build

* Annotated Runtime version with constant, exposing it via metadata

* Add metadata attribute

Co-authored-by: Guillaume Thiolliere <[email protected]>
Co-authored-by: Alexander Theißen <[email protected]>

* Disable Nagle algorithm

* Oops, didn't compile

* Sync: Propagate block announcement data

This pr adds a feature to the sync protocol to propagate the data that
we received alongside a block announcement. This is done by adding a
cache that caches the last X block announcement data where X is set to
the number of `in_peers` (giving every peer the chance to send us a
different block). This will be required by parachains to ensure that
even peers who are not connected to a collator receive the data
alongside the block announcement to properly validate it and request the
block.

* Review comment

* Bring back the code and add new variant to ensure we don't insert block
announce data when something wasn't checked

* Also use out_peers