* BlockId removal: refactor: StorageProvider

It changes the arguments of `Backend::StorageProvider` trait from:
block: `BlockId<Block>` to: hash: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/api/src/backend.rs

Co-authored-by: Dmitrii Markin <[email protected]>

* GrandpaBlockImport::current_set_id reworked

* ExportStateCmd reworked

* trigger CI job

* trigger CI job

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Dmitrii Markin <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

* implement storage decode length for BTreeSet

* Orderly moving of things around

* include test for append and decode_len

* fix cargo clippy issue

* Bump clap to 3.2.22

* Replace `from_os_str` with `value_parser`

* Replace `from_str` and `try_from_str` with `value_parser`

* Move possible_values to the new format

* Remove unwanted print

* Add missing match branch

* Update clap to 4.0.9 and make it compile

* Replace deprecated `clap` macro with `command` and `value`

* Move remaining `clap` attributes to `arg`

* Remove no-op value_parsers

* Adjust value_parser for state_version

* Remove "deprecated" feature flag and bump to 4.0.11

* Improve range

Co-authored-by: Bastian Köcher <[email protected]>

* Apply suggestions

* Trigger CI

* Fix unused error warning

* Fix doc errors

* Fix ArgGroup naming conflict

* Change default_value to default_value_t

* Use 1.. instead of 0..

Co-authored-by: Bastian Köcher <[email protected]>

* execute try-state at end of tests

* run post condition only with try runtime

* Revert "run post condition only with try runtime"

This reverts commit 7db0ecf7eaa2ee5afa5a995487b73d023ba3bcd9.

* voterlist contains validators as well

* fmt

* simplify

* fmt

Co-authored-by: parity-processbot <>

* cargo upgrade libp2p

* Get rid of `NetworkBehaviourEventProcess` in handling of `CustomMessageOutcome`

* Get rid of `NetworkBehaviourEventProcess` in handling of `request_responses::Event`

* Get rid of `NetworkBehaviourEventProcess` in handling of `peer_info::PeerInfoEvent`

* Get rid of `NetworkBehaviourEventProcess` in handling of `DiscoveryOut`

* Get rid of `poll()` method in `Bahaviour`

* minor: comments

* Upgrade libp2p to 0.49.0 (unreleased)

* Support multiple Kad protocol names

* Make borrow checker happy

* minor: wording

* Make substrate build with libp2p-0.49.0

* rustfmt

* Get rid of MdnsWrapper

* Resolve deprecation warnings

* Fix documentation

* Apply suggestions from code review: fix typos

Co-authored-by: Aaro Altonen <[email protected]>

* Apply suggestion: simplify kad protocol name matching

Co-authored-by: Aaro Altonen <[email protected]>

Fix: typo

* pallet-mmr: cosmetic improvements

* pallet-mmr: fix offchain storage for initial sync

* address review comments

* pallet-mmr: change offchain fork-resistant key to `(prefix, pos, parent_hash)`

Do this so that both canon and fork-resitant keys have the same
`(prefix, pos).encode()` prefix. Might be useful in the future if we'd
be able to to "get" offchain db entries using key prefixes as well.

Signed-off-by: acatangiu <[email protected]>

Signed-off-by: acatangiu <[email protected]>

The runtime api implementation contained invalid unsafe trait bounds. `Sync` was never correct there
and `Send` should have not been "force implemented".

* Introduce `ChainSyncInterface`

`ChainSyncInterface` provides an asynchronous interface for other
subsystems to submit calls to `ChainSync`. This allows `NetworkService`
to delegate calls to `ChainSync` while still providing the same API
for other subsystems (for now). This makes it possible to move the
syncing code in piecemeal fashion out of `protocol.rs` as the calls
are just forwarded to `ChainSync`.

* Apply review comments

* Fix tests

Co-authored-by: parity-processbot <>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Minor naming improved

* BlockId removal refactor: Backend::state_at

* formatting

* pallet-mmr: RPC API works with block_numbers

* fixes

* update rpc

* fmt

* final touches in the rpc

* temporary fix

* fix

* fmt

* docs

* Update lib.rs

* use NumberFor

* validate input

* update runtime

* convert block_number to u64

* small edit

* update runtime api

* test fix

* runtime fix

* update test function

* fmt

* fix nits

* remove block_num_to_leaf_index from runtime api

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Robert Hambrock <[email protected]>

* fix tests

* get the code to compile after merge

* get the tests to compile

* fix in tests?

* fix test

* Update frame/merkle-mountain-range/src/tests.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* Update primitives/merkle-mountain-range/src/lib.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* fix errors & nits

* change block_num_to_leaf_index

* don't make any assumptions

* Update frame/merkle-mountain-range/src/tests.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* Update frame/merkle-mountain-range/src/tests.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* Update frame/merkle-mountain-range/src/tests.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* fix

* small fix

* use best_known_block_number

* best_known_block_number instead of leaves_count

* more readable?

* remove warning

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Robert Hambrock <[email protected]>

* simplify

* update docs

* nits

* fmt & fix

* merge fixes

* fix

* small fix

* docs & nit fixes

* Nit fixes

* remove leaf_indices_to_block_numbers()

* fmt

Co-authored-by: Robert Hambrock <[email protected]>
Co-authored-by: Adrian Catangiu <[email protected]>

* Introduce mockable `ChainSync` object for testing

`mockall` allows to mock `ChainSync` and to verify that the calls made
to `ChaiSync` are firstly executed at all, that they're executed in
correct order and with correct parameters.

This allows to verify, e.g., that delegating calls directly to
`ChainSync` from `NetworkService` still calls the correct functions with
correct arguments even if `Protocol` middleman is removed.

* Add Cargo.lock

* Fix tests

* Update client/network/Cargo.toml

Co-authored-by: Bastian Köcher <[email protected]>

* Update Cargo.lock

* Fix clippy and documentation

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: parity-processbot <>

* Decrease peer reputation for duplicate GRANDPA neighbor messages.

* Fix comparison

* Fix update_peer_state() validity condition

* Add negative test

* Rework update_peer_state() validity condition, add tests

* update_peer_state() validity condition: invert comparison

* Split InvalidViewChange and DuplicateNeighborMessage misbehaviors

* Enforce rate-limiting of duplicate GRANDPA neighbor packets

* Update client/finality-grandpa/src/communication/gossip.rs

Co-authored-by: André Silva <[email protected]>

* Make rolling clock back in a test safer

Co-authored-by: André Silva <[email protected]>

* dont use benchmark range on constant function

* update weights

* fix

* new weights

* Update frame/examples/basic/src/benchmarking.rs

Co-authored-by: parity-processbot <>

* Trivial BlockId::Number => Hash

* missed BlockId::Hash added

* finalized block event triggers tx maintanance

* tx-pool: enactment helper introduced

* tx-pool: ChainApi: added tree_route method

* enactment logic implemented + tests

Signed-off-by: Michal Kucharczyk <[email protected]>

* Some additional tests

* minor improvements

* trigger CI job

* fix compilation errors

ChainApi::tree_route return type changed to Result<Option<..>>, as some
implementations (tests) are not required to provide this tree route.

* formatting

* trait removed

* implementation slightly simplified

(thanks to @koute

)

* get rid of Arc<> in EnactmentState return value

* minor improvement

* Apply suggestions from code review

Co-authored-by: André Silva <[email protected]>

* Apply suggestions from code review

* comment updated + formatting

* Apply suggestions from code review

Co-authored-by: André Silva <[email protected]>
Co-authored-by: Davide Galassi <[email protected]>

* formatting

* finalization notification bug fix

+ new test case
+ log::warn message when finalized block is being retracted by new event

* added error message on tree_route failure

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* use provided tree_route in Finalized event

* Option removed from ChainApi::tree_route

* doc added, test and logs improved

* handle_enactment aligned with original implementation

* use async-await

* Apply suggestions from code review

Co-authored-by: André Silva <[email protected]>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* formatting + warn->debug

* compilation error fix

* enactment_state initializers added

* enactment_state: Option removed

* manual-seal: compilation & tests fix

* manual-seal: tests fixed

* tests cleanup

* another compilation error fixed

* TreeRoute::new added

* get rid of pub hack

* one more test added

* formatting

* TreeRoute::new doc added + formatting

* Apply suggestions from code review

Co-authored-by: Davide Galassi <[email protected]>

* (bool,Option) simplified to Option

* log message improved

* yet another review suggestions applied

* get rid of hash in handle_enactment

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/transaction-pool/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* minor corrections

* EnactmentState moved to new file

* File header corrected

* error formatting aligned with codebase

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* remove commented code

* small nits

Signed-off-by: Michal Kucharczyk <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: Davide Galassi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: André Silva <[email protected]>

* dont use unstable sort

* remove comment

* add clippy rule

* Rename `from_components` to `from_parts`

* Fixes

* Spelling

Sometimes `NotificationStreamOpenened` would be received for the
other protocol before `SyncConnected` was received so when the
connection was closed, an incorrect event was read from the event
stream.

* Remove the unused light client requests

* Add comment about new ids

* rpc/tx: Add transaction structures for serialization

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Add public facing `TransactionEvent`

To circumvent the fact that serde does not allow mixing
`#[serde(tag = "event")]` with
`#[serde(tag = "event", content = "block")]`
the public facing subscription structure is serialized
and deserialized to an intermmediate representation.

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Add trait for the `transaction` API

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Convert RPC errors to transaction events

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Implement `transaction` RPC methods

Signed-off-by: Alexandru Vasile <[email protected]>

* tx-pool: Propagate tx index to events

Signed-off-by: Alexandru Vasile <[email protected]>

* tx-pool: Adjust testing to reflect tx index in events

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Convert tx-pool events for the new RPC spec

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Convert tx-pool `FinalityTimeout` event to `Dropped`

Signed-off-by: Alexandru Vasile <[email protected]>

* service: Enable the `transaction` API

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Add tests for tx event encoding and decoding

Signed-off-by: Alexandru Vasile <[email protected]>

* tx: Add indentation for subscriptions

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Fix documentation

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Serialize usize to hex

Signed-off-by: Alexandru Vasile <[email protected]>

* tx-pool: Rename closure parameters

Signed-off-by: Alexandru Vasile <[email protected]>

* service: Separate RPC spec versions

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Use `H256` for testing block's hash

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Serialize numbers as string

Signed-off-by: Alexandru Vasile <[email protected]>

* tx-pool: Backward compatibility with RPC v1

Signed-off-by: Alexandru Vasile <[email protected]>

* Update client/rpc-spec-v2/src/transaction/transaction.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* rpc/tx: Remove comment about serde clone

Signed-off-by: Alexandru Vasile <[email protected]>

* rpc/tx: Use RPC custom error code for invalid tx format

Signed-off-by: Alexandru Vasile <[email protected]>

* Update client/rpc-spec-v2/src/transaction/event.rs

Co-authored-by: James Wilson <[email protected]>

* rpc/tx: Adjust internal structures for serialization/deserialization

Signed-off-by: Alexandru Vasile <[email protected]>

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>
Co-authored-by: James Wilson <[email protected]>

* Update UI tests for Rust 1.64

* Test with the staging image

* Switch back to production

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Move Role(s) to `sc-network-common`

* Introduce `NotificationHandshake` type

* Move block announce protocol config creation to `ChainSync`

* Include block announcement into `notification_protocols`

* Apply review comments

* Remove unneeded include

* Add missing include

* Apply review comments

* Vesting pallet - make WithdrawReasons configurable

* Update `pallet-vesting` README

Co-authored-by: parity-processbot <>

* Remove "to_block" field from BlockRequests

* Maybe fix the tests

* Define CustomVerify trait

Signed-off-by: Serban Iorga <[email protected]>

* Use ECDSA CustomVerify for MultiSignature

Signed-off-by: Serban Iorga <[email protected]>

* beefy: small simplifications

Signed-off-by: Serban Iorga <[email protected]>

* Revert "Use ECDSA CustomVerify for MultiSignature"

This reverts commit 136cff82505662dd92c864491814629d2bc349f0.

* Revert "Define CustomVerify trait"

This reverts commit adf91e9e6d1bdea6f00831f6067b74c3d945f9a2.

* Define BeefyAuthorityId and BeefyVerify traits

* Improve BeefyVerify unit tests

Co-authored-by: Robert Hambrock <[email protected]>

* fmt & import sp_core::blake2_256

* Renamings

* remove SignerToAccountId

* fix

Signed-off-by: Serban Iorga <[email protected]>
Co-authored-by: Robert Hambrock <[email protected]>

* Extend the lower bounds of some of the benchmarks to also include `0`

* Fix verify snippet for `pallet_bounties/spend_funds`

This fixes some warnings on latest nightly.

macro stubs for all pallet:: macros to improve documentation visibility and discovery + revamp of pallet macro documentation (#12334)

* proof of concept working for pallet::whitelist_storage

* fix comments

* pallet macros docs rewrite WIP

* fix issue with cargo fmt cobbling links

* tweak capitalization

* fix docs for storage_version

* fix docs for pallet::hooks

* fix several comments

* fix invalid link

* fix wrapping and add missing links for pallet::hooks docs

* run rewrap on all text blocks in frame_support::pallet docs

* cargo fmt

* fix up pallet::call_index docs

* fix docs for pallet::extra_constants

* fix docs for pallet::error

* fix docs for pallet::event

* fix docs for pallet::event

* * fix docs for pallet::storage
* fix docs for pallet::getter
* fix docs for pallet::storage_prefix
* fix docs for pallet::unbounded
* fix docs for pallet::whitelist_storage
* fix docs for #[cfg(..)] (for storage items and attributes)
* fix docs for pallet::storage macro expansion

* fix docs for pallet::type_value

* fix docs for pallet::genesis_config

* fix docs for pallet::genesis_build

* fix docs for pallet::inherent

* fix docs for pallet::validate_unsigned

* fix docs for pallet::origin

* fix docs for general notes on instantiable pallets

* fix docs for example of a non-instantiable pallet

* fix docs for example of an instantiable pallet

* fix docs for upgrade guidelines

* fix docs for upgrade guidelines

* fix docs for upgrade checking and final notes

* fix some examples near the beginning

* extract docs for `pallet::whitelist_storage`

* add docs for pallet_macro_stub

* fix order of pallet::config and pallet::constant

* set up stub for pallet::config

* set up stub for pallet::constant

* fix

* set up stub for pallet::disable_frame_system_supertrait_check

* set up stub for pallet::generate_storage_info

* set up stub for pallet::storage_version

* set up stub for pallet::hooks

* set up stub for pallet::weight

* set up stub for pallet::compact

* set up stub for pallet::call_index

* set up stub for pallet::extra_constants

* set up stub for pallet::error

* set up stub for pallet::event

* set up stub for pallet::generate_deposit

* set up stub for pallet::storage

* set up stub for pallet::getter

* set up stub for pallet::storage_prefix

* set up stub for pallet::unbounded

* set up stub for pallet::type_value

* set up stub for pallet::genesis_config

* set up stub for pallet::genesis_build

* set up stub for pallet::inherent

* set up stub for pallet::validate_unsigned

* set up stub for pallet::origin

* fix comment

* cargo fmt

* tweak error message

* Update frame/support/procedural/src/lib.rs

Co-authored-by: Keith Yeung <[email protected]>

* Update frame/support/procedural/src/lib.rs

Co-authored-by: Keith Yeung <[email protected]>

* switch order of derives

Co-authored-by: Squirrel <[email protected]>

* tweak wording

Co-authored-by: Squirrel <[email protected]>

* add more context info about `MAX_MODULE_ERROR_ENCODED_SIZE`

Co-authored-by: Squirrel <[email protected]>

* tweak wording about where clause

Co-authored-by: Squirrel <[email protected]>

* clarify wording about system/events key

Co-authored-by: Squirrel <[email protected]>

* use "The Event enum" instead of "item"

Co-authored-by: Squirrel <[email protected]>

* fix bad wording

Co-authored-by: Squirrel <[email protected]>

* use enum instead of type

Co-authored-by: Squirrel <[email protected]>

* expect => expects

Co-authored-by: Squirrel <[email protected]>

* add additional note about storage prefix

Co-authored-by: Squirrel <[email protected]>

* clearer note about GenesisConfig

Co-authored-by: Squirrel <[email protected]>

* Use "The impl" instead of "The item"

Co-authored-by: Squirrel <[email protected]>

* add note and link to tight-coupling docs

Co-authored-by: Squirrel <[email protected]>

* cargo fmt

* remove spaces around parenthesis

* fix missing text for pallet::config

* fix issue with pallet::constant intro

* fix wording about codec index

* fix pallet::error wording

* fix comment about 1 byte => 256 errors

* fix where clause comment

* fix comment about where pallet events are stored

* rewrap some text

* fix pallet::storage docs

* fix pallet::storage_prefix docs

* tweak docs for pallet::genesis_build

* tweak docs for pallet::config

* specify that pallet::event must be present if pallet::config is present

* add note about why we would want to bypass the supertrait check

* mention that pallet::generate_store attribute is only valid on pallet struct

* add note about adding new calls to the end to maintain existing order

* add note about pallet::type_value and pallet::storage

Co-authored-by: Squirrel <[email protected]>

* add note about using pallet::type_value alongside pallet::storage

* include warning about modifying disaptchables on other pallet::call_index docs page

* fix incorrect comment

* add much more information for pallet::inherent

* move pallet::pallet macro expansion notes back to their rightful place

* re-run CI

* fix macro expansion appearing in wrong place for pallet::pallet

* replicate pallet::pallet docs on the pallet::pallet macro stub

* force CI re-run

Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Squirrel <[email protected]>
Co-authored-by: parity-processbot <>