* store block number ++ state root in parachains pallet

* fixed parachains finality APIs

* (test commit)

* removed test code

* deduplicated code a bit

* removed commented code

* spelling

* Update modules/parachains/src/lib.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* Update modules/parachains/src/lib.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* Update modules/parachains/src/mock.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* added comment

Co-authored-by: Adrian Catangiu <[email protected]>

* Replace const parameters

* fmt

* missed out Maxlocks

* update Substrate + Polkadot + Cumulus refs

* Origin -> RuntimeOrigin

* weights v1.5

* update refs once again + `cargo test -p pallet-bridge-grandpa` works

* started work on `cargo test -p pallet-bridge-messages`

* cargo test -p pallet-bridge-relayers

* cargo test -p pallet-bridge-parachains

* cargo test -p millau-runtime

* cargo test -p bridge-runtime-common

* cargo test -p rialto-runtime

* cargo test -p rialto-parachain-runtime

* cargo test -p millau-bridge-node

* cargo test -p rialto-bridge-node

* cargo test -p rialto-parachain-collator

* cargo test -p messages-relay

* cargo test -p parachains-relay

* cargo test -p substrate-relay

* cargo test --all

* cargo check -p millau-runtime --locked --features runtime-benchmarks

* fix remaining test

* fmt

* try to allow clippy failure temporarily

* Revert "try to allow clippy failure temporarily"

This reverts commit d1b65935.

* use min_by

* Revert "use min_by"

This reverts commit 33042f49.

* Revert "Revert "use min_by""

This reverts commit 1d2204f0.

* trigger CI

* Revert "trigger CI"

This reverts commit 259d91b5.

* new day, new clippy warning

* more clippy issues

* remove without_storage_info from pallet-bridge-parachains

* fix benchmarks

* remove pallet::without_storage_info from bridge GRANDPA pallet

* StoredBridgedHeader

* spelling

* fix benchmarks

* MAX_BRIDGED_AUTHORITIES: 256 -> 2048

* Update modules/grandpa/src/storage_types.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* Update modules/grandpa/src/storage_types.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* moved max authorities + header size to chain primitives

* removed unused code

* new -> try_new

* fix benchmarks compilation

Co-authored-by: Adrian Catangiu <[email protected]>

* add events to the bridge parachains pallet

* clippy + spellcheck

* fix compilation

* untracked is not a word?

* remove duplicate parachain heads exension

* fix benchmarks compilation

* actually fix it

* added parachains pallet benchmarks

* deduplicate insertion of bridged header

* pruning weight

* fixes

* fix compilation

* added trackerd parachains filter + changed some docs

* fix compilation

* parachains finality

* parachains pallet test

* demo of how to configure GRANDPA pallet instance

* allow instances in parachains pallet

* spellcheck

* TODO + fix

* fmt

* removed invalid storage_keys file

* change all hashers to Blake2_128Concat

* use Twox64Concat for insertion position

* fix build

* fix compilation

* change ParaId and ParaHead types

* TODOs -> TODOs with issues refs

Substrate: 31d90c202d6df9ce3837ee55587b604619a912ba
Polkadot: 60df3c55c711c2872872d6220f98b2611340e051
Cumulus: a9630551c2cd877952ab769c862af4c81b0ccd3c

* refactor message relay helpers

* single standalone_metrics function

* fixed tests

* clippy + fmt

* removed commented code

* add calls tracing

* fix spelling

* cargo fmt

* -commented code

* fix build again

* post-merge build fix

* clippy + fmt

* substrate: d0f6c1c60da22e04dd25c2eca46ebfe6f1571af0
polkadot: dd4b2e6a34a08a01b876d14641e99e7011be3463
cumulus: 9379cd6c1863ea846ad6c6a8cbbc99848dd5d693

* fmt

* fixed lost refs

* spelling

* benckhmarks

* fmt

* Substrate: 63b32fbaa2764c0a8ee76b70cdfa0fcb59b7181f
Polkadot:  7229ab87acf5bc5d4d10655ad1a9819a1e317442
Cumulus:   d5284b0e78be3a18a3371103dd00449aceb082df

* rialto parachain runtime

* fixed tests

* add node + bump refs (not compiling yet):
Substrate:           630422d6108cbaaca893ab213dde69f3bdaa1f6b
Polkadot:            7229ab87acf5bc5d4d10655ad1a9819a1e317442
Cumulus:             5af2990cfd3711f86c7cb8606b22364208a5c32d
GrandpaBridgeGadget: c152c45ac331eb8ab40d956ab1d008d181810ef4

* fix compilation (collator is not working)

* more fixes

* fmt

* spellcheck

* fix warnings

* fix compilation

* fmt

* trigger CI

* trigger CI

* Revert "trigger CI"

This reverts commit a31f53ce.

* benchmarks

* fix benchmarks

* fix again

* Revert "Revert "trigger CI""

This reverts commit 1dea8b42.

* Revert "Revert "Revert "trigger CI"""

This reverts commit 8fb74fa5.

* try fix

* lost lock file

* spellcheck

* try to disable sccache for cargo check

* Revert "Revert "Revert "Revert "trigger CI""""

This reverts commit f1574614.

* try to disable again

* disable sccache for test

* move some associated types from relay_substrate_client::Chain to bp_runtime::Chain

* dummy commit

* Revert "dummy commit"

This reverts commit 81bc64aa.

* Pruning.

* Add tests.

* Address review.

* Bump Substrate

* Change usage of "Module" to "Pallet"

Related Substrate PR: https://github.com/paritytech/substrate/pull/8372

* Add `OnSetCode` config param

Related Substrate PR: https://github.com/paritytech/substrate/pull/8496

* Update Aura Slot duration time type

Related Substrate PR: https://github.com/paritytech/substrate/pull/8386

* Add `OnSetCode` to mock runtimes

* Add support for multiple justifications

Related Substrate PR: https://github.com/paritytech/substrate/pull/7640

* Use updated justification type in more places

* Make GenesisConfig type non-optional

Related Substrate PR: https://github.com/paritytech/substrate/pull/8275

* Update service to use updated telemetry

Related Substrate PR: https://github.com/paritytech/substrate/pull/8143

* Appease Clippy

* Use more accurate weight calculation in declared weight

* Remove session length and validator set size config constants

* Remove config params from mock

* Allow specifying total number of votes-ancestries per justification

* Change limits used during benchmarking

* Regenerate weights

* Use simplified weight annotation

* Remove comment

* Address leftover TODO

* Prevent possible divide by zero errors

* Use correct argument order in weight declaration

* Add benchmarking skeleton

* Allow runtime to indicate concrete header type for benches

* Set up skeleton for benchmark tests

* Play around with mutating bench header

* Create a working test for benchmarking

* Add benches related to enacting authority set changes

* Add bench for checking effect of prec-commits/vote ancestries

* Use new `no_std` test utils in benchmarks

* Support pallet instances in benchmarking

* Use correct benchmarking instance macro

* Add instance to runtime benchmark helper impl

* Start using new justification creation API

* Allow mock header's number to be specified

* Set up benches with correct fork/depth parameters

* Use new pallet name during runtime bench setup

* Use correct `set_id` in tests

* Limit number of forks as workaround to get tests passing

* Use number of authorities which matches number of forks

* Make sure test post-conditions are checked properly

* Only read `Current...

* Rename `pallet-finality-verifier` to `pallet-bridge-grandpa`

* Missed some CamelCase ones

* Update logging target in GRANDPA pallet

* Rename `pallet-bridge-call-dispatch` to `pallet-bridge-dispatch`

* Rename the dispatch pallet folder

* Update logging target in Dispatch pallet

* Missed a couple

* Format the repo

* Stop listing individual pallets in Compose logs

* Use correct pallet name in module doc comments

* Add `pallet-bridge-dispatch` to README project layout

* Sort crate names in TOML files

* Rename `pallet-bridge-grandpa` runtime Call alias

* make finality verifier pallet instantiable

* try to fix benchmarks

* fix benchmarks compilation

* Remove unused Config types from `pallet-finality-verifier`

* Remove unused AncestryChecker trait

* Remove ancestry proof parameter from relayer calls

* Update docs to reflect current state of pallet

* Remove mock ancestry checker

* Remove unused error

* Write headers outside of function used for authority set changes

* Move justification verification into helper function

* Add documentation suggestions

Co-authored-by: Tomasz Drwięga <[email protected]>

* Clean up module level documentation a bit

Co-authored-by: Tomasz Drwięga <[email protected]>

* Add relevant storage items from Substrate pallet

* Add function for importing finalized headers to storage

* Remove unused getter functions

* Add GenesisConfig to pallet

* Add initialization extrinsic

* Add operational extrinsic

* Get existing finality verifier tests compiling again

* Add tests for pallet initialization

* Add tests related to pallet's operational status

* Update tests which were using `pallet-substrate-bridge`

* Add tests related to header imports

* Use wrapper function when init-ing some tests

* Add prefix to tests related to rate limiter

* Fix failed compilation related to GenesisConfig

* Add some documentation

* Change some extrinsics to be Operational

* Add public interface to pallet

* Implement runtime APIs for finality-verifier pallet

* Justify use of `expect` when importing headers

* Reject headers with forced changes

* Add weight to initialize extrinsic

* Remove TODO which will be addressed later

* Move succesful import log to correct location

* Expand proof for when `best_finalized` is fetched

* Move check for newer finalized blocks earlier in pipeline

* Rename `ConflictingFork` error to be more generic

* Only compute finality_target's hash once

* Add missing documentation to Runtime APIs

* Add TODO about using `set_id` from `ScheduledChange` digest

* Revert "Pin Rust Nightly to 2020-12-17 (#652)"

This reverts commit e54e6f7e.

* fix clippy

* clippy again

* more clippy in test code

* and new cargo fmt

* another try

* Bump substrate deps.

* Bump scale.

* Bump ethabi.

* Remove inherent from aura.

* Migrate to construct_runtime

* cargo fmt --all

* Add simple rate limiting mechanism

* Add tests

* Small test cleanup

* Hook MaxRequests into runtimes

* Make AncestryProof type more flexible

* Only import single finalized header instead of a chain

* Fix unchecked header import tests

* Add option for limiting ancestry proof size

* Update finality verifier Config in runtimes

* Update some documentation

* Fix Clippy warning

* Allow AncestryChecker to return proof size

Stops us from abusing the `Size` trait

* Remove Size impl for Vec<T>

* Remove size contraints for ancestry proofs

With different proof types its unclear how to "size" should be interpreted,
so we remove this requirement all together to avoid confusion.

* Get pallet compiling with FRAME v2

* Get tests compiling

* Stop printing metadata in tests

* Remove more metadata related code

* Remove unecessary storage attribute

* Add skeleton for `pallet-finality-verifier`

* Sketch out implementation for importing finality proofs

* Get pallet compiling

* Introduce skeleton for mock runtime

* Start using real Grandpa types in finality pallet

* Redefine types in header chain primitives crate

* Implement HeaderChain for Substrate bridge pallet

* Plug Substrate Bridge Pallet into verifier mock

* Fix compilation of `header-chain` primitives

* Start writing to base pallet storage

* Add first "cross-pallet" test

* Move keyring primitives used in tests to shared crate

* Stop pulling `std` deps into `no_std` builds

* Revert "Stop pulling `std` deps into `no_std` builds"

This reverts commit f74dd660.

* Revert "Move keyring primitives used in tests to shared crate"

This reverts commit b774fa73.

* Use new SS58Prefix type in mock

* Start using `bp-test-utils` in finality pallet

* Start using real justification code

* Get a test working with real justification verification

* Add basic tests for invalid proofs

* Get rid of AncestryProof config type

* Add error types to transaction outcome

* Bound number of headers allowed in a single ancestry proof

* Disallow invalid authority sets

* Remove unused items

* Add some documentation

* Get rid of Clippy warnings

* Rename BaseHeaderChain to TransactionVerifier

* Remove unused code

* Make dummy trait implementations more generic

* Fix more Clippy complaints

* Update tests to use fix for duplicate headers

* Fix benchmarking compilation

* Rename TransactionVerifier to InclusionProofVerifier

* Move justification module to header-chain primitives crate

* Get justification module compiling in new location

* Get justification module tests compiling

* Use justification code from `header-chain` crate

Mostly compiles, having issues with std/test feature flags across crates.

* Move some code around

* Move justification tests to integration testing crate

* Add `test-utils` crate

* Remove tests and test-helper module from justification code

* Use `test-utils` in Substrate bridge pallet tests

* Remove `sp-keyring` related code from `pallet-substrate-bridge`

* Remove `helpers` module from `pallet-substrate-bridge`

* Add some documentation

* Add more documentation

* Fix typo

Co-authored-by: Tomasz Drwięga <[email protected]>

Co-authored-by: Tomasz Drwięga <[email protected]>

* Update `sp-io` dependency

* Rename Trait to Config

* RustFmt

* Bump `sp-io` again

* Use new frame_system weight types in Rialto and Millau runtimes

* Update test Runtimes to use new weight types

* Bump `sp-io` again

* Update to not-the latest first.

* Update benchmarks.

* Another Trai.

* Move new weight types into runtime primitive crates

This allows us to check limits for extrinsics from other parts
of the codebase without pulling in the entire chain runtime.

* Remove leftover comments

* Move new functions to a better location

* Small formatting fixes

* Add actual documentation to new weight config types

* Decrease maximum block weight of Millau chain

* Decreease maximum block length of Millau chain

Co-authored-by: Tomasz Drwięga <[email protected]>

* parse substrate message proof

* unfinalized_header

* extract ChainBase to bp-runtime

* post-merge fixes

* cargo fmt --all

* compilation fixes

* reexport BlockNumberOf, HashOf, HeaderOf

* Remove the Substrate primitives crate

The types here were only used in one place, the pallet itself. If other
components start using these types we can considering moving them back
into a standalone crate.

* Start trying to integrate justification module

* Make Substrate blocks configurable in Pallet

* WIP: Try and generalize justification test helpers

* Fix tests which use "real" justifications

* Put common test helpers alongside mock code

* Use common helper for creating headers

* Remove usage of UintAuthorityId

This change favours the use of the Ed25519Keyring authorities
in order to keep things consistent with the tests.

* Add documentation around config trait types

* Make test header, hash, and number types consistent

* Update modules/substrate/src/verifier.rs

Co-authored-by: Svyatoslav Nikolsky <[email protected]>

* Update modules/substrate/src/lib.rs

Co-authored-by: Tomasz Drwięga <[email protected]>

* Update modules/substrate/Cargo.toml

Co-authored-by: Svyatoslav Nikolsky <[email protected]>

* Derive `RuntimeDebug` instead of `Debug`

* Add `Paramter` as a trait constraint on config types

Since we use these types as part of the dispatchable functions
we should explicitly require this.

* Enforce that hasher output matches expected hash type

* Accept headers over indexes when making test justifications

* Check that authority sets are valid

* Make Clippy happy

* Apply correct Clippy fix

* Move justification code into primitives module

* Use new module in verifier code

* Add primitives module for Substrate test helpers

* WIP

* Move justification generation into test_helpers

* Revert commits which move `justification` into primitives

This reverts commit 03a381f0

.

Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>

* Change script to update versions.

* Bump versions.

* Address remainders.

* cargo fmt --all

* Fix tests.

* Whitelist BlueOak license

* Fix benchmarks?

* Add pallet template from Substrate Dev Hub

* Clean up un-needed stuff from template

* Sketch out dispatchable interface

* Introduce notion of finality chain

* Add dependencies which were removed during a rebase

* Sketch out idea for finality header-chain pallet

* Sketch out ChainVerifier trait

* Add storage parameter to verifier

* Write out some things I think I need for finality verification

* Add some pseudocode for marking finalized headers

* Remove parity_scale_codec duplicate

* Move verification logic into pallet

I've been struggling with getting the generic types between the storage and verifier
traits to play nice with each other. As a way to continue making progress I'm moving
everything to the pallet. This way I hope to make progress towards a functional
pallet.

* Start doing verification around authority set changes

* Remove commented BridgeStorage and ChainVerifier traits

* Create Substrate bridge primitives crate

* Add logic for updating scheduled authority sets

* Introduce notion of imported headers

* Implement basic header ancestry checker

* Add mock runtime for tests

* Add testing boilerplate

* Add some storage read/write sanity tests

* Add some basic header import tests

* Add tests for ancestry proofs

* Create helper for changing authority sets

* Fix authority set test

Fixes a problem with how the scheduled change was counted as well as
a SCALE encoding issue

* Correctly check for scheduled change digests

There's no guarantee that the consensus digest item will be the last
one in a header, which is how it was previously being checked.

Thanks to Andre for pointing me to the Grandpa code that does this.

* Mark imported headers as finalized when appropriate

When a header that finalizes a chain of headers is succesfully imported
we also want to mark its ancestors as finalized.

* Add helper for writing test headers

* Add test helper for scheduling authority set changes

* Bump Substrate pallet and primitives to rc6

* Remove Millau verifier implementation

* Add some doc comments

* Remove some needless returns

* Make Clippy happy

* Split block import from finalization

* Make tests compile again

* Add test for finalizing header after importing children

* Create a test stub for importing future justifications

* Start adding genesis config

* Reject justifications from future

We should only be accepting justifications for the header
which enacted the current authority set. Any ancestors of
that header which require a justification can be imported
but they must not be finalized.

* Add explanation to some `expect()` calls

* Start adding GenesisConfig

* Plug genesis config into runtime

* Remove tests module

* Check for overflow when updating authority sets

* Make verifier take ownership of headers during import

* Only store best finalized header hash

Removed the need to store the whole header, since we store
it was part of the ImportedHeaders structure anyways

* Add some helpers to ImportedHeader

* Update ancestry checker to work with ImportedHeaders

* Update ancestry tests to use ImportedHeaders

* Update import tests to use ImportedHeaders

* Clean up some of the test helpers

* Remove stray dbg!

* Add doc comments throughout

* Remove runtime related code

* Fix Clippy warnings

* Remove trait bound on ImportedHeader struct

* Simplify checks in GenesisConfig

* Rename `get_header_by_hash()`

* Alias `parity_scale_codec` to `codec`

* Reword Verifier documentation

* Missed codec rename in tests

* Split ImportError into FinalizationError

* Remove ChainVerifier trait

This trait was a remenant of the original design, and it is not required
at the moment. Something like it should be added back in the future to
ensure that other chains which conform to this interface can be used
by higher-level bridge applications.

* Fix the verifier tests so they compile

* Implement Deref for ImportedHeader

* Get rid of `new` methods for some Substrate primitives

* Ensure that a child header's number follows its parent's

* Prevent ancestry checker from aimlessly traversing to genesis

If an ancestor which was newer than the child header we were checking we
would walk all the way to genesis before realizing that we weren't related.
This commit fixes that.

* Remove redundant clones

* Ensure that old headers are not finalized

Prevents a panic where if the header being imported and `best_finalized`
were the same header the ancestry checker would return an empty list. We
had made an assumption that the list would always be populated, and if this
didn't hold we would end up panicking.

* Disallow imports at same height as `best_finalized`

* Fix Clippy warnings

* Make NextScheduledChange optional

* Rework how scheduled authority set changes are enacted

We now require a justification for headers which _enact_ changes
instead of those which _schedule_ changes. A few changes had to
be made to accomodate this, such as changing when we check for
scheduled change logs in incoming headers.

* Update documentation for Substrate Primitives

* Clarify why we skip header in requires_justification check

* Add description to assert! call

* Fix formatting within macros

* Remove unused dependencies from runtime

* Remove expect call in GenesisConfig

* Turn FinalityProof into a struct

* Add some inline TODOs for follow up PRs

* Remove test which enacted multiple changes

This should be added back at some later point in time, but right now
the code doesn't allow for this behaviour.

* Use `contains_key` when checking for header

This is better than using `get().is_some()` since we skip
decoding the storage value

* Use initial hash when updating best_finalized

* Add better checks around enacting scheduled changes

* Rename finality related functions

* Appease Clippy