* maintain MillauHeadersToRialto sync by subscribing to Millau justifications

* more tracing in maintain

* Update relays/substrate/src/headers_maintain.rs

Co-authored-by: Hernando Castano <[email protected]>

* Update relays/substrate/src/headers_maintain.rs

Co-authored-by: Hernando Castano <[email protected]>

* -Please

* -TODO

* revert raise recursion limit

* updated comment

Co-authored-by: Hernando Castano <[email protected]>

* millau -> rialto lanes integration

* extrace common message-lane integration types into bridge-runtime-common

* rialto_messages.rs in Millau runtime

* tests

* Update bin/rialto/runtime/src/millau_messages.rs

Co-authored-by: Hernando Castano <[email protected]>

* RELAYER_INTEREST_PERCENT -> RELAYER_FEE_PERCENT

* Update bin/runtime-common/src/messages.rs

Co-authored-by: Hernando Castano <[email protected]>

* estimate_message_dispatch_and_delivery_fee returns Result

* Update bin/rialto/runtime/src/millau_messages.rs

Co-authored-by: Hernando Castano <[email protected]>

* Update bin/rialto/runtime/src/millau_messages.rs

Co-authored-by: Hernando Castano <[email protected]>

* Update bin/rialto/runtime/src/millau_messages.rs

Co-authored-by: Hernando Castano <[email protected]>

* fmt

* mowed weight formula to primitives

Co-authored-by: Hernando Castano <[email protected]>

* InboundLaneApi::latest_confirmed_nonce

* Fix wording and typo

Co-authored-by: Hernando Castano <[email protected]>

* reward relayers for dispatching messages

* clippy

* Update modules/message-lane/src/lib.rs

Co-authored-by: Hernando Castano <[email protected]>

* added comment

* Update modules/message-lane/src/inbound_lane.rs

Co-authored-by: Tomasz Drwięga <[email protected]>

* Update modules/message-lane/src/inbound_lane.rs

Co-authored-by: Tomasz Drwięga <[email protected]>

* SubmitterId + RelayerId -> AccountId

* add confirmation_relayer arg to pay_relayer_reward

* cargo fmt --all

* removed verify_and_decode_messages_proof from SourceHeaderChain

* &mut self -> RefCell

* Optimize max messages at inbound lane (#418)

* Add tests for checking messages above max limit

Signed-off-by: MaciejBaj <[email protected]>

* Extend the relayers entry of inbound lane by additional msg nonce

Signed-off-by: MaciejBaj <[email protected]>

* Support additional message nonce from inbound relayers

Signed-off-by: MaciejBaj <[email protected]>

* Code format

Signed-off-by: MaciejBaj <[email protected]>

* Merge messages range for highest relayers

* Change unwrap() to ensure() while accessing relayers

* Edit rustdocs for relayers deque at inbound lane data

Co-authored-by: Hernando Castano <[email protected]>

* Declare additional relayers A & B and use across tests consistently

Signed-off-by: MaciejBaj <[email protected]>

* Remove duplicates and improve naming for inbound lane tests

* Fix test checking max limit per inbound lane

* Correct relayers rewards loop after a proof is received

* Remove redundant check for messages ahead of received range

* Correct grammar at inbound lane tests rustdocs

Co-authored-by: Hernando Castano <[email protected]>

* Improve code quality of relayers updates 💅



Co-authored-by: Tomasz Drwięga <[email protected]>

* Test dispatches above max limit from same relayer

Co-authored-by: Hernando Castano <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>

* Fix typo.

Co-authored-by: Hernando Castano <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Maciej Baj <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>

* working-millau-to-rialto-relay

* fix Millau state root

* properly fix synced+incomplete headers for sub2sub

* removed wrong TODO

* fmt + clippy

* Update relays/headers-relay/src/headers.rs

Co-authored-by: Hernando Castano <[email protected]>

* moved SubstrateTransactionMaker definition

* removed M from phantomdata

* removed prune_synced_children

* methods names as consts

Co-authored-by: Hernando Castano <[email protected]>

* add HeaderTimestamp associated type

* use Header Timestamp

* rename HeaderTimestamp to ChainTime

* add unit test

* deal with clippy

* Apply suggestions from code review

Commit review suggestions

Co-authored-by: Tomasz Drwięga <[email protected]>

* code review

* cargo fmt

* get rid of additional test runtime

* unit test asserts against concrete import context

Co-authored-by: Tomasz Drwięga <[email protected]>

* expose GRANDPA RPC API from Millau node

* fmt

* Add Rialto as a target for the Millau node

* Rename Rialto module to Rialto-PoA

This will reduce confusion as the Millau runtime has a
Rialto module as well which refers to the Substrate chain.

* Add Millau as a target for the Rialto node

* Be more explicit about Rialto PoA related code

* Missed some name changes in the Ethereum PoA relay

* Re-export Substrate pallet structs used by node

* Remove `first_scheduled_change` of Millau in Rialto node

* Make Millau's genesis config for Rialto bridge more accurate

* Set initial header for Millau config

* Update initial Millau authorities

Co-authored-by: Svyatoslav Nikolsky <[email protected]>

* RustFmt Millau authorities

Co-authored-by: Svyatoslav Nikolsky <[email protected]>

* extract ChainBase to bp-runtime

* post-merge fixes

* cargo fmt --all

* compilation fixes

* reexport BlockNumberOf, HashOf, HeaderOf

* Implement public helpers for querying header info

* Update `best_header` when importing headers

* Add BestHeader to GenesisConfig

* Define extra types for Millau primitives

* Start implementing runtime APIs in Millau runtime

* Add helper for getting headers which require a justification

* Add runtime API for getting headers requiring a justification

* Reword `expect()` proof for valid authority sets

* Fix typo

* Clean up Hasher comment

* Add the Call Dispatch Pallet back to the Millau runtime

* Use types from Rialto in bridge pallet config

* Use the Rialto runtime APIS in the Millau runtime

* Include Millau bridge instance in Rialto runtime

* Add missing doc comment

* Use one storage function for setting and clearing `RequiresJustification`

* Remove TODO comments

* message lane metrics

* fmt and clippy

* Split the Rialto Node into a standalone crate

* Split the Millau Node into a standalone crate

* Remove `bridge-node` crate

* Add benchmarking feature to nodes

* Check that benchmarks compile in CI

* Remove nodes from CI matrix

* Update Rialto node to Substrate 2.0

* Get Millau node compiling with Substrate 2.0

* Remove extra license text

* Remove comments in TOML files

* Move nodes and runtimes into Rialto and Millau folders

* Fix dependency paths

* Remove unused script

* Add issue to benchmarking TODO

* Fix benchmark manifest paths

* get Substrate dependencies from crates.io

* removing unused dependencies

* cargo fmt --all

* remove commented dependencies

* remove commented dependencies again

* try to fix compilation

* substrate-relay: initial commit

* MillauHeaderApi and RialtoHeaderApi

* post-merge fixes + TODOs + compilation

* Remove the Substrate primitives crate

The types here were only used in one place, the pallet itself. If other
components start using these types we can considering moving them back
into a standalone crate.

* Start trying to integrate justification module

* Make Substrate blocks configurable in Pallet

* WIP: Try and generalize justification test helpers

* Fix tests which use "real" justifications

* Put common test helpers alongside mock code

* Use common helper for creating headers

* Remove usage of UintAuthorityId

This change favours the use of the Ed25519Keyring authorities
in order to keep things consistent with the tests.

* Add documentation around config trait types

* Make test header, hash, and number types consistent

* Update modules/substrate/src/verifier.rs

Co-authored-by: Svyatoslav Nikolsky <[email protected]>

* Update modules/substrate/src/lib.rs

Co-authored-by: Tomasz Drwięga <[email protected]>

* Update modules/substrate/Cargo.toml

Co-authored-by: Svyatoslav Nikolsky <[email protected]>

* Derive `RuntimeDebug` instead of `Debug`

* Add `Paramter` as a trait constraint on config types

Since we use these types as part of the dispatchable functions
we should explicitly require this.

* Enforce that hasher output matches expected hash type

* Accept headers over indexes when making test justifications

* Check that authority sets are valid

* Make Clippy happy

* Apply correct Clippy fix

* Move justification code into primitives module

* Use new module in verifier code

* Add primitives module for Substrate test helpers

* WIP

* Move justification generation into test_helpers

* Revert commits which move `justification` into primitives

This reverts commit 03a381f0bc4a8dbe4785c30d42ab252a06ba876c.

Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>

* extracting sub clients

* fmt + lost docs

* revert enum BridgeInstance

* apply suggestions from review

* explicite debug impl

* remove unused imports from Millau

* fix typo

* fix instance + API name

* Update relays/ethereum/src/ethereum_sync_loop.rs

Co-authored-by: Hernando Castano <[email protected]>

* separate crates for millau and rialto client

* cargo fmt

* fix

* fmt

* remove no_std support

* fix compilation again

* Update relays/substrate-client/Cargo.toml

* Update relay clients to Substrate 2.0

Co-authored-by: Hernando Castano <[email protected]>
Co-authored-by: Hernando Castano <[email protected]>

* split payloads (inbound/outbound/opaque/dispatch) + fees (inbound/outbound)

* added tests

Co-authored-by: Hernando Castano <[email protected]>

* Change script to update versions.

* Bump versions.

* Address remainders.

* cargo fmt --all

* Fix tests.

* Whitelist BlueOak license

* Fix benchmarks?

* introduce BridgedHeaderChain trait

* LaneMessageVerifier + tests

* fixed tests

* do not expose intenal functions

* cargo fmt --all + fix no_std compilation

* ByWeightDispatcher

* process queued messages from message-lane::on_initialize

* scheduled_messages_are_processed_from_on_initialize

* flush

* deal with fees + weights

* drop heavy messages on dispatch

* cargo fmt

* clippy

* fix comment

* Update primitives/message-lane/src/source_chain.rs

Co-authored-by: Tomasz Drwięga <[email protected]>

* removed messages_processed

* Update primitives/message-lane/src/source_chain.rs

Co-authored-by: Hernando Castano <[email protected]>

* Update modules/message-lane/src/lib.rs

Co-authored-by: Hernando Castano <[email protected]>

* remove queueing from message-lane

* also remove queueing from RPCs

* remove by-weight traces

* dispatch fee

* receiving -> delivery

* receival -> delivery

* remove extra line

* Update primitives/message-lane/src/source_chain.rs

Co-authored-by: Hernando Castano <[email protected]>

* cargo fmt --all

* clippy

* let dispatch_weight to be larger than actual_dispatch_weight

* post-merge fix

Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Hernando Castano <[email protected]>

* Add pallet template from Substrate Dev Hub

* Clean up un-needed stuff from template

* Sketch out dispatchable interface

* Introduce notion of finality chain

* Add dependencies which were removed during a rebase

* Sketch out idea for finality header-chain pallet

* Sketch out ChainVerifier trait

* Add storage parameter to verifier

* Write out some things I think I need for finality verification

* Add some pseudocode for marking finalized headers

* Remove parity_scale_codec duplicate

* Move verification logic into pallet

I've been struggling with getting the generic types between the storage and verifier
traits to play nice with each other. As a way to continue making progress I'm moving
everything to the pallet. This way I hope to make progress towards a functional
pallet.

* Start doing verification around authority set changes

* Remove commented BridgeStorage and ChainVerifier traits

* Create Substrate bridge primitives crate

* Add logic for updating scheduled authority sets

* Introduce notion of imported headers

* Implement basic header ancestry checker

* Add mock runtime for tests

* Add testing boilerplate

* Add some storage read/write sanity tests

* Add some basic header import tests

* Add tests for ancestry proofs

* Create helper for changing authority sets

* Fix authority set test

Fixes a problem with how the scheduled change was counted as well as
a SCALE encoding issue

* Correctly check for scheduled change digests

There's no guarantee that the consensus digest item will be the last
one in a header, which is how it was previously being checked.

Thanks to Andre for pointing me to the Grandpa code that does this.

* Mark imported headers as finalized when appropriate

When a header that finalizes a chain of headers is succesfully imported
we also want to mark its ancestors as finalized.

* Add helper for writing test headers

* Add test helper for scheduling authority set changes

* Bump Substrate pallet and primitives to rc6

* Remove Millau verifier implementation

* Add some doc comments

* Remove some needless returns

* Make Clippy happy

* Split block import from finalization

* Make tests compile again

* Add test for finalizing header after importing children

* Create a test stub for importing future justifications

* Start adding genesis config

* Reject justifications from future

We should only be accepting justifications for the header
which enacted the current authority set. Any ancestors of
that header which require a justification can be imported
but they must not be finalized.

* Add explanation to some `expect()` calls

* Start adding GenesisConfig

* Plug genesis config into runtime

* Remove tests module

* Check for overflow when updating authority sets

* Make verifier take ownership of headers during import

* Only store best finalized header hash

Removed the need to store the whole header, since we store
it was part of the ImportedHeaders structure anyways

* Add some helpers to ImportedHeader

* Update ancestry checker to work with ImportedHeaders

* Update ancestry tests to use ImportedHeaders

* Update import tests to use ImportedHeaders

* Clean up some of the test helpers

* Remove stray dbg!

* Add doc comments throughout

* Remove runtime related code

* Fix Clippy warnings

* Remove trait bound on ImportedHeader struct

* Simplify checks in GenesisConfig

* Rename `get_header_by_hash()`

* Alias `parity_scale_codec` to `codec`

* Reword Verifier documentation

* Missed codec rename in tests

* Split ImportError into FinalizationError

* Remove ChainVerifier trait

This trait was a remenant of the original design, and it is not required
at the moment. Something like it should be added back in the future to
ensure that other chains which conform to this interface can be used
by higher-level bridge applications.

* Fix the verifier tests so they compile

* Implement Deref for ImportedHeader

* Get rid of `new` methods for some Substrate primitives

* Ensure that a child header's number follows its parent's

* Prevent ancestry checker from aimlessly traversing to genesis

If an ancestor which was newer than the child header we were checking we
would walk all the way to genesis before realizing that we weren't related.
This commit fixes that.

* Remove redundant clones

* Ensure that old headers are not finalized

Prevents a panic where if the header being imported and `best_finalized`
were the same header the ancestry checker would return an empty list. We
had made an assumption that the list would always be populated, and if this
didn't hold we would end up panicking.

* Disallow imports at same height as `best_finalized`

* Fix Clippy warnings

* Make NextScheduledChange optional

* Rework how scheduled authority set changes are enacted

We now require a justification for headers which _enact_ changes
instead of those which _schedule_ changes. A few changes had to
be made to accomodate this, such as changing when we check for
scheduled change logs in incoming headers.

* Update documentation for Substrate Primitives

* Clarify why we skip header in requires_justification check

* Add description to assert! call

* Fix formatting within macros

* Remove unused dependencies from runtime

* Remove expect call in GenesisConfig

* Turn FinalityProof into a struct

* Add some inline TODOs for follow up PRs

* Remove test which enacted multiple changes

This should be added back at some later point in time, but right now
the code doesn't allow for this behaviour.

* Use `contains_key` when checking for header

This is better than using `get().is_some()` since we skip
decoding the storage value

* Use initial hash when updating best_finalized

* Add better checks around enacting scheduled changes

* Rename finality related functions

* Appease Clippy

* relay-ethereum-client

* use relay-ethereum-client from ethereum-poa-relay

* cargo fmt --all

* #![warn(missing_docs)]

* EthereumRpcClient -> EthereumClient

* make EthereumHeadersSyncPipeline private

* return concrete type from crate::new

* cleanup dependencies

* *self -> self

* remove trait Client

* sort deps

* extracted relay crates

* moved metrics to utils

* exchange-relay compilation

* fix compilation of headers-relay

* fixed messages-relay compilation

* fixed ethereum-poa-relay compilation

* cargo lock

* cargo fmt --all

* clippy

* cargo fmt --all

* fix tests compilation

* clippy

* eof

* module level docs

* removed obsolete comment

* #![warn(missing_docs)]

* .0 -> Deref

* post-merge fix

* cargo fmt

* Update relays/headers-relay/src/headers.rs

Co-authored-by: Hernando Castano <[email protected]>

* Update relays/headers-relay/src/headers.rs

Co-authored-by: Hernando Castano <[email protected]>

* Update relays/headers-relay/src/lib.rs

Co-authored-by: Hernando Castano <[email protected]>

Co-authored-by: Hernando Castano <[email protected]>

* call dispatch origin

* Update modules/call-dispatch/src/lib.rs

Co-authored-by: Tomasz Drwięga <[email protected]>

* cargo fmt --all

Co-authored-by: Tomasz Drwięga <[email protected]>

* relay receiving + processing confirmations

* fmt && clippy

* removed message processing race

* remove more traces

* generic args names