Changes the POV format used by pov exporter and validator. The
`pov-validator` will now work with PoVs recovered from relay chain DA
and created by the collator pov exporter.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: ordian <write@reusable.software>

Closes #7573

@skunert  @bkchr



---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

- Closes https://github.com/paritytech/license-scanner/issues/44
- Silent because only comments are changed in the crates.

## What's inside

First, we change the file traversal mechanism from shell globbing to
walking through files which happens inside the `license-scanner` - so it
has knowledge about directory structure and can correlate files with
corresponding Cargo manifest.

Next, added `MIT-0` and `Unlicense` to the allowed list of licenses.
`Unlicense` appears only once across {Substrate,Cumulus,Polkadot} - in
`penpal-runtime`.

Finally, updated headers in files that do not match the corresponding
manifest license.

---------

Co-authored-by: cornholio <0@mcornholio.ru>

Fixing: 
```
Verification failed for block 0x07bbf1e04121d70a4bdb21cc055132b53ac2390fa95c4d05497fc91b1e8bf7f5 received from (12D3KooWJzLd8skcAgA24EcJey7aJAhYctfUxWGjSP5Usk9wbpPZ): "Header 0x07bbf1e04121d70a4bdb21cc055132b53ac2390fa95c4d05497fc91b1e8bf7f5 rejected: too far in the future"   
```

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: Dmitry Sinyavin <dmitry.sinyavin@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

Given how the block production is driven for Parachains right now, with
the enabling of async backing we would produce two blocks per slot.
Until we have a proper collator implementation, the "hack" is to prevent
the production of multiple blocks per slot.


Closes: https://github.com/paritytech/polkadot-sdk/issues/3282

* Add missing Cumulus licenses

* Typo

* Add missing Substrate licenses

* Single job checking the sub-repos in steps

* Remove dates

* Remove dates

* Add missing (C)

* Update FRAME UI tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update more UI tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update substrate & polkadot

* min changes to make async backing compile

* (async backing) parachain-system: track limitations for unincluded blocks (#2438)

* unincluded segment draft

* read para head from storage proof

* read_para_head -> read_included_para_head

* Provide pub interface

* add errors

* fix unincluded segment update

* BlockTracker -> Ancestor

* add a dmp limit

* Read para head depending on the storage switch

* doc comments

* storage items docs

* add a sanity check on block initialize

* Check watermark

* append to the segment on block finalize

* Move segment update into set_validation_data

* Resolve para head todo

* option watermark

* fix comment

* Drop dmq check

* fix weight

* doc-comments on inherent invariant

* Remove TODO

* add todo

* primitives tests

* pallet tests

* doc comments

* refactor unincluded segment length into a ConsensusHook (#2501)

* refactor unincluded segment length into a ConsensusHook

* add docs

* refactor bandwidth_out calculation

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* test for limits from impl

* fmt

* make tests compile

* update comment

* uncomment test

* fix collator test by adding parent to state proof

* patch HRMP watermark rules for unincluded segment

* get consensus-common tests to pass, using unincluded segment

* fix unincluded segment tests

* get all tests passing

* fmt

* rustdoc CI

* aura-ext: limit the number of authored blocks per slot (#2551)

* aura_ext consensus hook

* reverse dependency

* include weight into hook

* fix tests

* remove stray println

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>

* fix test warning

* fix doc link

---------

Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>

* parachain-system: ignore go ahead signal once upgrade is processed (#2594)

* handle goahead signal for unincluded segment

* doc comment

* add test

* parachain-system: drop processed messages from inherent data (#2590)

* implement `drop_processed_messages`

* drop messages based on relay parent number

* adjust tests

* drop changes to mqc

* fix comment

* drop test

* drop more dead code

* clippy

* aura-ext: check slot in consensus hook and remove all `CheckInherents` logic (#2658)

* aura-ext: check slot in consensus hook

* convert relay chain slot

* Make relay chain slot duration generic

* use fixed velocity hook for pallets with aura

* purge timestamp inherent

* fix warning

* adjust runtime tests

* fix slots in tests

* Make `xcm-emulator` test pass for new consensus hook (#2722)

* add pallets on_initialize

* tests pass

* add AuraExt on_init

* ".git/.scripts/commands/fmt/fmt.sh"

---------

Co-authored-by: command-bot <>

---------

Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>

* update polkadot git refs

* CollationGenerationConfig closure is now optional (#2772)

* CollationGenerationConfig closure is now optional

* fix test

* propagate network-protocol-staging feature (#2899)

* Feature Flagging Consensus Hook Type Parameter (#2911)

* First pass

* fmt

* Added as default feature in tomls

* Changed to direct dependency feature

* Dealing with clippy error

* Update pallets/parachain-system/src/lib.rs

Co-authored-by: asynchronous rob <rphmeier@gmail.com>

---------

Co-authored-by: asynchronous rob <rphmeier@gmail.com>

* fmt

* bump deps and remove warning

* parachain-system: update RelevantMessagingState according to the unincluded segment (#2948)

* mostly address 2471 with a bug introduced

* adjust relevant messaging state after computing total

* fmt

* max -> min

* fix test implementation of xcmp source

* add test

* fix test message sending logic

* fix + test

* add more to unincluded segment test

* fmt

---------

Co-authored-by: Chris Sosnin <chris125_@live.com>

* Integrate new Aura / Parachain Consensus Logic in Parachain-Template / Polkadot-Parachain (#2864)

* add a comment

* refactor client/service utilities

* deprecate start_collator

* update parachain-template

* update test-service in the same way

* update polkadot-parachain crate

* fmt

* wire up new SubmitCollation message

* some runtime utilities for implementing unincluded segment runtime APIs

* allow parachains to configure their level of sybil-resistance when starting the network

* make aura-ext compile

* update to specify sybil resistance levels

* fmt

* specify relay chain slot duration in milliseconds

* update Aura to explicitly produce Send futures

also, make relay_chain_slot_duration a Duration

* add authoring duration to basic collator and document params

* integrate new basic collator into parachain-template

* remove assert_send used for testing

* basic-aura: only author when parent included

* update polkadot-parachain-bin

* fmt

* some fixes

* fixes

* add a RelayNumberMonotonicallyIncreases

* add a utility function for initializing subsystems

* some logging for timestamp adjustment

* fmt

* some fixes for lookahead collator

* add a log

* update `find_potential_parents` to account for sessions

* bound the loop

* restore & deprecate old start_collator and start_full_node functions.

* remove unnecessary await calls

* fix warning

* clippy

* more clippy

* remove unneeded logic

* ci

* update comment

Co-authored-by: Marcin S. <marcin@bytedude.com>

* (async backing) restore `CheckInherents` for backwards-compatibility (#2977)

* bring back timestamp

* Restore CheckInherents

* revert to empty CheckInherents

* make CheckInherents optional

* attempt

* properly end system blocks

* add some more comments

* ignore failing system parachain tests

* update refs after main feature branch merge

* comment out the offending tests because CI runs ignored tests

* fix warnings

* fmt

* revert to polkadot master

* cargo update -p polkadot-primitives -p sp-io

---------

Co-authored-by: asynchronous rob <rphmeier@gmail.com>
Co-authored-by: Ignacio Palacios <ignacio.palacios.santos@gmail.com>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
Co-authored-by: Marcin S. <marcin@bytedude.com>
Co-authored-by: eskimor <eskimor@users.noreply.github.com>
Co-authored-by: Andronik <write@reusable.software>

* Companion for Substrate#14612

https://github.com/paritytech/substrate/pull/14612

* Remove patch

* Cargo.lock

* Fix

* Fix compilation

* Fix Fix

* ...

* 

* .................

* update lockfile for {"polkadot", "substrate"}

* ".git/.scripts/commands/fmt/fmt.sh"

---------

Co-authored-by: parity-processbot <>

* Companion for substrate 14647

* Futher cleanup

* Apply review suggestion

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* Apply suggestions from code review

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

* update lockfile for {"polkadot", "substrate"}

---------

Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: parity-processbot <>

* rough draft of potential parent search

* get things compiling

* fmt

* add new function to all RelayChainInterface implementations

* fix compilation

* set slot and timestamp based on relay parent, prepare for find-parent

* skeleton of new aura logic

* fmt

* introduce a collator module in the Aura crate

* extract different implementations into own modules

* make interface more convenient

* docs and todos for lookahead

* refactor basic collator to use new collator utility

* some more refactoring

* finish most of the control flow for new aura

* introduce backend as parameter

* fix compilation

* fix a couple more TODOs

* add an `announce_block` function to collator service

* announce with barrier

* rename block announcement validator to be more specific

* fmt

* clean up unused import errors

* update references to BlockAnnounceValidator

* rename unstable_reimpl

* add AuraUnincludedSegmentApi

* finish rename

* integrate AuraUnincludedSegmentApi

* add a new block announcement validator for backwards compatibility

* add some naive equivocation defenses

* rustfmt

* clean up remaining TODO [now]s

* fmt

* try to fix inprocess-interface

* actually fix compilation

* ignored -> rejected rephrase

* fix test compilation

* fmt

* clippy

* implement a proposer utility for consensus

* tidy up deps of new proposer crate

* implement a collator-service crate

* rewrite cumulus-collator to use new service struct

* implement a module for relay-chain-driven collators

* adapt start_collator to use the new relay_chain_driven module

* move collator-service to a public submodule

* create an interface trait for the proposer

* begin aura reimplementation

* address review comments

* update substrrate git ref

* update polkadot-primitives refs

* rough draft of aura collation using standalone fns

* add a ServiceInterface

* port aura reimpl to use new service trait

* add an import queue utility crate

* remove import queue crate in favor of module in common

* implement new verification queue for aura

* implement remaining behaviors

* split 'collate' into smaller functions that could be pub

* add telemetry

* fix doc job?

* Specify async-trait patch version

Co-authored-by: Bastian Köcher <git@kchr.de>

* remove 'fn@' in doc string.

Co-authored-by: Bastian Köcher <git@kchr.de>

* update variable names to be more readable

* refactor proposer errors to anyhow/thiserror

* remove manual span instrumentation

Co-authored-by: Bastian Köcher <git@kchr.de>

* make slot_claim private

* fix unused import

* fmt

* fmt

* make clippy happy

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

* Align to Substrate keystore overhaul

* 'sync_keystore' renamed to 'keystore'

* Further fixes after removal of async keystore trait

* update lockfile for {"polkadot", "substrate"}

---------

Co-authored-by: parity-processbot <>

* Use primitives reexported from `polkadot_primitives` crate root

* restart CI

* Fixes after merge

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>

Prevents the StateDbError::TooManySiblingBlocks error from being triggered by eagerly removing 
stale blocks from the backend on block import and before the error condition is met.

Introduces a just in time block recovery mechanism for blocks that were wrongly removed
via an explicit pov-recovery method

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Provisioner companion changes

* update substrate commit hash

* Update Substrate & Polkadot

Co-authored-by: Bastian Köcher <info@kchr.de>

* Companion for aura CompatibilityMode

As no parachain was launched with the broken authority set change handling we don't need to expose
the compatibility mode.

* FMT

* update lockfile for {"substrate", "polkadot"}

Co-authored-by: parity-processbot <>

* Companion for Substrate #12224

* update lockfile for {"polkadot", "substrate"}

Co-authored-by: parity-processbot <>

* Aura: Switch to `SimpleSlotWorker`

* Fix build

* Update Substrate & Polkadot

* Update again

* branches

* update all primitives references

* fmt

* Update Polkadot & Substrate

* Again

Co-authored-by: Bastian Köcher <info@kchr.de>

* companion for slot duration changes

* fix aura inherent data provider

* update lockfile for {"polkadot"}

Co-authored-by: parity-processbot <>

* chore: fix typos for consensus

* chore: fix typos for RelayChainError

* Update for latests substrate & polkadot

* fmt

* ci: add quick-check with clippy and rustfmt

* chore: rustfmt round

* chore: set the same rustfmt config than substrate

* chore: fix formatting

* cI: remove clippy

* ci: switch to nightly for the checks

* ci: fix toolchains and naming

* ci: Limit the check to formatting

* chore: fix formatting

* Update .rustfmt.toml

* Update .rustfmt.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Bump Substrate, Polkadot, and BEEFY

* Add `Sync` bound to Aura trait bounds

* Update Substrate & Polkadot

* Format

* Update again

* use max_block_proposal_slot_portion

* remove patch now that Substrate PR is merged

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* compact, need to be made optional and look into/compress child trie
roots/state.

* proto with child trie support

* Missing set_offchain_storage overload.

* right name

* Ignore offchain indexing in validation function.

* patch trie-db

* decompress from iter

* use compressed proof

* remove wasm blob from proof (no inject plugged yet)

* change lock

* update trie

* change in toml

* Revert "change in toml"

This reverts commit e588583b79e1a64e2a54157b40968aa6b586fbbe.

* use patch to branches

* i

* i:wq

* switch branch

* ii

* ok, needed to patch the runtime by putting substrate patch in polkadot
project.

* test passing with this conf

* actual lazy code fetch

* patch issue

* Code reorg

* restore commented tests.

* update deps.

* remove polka patch

* fixes

* remove patch

* revert cargo.lock

* cargo update -p sp-trie polkadot-service

* fix collator test (using parent state root).

* Update pallets/parachain-system/src/validate_block/implementation.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove encode_witness test function.

* Update pallets/parachain-system/src/validate_block/implementation.rs

* Fix compilation

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <info@kchr.de>

* Make Aura and relay chain verifier buildable

* AHHH

* Ahhh2

* Ahhh3

* Move `ParachainBlockImport`

* Updates because of Substrate

* Revert "AHHH"

This reverts commit 3f7c84327e1036ed71a8e2bd30f6416d32bae5a2.

* Fix warning

* Update polkadot

* Migrate all uses of MQC heads to merkle proofs

* Mass rename `relay_parent_storage_root`

* Restore parachain-system tests

* Update polkadot and libp2p swarm for testing

* Collapse match into an if let

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Start with something

* Update Substrate & Polkadot

* Start to make it compile

* Make it compile

* Begin with something

* Yep

* I'm a hacker

* Bring back the builder

* Make it work in some way

* Compile

* Parachains use their own "slot"

* Adds cumulus-pallet-aura

* Wrap AuRa import queue to disable equivocation checking by default

* Pass slot duration

* Check the seal when validating a block

* Adds missing file

* Try to make the seal working

* Fix it

* Some fixes

* Bring in the latest features to cleanup the code

* Update and make it compile

* Improve the import

* Start fixing

* More work

* Fix fix fix

* Make everything compile

* Small cleanups

* Rename and more docs

* Docs

* Fixes fixes fixes

* Update rococo-parachains/src/chain_spec.rs

* Update client/consensus/aura/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/aura/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/parachain-inherent/Cargo.toml

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/parachain-inherent/Cargo.toml

* Update primitives/parachain-inherent/Cargo.toml

* Update primitives/parachain-inherent/Cargo.toml

Co-authored-by: Sergei Shulepov <sergei@parity.io>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>