This adds support for returning a compressed PoV directly to the
collation generation. This is mainly to not requiring to compress a PoV
twice.

Cumulus actually uses the `build_full` function. I missed that.

* [ci] Fix scheduled pipeline

* empty commit for pipeline rerun

* staking miner: use config for emergency solution

Fixes #4678

* bump jsonrpsee

* run `monitor_cmd_for` until the connection is closed

* new tokio task for submit_and_watch xt

* re-use header subscription

* update jsonrpsee + simplify code

* revert polkadot runtime changes

* fix grumbles

* Update utils/staking-miner/src/monitor.rs

* grumbles: fix logs + nits

Closes #4763

Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Bump spec_version

* Bump version to 0.9.16

* Bump substrate

* Update Substrate

Co-authored-by: Bastian Köcher <info@kchr.de>

* bump transaction_version for kusama

* bump transaction_version for polkadot

* bump transaction_version for westend

We received several alerts about disk space in gitlab server.
Even if we have 1TiB allocated, disk space is consumed very fast.
We need to address this problem by expiring artifacts after 7 days.

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Don't merge this

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* Add baseline benches

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Don't merge this"

This reverts commit d0fcc4f9cf83492862618c3b254cb6589f1b1831.

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=frame_benchmarking::baseline --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/frame_benchmarking_baseline.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=frame_benchmarking::baseline --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/frame_benchmarking_baseline.rs

* Add weight modules

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Apply suggestions from code review

Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* fixup

* fmt

* fix tests

* Move PersistedValidationData check into

* Address feedback

* Remove incorrect comment

* Update runtime/parachains/src/inclusion/mod.rs

* fmt

* Add logging

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Andronik <write@reusable.software>

* Fix incomplete sorting.

* fmt.

* Better test.

* Update runtime/parachains/src/disputes.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* chore fmt

* simplify the sorting for two items

* add test for assure duplicates are detected

* fixup tests

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>

* Adding `GovernanceFallback` after changes to
substrate

* Update Substrate

* Update Substrate again

Co-authored-by: Bastian Köcher <info@kchr.de>

* Rename/refactor around get_session_index

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* choose proper head for fetching session

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* revert rename

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* renaming and more comments

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Run Fetch task in correct session

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Log warning when ancestors unavailable

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Dont error in finality_target_with_longest_chain

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add error flag

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Add error flag in dispute-coordinator

Make sure to send errors to subsystems requesting data depending on missing session info

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Scrape ancestors

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Fix naming

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review feedback

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fmt

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 

 fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* consume

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix tests

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* typo

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* review fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Bump scraped blocks LRU capacity

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* 🧯 



Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* remove prints

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Increase scraped blocks cache size

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* more review fixes

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* another fix

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* fix target_ancestor

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Scrape up to max finalized block

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* undo comment change

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* Limit ancestry lookup to last finalized block or
max finality lag

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* debug damage

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

* availability-distribution: look for leaf ancestors

* Re-use subsystem-util

* Rework ancestry tasks scheduling

* Requester tests

* Improve readability for ancestors lookup

* Fixing polkadot after subtrate changes
`MAX_NOMINATIONS` was changed to `MaxNominations`

* Fixing some more places

* Apply suggestions from code review

* update lockfile for {"substrate"}

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: parity-processbot <>

* Add codeden-units=1

ref #4311

* opt-level to 3

* Fix opt-level

* Refactor apt-level into the release profile

* Companion for substrate#10632

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* cargo format

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Fix bags-list

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Update Substrate

* Fix

Signed-off-by: koushiro <koushiro.cqx@gmail.com>

* Fix bridges test

* FMT

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Bastian Köcher <info@kchr.de>

* [ci] WIP: pipeline chores

* fix vault

* fix secrets

* Add templates

* Add folder for local storage of the digests

* Add first draft of the changelog scripts

* Enable Audits in the change template

* Fixes for Polkadot

* Fix templating issue in case there is no high prio change

* Fix Ruby setup

* Remove shell

* Fix chain names

* Fix ENV

* Fix how to get runtime

* Fix runtime_dir

* Fix context location

* Pin changelogerator to a specific version

* Companion for Substrate#10655

https://github.com/paritytech/substrate/pull/10655

This removes the last usages of `Default` in conjunction with `AccountId`

* More fixes

* More of them!

* FMT

* Update Substrate

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

We use the `Compiled` wasm execution and for that the `wasmtime` needs
to be enabled.

* wip template for hrmp benchmarks

* add all of the benchmarks, first draft

* File was not saved :/

* cargo +nightly fmt

* Use configs

* add configs

* Fix rococo

* Final touches

* revert fmt changes, one last time

* Fix wrappings

* Fix a bunch of tests

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_hrmp.rs

* add to westend

* actually use everything

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_hrmp.rs

* Update runtime/parachains/src/hrmp.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* use real weight in wnd

* reorg

* minor cleanup

* weigh some of the internal stuff as well

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_hrmp.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_hrmp.rs

* add files

* Master.into()

* add validation

* fmt

* fmt

* final fixes

* all runtimes build

* undo formatting

* Update runtime/parachains/src/hrmp.rs

Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* non-controversial changes

* do it the parachain-way: use const instead of type configs for simplicity.

* borrow assert_storage_consistency_exhaustive

* move assert_storage_consistency_exhaustive to Pallet, so it can be reused for benchmarks as well.

* fix typo

Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Sergey Shulepov <s.pepyakin@gmail.com>
Co-authored-by: Zeke Mostov <z.mostov@gmail.com>

* Refactor ParentIsDefault to ParentIsAllZeroes

* Remove Default bound on all AccountId types under the xcm directory

* Change to ParentIs<A: Get<AccountId>, AccountId>

* Provide a better account for ParentIs

* Fixes

* Fixes

* Fixes

* Fixes

* Update xcm/xcm-builder/src/currency_adapter.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Use preset account ID value for parent MultiLocations

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Add `without_storage_info`

The MaxEncodedLen trait is now enforced by default in Substrate.
All pallets missing an implementation need to be marked with
`without_storage_info` now.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove `generate_storage_info`

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add more `without_storage_info`

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>