This adds support for returning a compressed PoV directly to the
collation generation. This is mainly to not requiring to compress a PoV
twice.

Cumulus actually uses the `build_full` function. I missed that.

* [ci] Fix scheduled pipeline

* empty commit for pipeline rerun

* staking miner: use config for emergency solution

Fixes #4678

* bump jsonrpsee

* run `monitor_cmd_for` until the connection is closed

* new tokio task for submit_and_watch xt

* re-use header subscription

* update jsonrpsee + simplify code

* revert polkadot runtime changes

* fix grumbles

* Update utils/staking-miner/src/monitor.rs

* grumbles: fix logs + nits

Closes #4763

Co-authored-by: Bernhard Schuster <[email protected]>

* Bump spec_version

* Bump version to 0.9.16

* Bump substrate

* Update Substrate

Co-authored-by: Bastian Köcher <[email protected]>

* bump transaction_version for kusama

* bump transaction_version for polkadot

* bump transaction_version for westend

We received several alerts about disk space in gitlab server.
Even if we have 1TiB allocated, disk space is consumed very fast.
We need to address this problem by expiring artifacts after 7 days.

Signed-off-by: Andrei Sandu <[email protected]>

* Don't merge this

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_balances --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* Add baseline benches

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Don't merge this"

This reverts commit d0fcc4f9cf83492862618c3b254cb6589f1b1831.

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=frame_benchmarking::baseline --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/frame_benchmarking_baseline.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=frame_benchmarking::baseline --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/frame_benchmarking_baseline.rs

* Add weight modules

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Apply suggestions from code review

Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* fixup

* fmt

* fix tests

* Move PersistedValidationData check into

* Address feedback

* Remove incorrect comment

* Update runtime/parachains/src/inclusion/mod.rs

* fmt

* Add logging

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Andronik <[email protected]>

* Fix incomplete sorting.

* fmt.

* Better test.

* Update runtime/parachains/src/disputes.rs

Co-authored-by: Bastian Köcher <[email protected]>

* chore fmt

* simplify the sorting for two items

* add test for assure duplicates are detected

* fixup tests

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Bernhard Schuster <[email protected]>

* Adding `GovernanceFallback` after changes to
substrate

* Update Substrate

* Update Substrate again

Co-authored-by: Bastian Köcher <[email protected]>

* Rename/refactor around get_session_index

Signed-off-by: Andrei Sandu <[email protected]>

* choose proper head for fetching session

Signed-off-by: Andrei Sandu <[email protected]>

* revert rename

Signed-off-by: Andrei Sandu <[email protected]>

* fix comments

Signed-off-by: Andrei Sandu <[email protected]>

* renaming and more comments

Signed-off-by: Andrei Sandu <[email protected]>

* review feedback

Signed-off-by: Andrei Sandu <[email protected]>

* Run Fetch task in correct session

Signed-off-by: Andrei Sandu <[email protected]>

* Log warning when ancestors unavailable

Signed-off-by: Andrei Sandu <[email protected]>

* Fixes

Signed-off-by: Andrei Sandu <[email protected]>

* fix

Signed-off-by: Andrei Sandu <[email protected]>

* Dont error in finality_target_with_longest_chain

Signed-off-by: Andrei Sandu <[email protected]>

* fix

Signed-off-by: Andrei Sandu <[email protected]>

* Add error flag

Signed-off-by: Andrei Sandu <[email protected]>

* Add error flag in dispute-coordinator

Make sure to send errors to subsystems requesting data depending on missing session info

Signed-off-by: Andrei Sandu <[email protected]>

* Scrape ancestors

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* fix

Signed-off-by: Andrei Sandu <[email protected]>

* Fix naming

Signed-off-by: Andrei Sandu <[email protected]>

* review feedback

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* 💬

 fixes

Signed-off-by: Andrei Sandu <[email protected]>

* consume

Signed-off-by: Andrei Sandu <[email protected]>

* fix tests

Signed-off-by: Andrei Sandu <[email protected]>

* typo

Signed-off-by: Andrei Sandu <[email protected]>

* review fixes

Signed-off-by: Andrei Sandu <[email protected]>

* Bump scraped blocks LRU capacity

Signed-off-by: Andrei Sandu <[email protected]>

* 🧯 🔥



Signed-off-by: Andrei Sandu <[email protected]>

* remove prints

Signed-off-by: Andrei Sandu <[email protected]>

* Increase scraped blocks cache size

Signed-off-by: Andrei Sandu <[email protected]>

* more review fixes

Signed-off-by: Andrei Sandu <[email protected]>

* another fix

Signed-off-by: Andrei Sandu <[email protected]>

* fix target_ancestor

Signed-off-by: Andrei Sandu <[email protected]>

* Scrape up to max finalized block

Signed-off-by: Andrei Sandu <[email protected]>

* undo comment change

Signed-off-by: Andrei Sandu <[email protected]>

* Limit ancestry lookup to last finalized block or
max finality lag

Signed-off-by: Andrei Sandu <[email protected]>

* debug damage

Signed-off-by: Andrei Sandu <[email protected]>

* availability-distribution: look for leaf ancestors

* Re-use subsystem-util

* Rework ancestry tasks scheduling

* Requester tests

* Improve readability for ancestors lookup

* Fixing polkadot after subtrate changes
`MAX_NOMINATIONS` was changed to `MaxNominations`

* Fixing some more places

* Apply suggestions from code review

* update lockfile for {"substrate"}

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: parity-processbot <>

* Add codeden-units=1

ref #4311

* opt-level to 3

* Fix opt-level

* Refactor apt-level into the release profile

* Companion for substrate#10632

Signed-off-by: koushiro <[email protected]>

* cargo format

Signed-off-by: koushiro <[email protected]>

* Fix bags-list

Signed-off-by: koushiro <[email protected]>

* Update Substrate

* Fix

Signed-off-by: koushiro <[email protected]>

* Fix bridges test

* FMT

Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* [ci] WIP: pipeline chores

* fix vault

* fix secrets

* Add templates

* Add folder for local storage of the digests

* Add first draft of the changelog scripts

* Enable Audits in the change template

* Fixes for Polkadot

* Fix templating issue in case there is no high prio change

* Fix Ruby setup

* Remove shell

* Fix chain names

* Fix ENV

* Fix how to get runtime

* Fix runtime_dir

* Fix context location

* Pin changelogerator to a specific version

* Companion for Substrate#10655

https://github.com/paritytech/substrate/pull/10655

This removes the last usages of `Default` in conjunction with `AccountId`

* More fixes

* More of them!

* FMT

* Update Substrate

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

We use the `Compiled` wasm execution and for that the `wasmtime` needs
to be enabled.

* wip template for hrmp benchmarks

* add all of the benchmarks, first draft

* File was not saved :/

* cargo +nightly fmt

* Use configs

* add configs

* Fix rococo

* Final touches

* revert fmt changes, one last time

* Fix wrappings

* Fix a bunch of tests

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_hrmp.rs

* add to westend

* actually use everything

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_hrmp.rs

* Update runtime/parachains/src/hrmp.rs

Co-authored-by: Keith Yeung <[email protected]>

* use real weight in wnd

* reorg

* minor cleanup

* weigh some of the internal stuff as well

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_hrmp.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_hrmp.rs

* add files

* Master.into()

* add validation

* fmt

* fmt

* final fixes

* all runtimes build

* undo formatting

* Update runtime/parachains/src/hrmp.rs

Co-authored-by: Zeke Mostov <[email protected]>

* non-controversial changes

* do it the parachain-way: use const instead of type configs for simplicity.

* borrow assert_storage_consistency_exhaustive

* move assert_storage_consistency_exhaustive to Pallet, so it can be reused for benchmarks as well.

* fix typo

Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Sergey Shulepov <[email protected]>
Co-authored-by: Zeke Mostov <[email protected]>

* Refactor ParentIsDefault to ParentIsAllZeroes

* Remove Default bound on all AccountId types under the xcm directory

* Change to ParentIs<A: Get<AccountId>, AccountId>

* Provide a better account for ParentIs

* Fixes

* Fixes

* Fixes

* Fixes

* Update xcm/xcm-builder/src/currency_adapter.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Use preset account ID value for parent MultiLocations

Co-authored-by: Bastian Köcher <[email protected]>

* Add `without_storage_info`

The MaxEncodedLen trait is now enforced by default in Substrate.
All pallets missing an implementation need to be marked with
`without_storage_info` now.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove `generate_storage_info`

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add more `without_storage_info`

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>