* Those should really be trace.

- Very spammy
- And they in fact trace the execution
- Should not be enabled lightly - will slow network bridge down.

* Make report peers debug again.

* clear_lease

* schedule upgrade and downgrade

* fix compile

* comments

* metered channel - sent & received

* Add for readouts

* metrics for both sent & received

* retract on send failure

* add number to `ActivatedLeavesUpdate`

* update subsystem util and overseer

* use new ActivatedLeaf everywhere

* sort view

* sorted and limited view in network bridge

* use live block hash only if it's newer

* grumples

* Check that para is registered before accepting a bid

* Update lib.rs

* Update lib.rs

* remove println

* fix benchmarks

* Update runtime/common/src/auctions.rs

Co-authored-by: Kian Paimani <[email protected]>

Co-authored-by: Kian Paimani <[email protected]>

* overseer: AllSubsystems magic and report subsystem channel sizes to prometheus

* fix tests

* add AvailableDataFetchingRequest

* rename AvailabilityFetchingRequest to ChunkFetchingRequest

* rename AvailabilityFetchingResponse to Chunk_

* add AvailableDataFetching request

* add available data fetching request to availability recovery message

* remove availability recovery message

* fix

* update network bridge

* port availability recovery to request/response

* use validators.len(), not shuffling

* fix availability recovery tests

* update guide

* Update node/network/availability-recovery/src/lib.rs

Co-authored-by: Bernhard Schuster <[email protected]>

* Update node/network/availability-recovery/src/lib.rs

Co-authored-by: Arkadiy Paronyan <[email protected]>

* remove println

Co-authored-by: Bernhard Schuster <[email protected]>
Co-authored-by: Arkadiy Paronyan <[email protected]>

* f+1 always approves

* guide

* grumbles

* grumbles

* fix test

* fix tests

* Update roadmap/implementers-guide/src/node/approval/approval-voting.md

Co-authored-by: Sergei Shulepov <[email protected]>

Co-authored-by: Sergei Shulepov <[email protected]>

* bump version and substrate

* bump old forgotten versions

* update weights

* bump substrate

* Revert "bump substrate"

This reverts commit 8b5004b6fe9ce9ccdf143d3fe878802931ea4f2f.

Co-authored-by: André Silva <[email protected]>

* Companion for https://github.com/paritytech/substrate/pull/8368

* "Update Substrate"

Co-authored-by: parity-processbot <>

* collator: authenticate collator protocol messages

* fix tests compilation

* node: verify collator protocol signatures in tests

* collator: fix tests

* implementers-guide: update CollatorProtocol messages

* collator: add test for verification of collator protocol signatures

* node: remove fixmes

* node: remove signature from advertisecollation message

* node: add magic constant to Declare message signature payload

* Polkadot companion for Substrate#8420

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Tweaked logging

* Debug for Statement

* improve tracing for approval voting

* assignment criteria tracing

* new syntax

* Companion for substrate/pull/8445

* cargo update -p sp-io

Co-authored-by: Shawn Tabrizi <[email protected]>

* ci: initial fuzzer job

* erasure-coding: update fuzzer Cargo.lock

* syntax fix

* try this first

* install honggfuzz deps

* try not

* try if else

* try SIGINT

* ignore hfuzz dirs

* ???

* bash is growing on me

* decouple builds from running

* fix a typo

* try copying dirs

* fix indentation

* try using absolute paths

* another try

* caching is not worth it

* remove outdated needs

* cleanup and add futher TODOs

* Update .github/workflows/honggfuzz.yml

* more diagnostic logs for approval-voting (#2618)

* Backing and collator protocol traces including para-id (#2620)

* improve backing/provisioner spans

* span for collation requests

* add para_id to unbacked candidate spans

* differentiate validation-construction and find-assignment in selection

* better find-assignment spans

* organize unbacked-candidate spans directly under job root

* Update node/core/provisioner/src/lib.rs

Co-authored-by: Andronik Ordian <[email protected]>

Co-authored-by: Andronik Ordian <[email protected]>

* add tracing when no assignment in candidate selection (#2623)

* New slots/auctions architecture (#2294)

* TODOs

* Add auctions.rs, comment on changes needed.

* Remove cruft from slots

* Remove more from auctions.rs

* More logic drafting in slots.

* More logic in slots.rs

* patch some errors

* more fixes

* last nit

* Cleanups in slots.rs

* Cleanups in slots.rs

* patches

* make build

* crowdloan to new api

* auction compile

* Use ParaId instead of FundIndex in Crowdloan (#2303)

* use paraid instead of fundindex

* Update crowdloan.rs

* check caller is manager

* Auction tests and fix build warnings.

* Configurable origin for initiating auctions

* Remove on_finalize

* #2303 (manual merge)

* Tests for Slots

* some registrar tests

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Slots uses Registrar for CurrentChains

* swap works test

* on swap impl

* traitify parachain cleanup

* explicit lifecycle tracking for paras

* initial implementation of lifecycles and upgrades

* clean up a bit

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* Revert "explicit lifecycle tracking for paras"

This reverts commit 4177af7ba473bbd9c26bccd861793f25265b6657.

* remove lifecycle tracking from registrar

* do not include transitioning paras in identifier

* Update paras_registrar.rs

* final patches to registrar

* Fix test

* use noop in test

* clean up pending swap on deregistration

* finish registrar tests

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* Remove direct need for Slots and Registrar from Crowdloan

* Rejig things slightly

* actions queue

* Revert "actions queue"

This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.

* Traitify Auction interface.

* Mockups and initial code for Crowdloan testing

* One test...

* collapse onboarding state

* fix some crowdloan tests

* one more

* start benchmarks for auctions

* benchmark bid

* fix more crowdloan tests

* onboard and begin retirement no longer exist

* Revert "onboard and begin retirement no longer exist"

This reverts commit 2e100fd94e3540bff5f172328b5d917896f1c6fc.

* Simplify crowdloan and make it work.

* Fixes

* fix some

* finish merge fixes

* fix refund bug in auctions

* Add traits to Registrar for tests and benchmarks

* fix more auction benchmarks

* Fix TestAuctioneer

* finish crowdloan benchmarks

* start setting up full integration tests

* expand integration tests

* finish basic integration test

* add more integration tests

* begin slots benchmarks

* start paras registrar benchmarks

* fix merge

* fix tests

* clean up paras registrar

* remove println

* remove outdated cleanup config

* update benchmarks

* Add WeightInfo

* enable runtime-benchmarks feature flag

* complete swap benchmark

* add parachains and onboarding into westend

* add benchmarks and genesis

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix benchmark execution

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Use `new_raise_len` in crowdloan on_initialize

* Update paras_registrar.rs

* fix westend merge

* impl on_swap for crowdloan

* Check fund exists before create

* update for crowdloan sig

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* slots on_initialize

* use integration tests environment for benchmarks

* fix hrmp event

* auction on_initialize

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix storage name in auctions

* add auction_index to winning data

* winning data takes into account current auction index

* remove println

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Revert "add auction_index to winning data"

* PastRandomness.

* Fixes

* Use new randomness

* fix use of randomness in auctions and runtime config

* expose consts

* fix auction test

* add deposit per byte for para registration

* basic swap integration test

* make swap test more comprehensive

* Add WinningVec for easier retrieval in the front-end.

* clean up `WinningVec` at the end

* Add event for when a new best bid comes in

* Fix propagation of winners in ending period

* fix benchmarks, refund weight in dissolve

* fix unused

* remove some TODOs

* setup opaque keys for paras in westend

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* remove unused

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* back to regular runtime config

* use saturating math where user input can be

* better first slot check

* Update runtime/common/src/claims.rs

* update westend onswap impl

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Guillaume Thiolliere <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>

* update ring to 0.16.20 (#2626)

* Remove TODO from substrate#2986 (#2628)

* Companion for sub/8176 (#2622)

* Merge

* Fixes

* Fix build

* remove dep.

* undo dep.

* upadte substrate

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* Fix lock

* revert lock; cargo update -p sp-io

* from_rational_approx -> from_rational

* Silence more warnings

Co-authored-by: Gav Wood <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Add /data symlink to Docker containers (#2627)

* add /data symlink to Docker

* fix comments

* Fix wrong deposit amount in council voters. (#2562)

* Fix wrong deposit amount in council voters.

* Fix some build

* make it all compile.. so far.

* Fix

* break build

* Okay fix it again

* re benchmark  (#2630)

* Change something

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Reduce number of active leaves at startup (#2631)

Currently we will take all leaves and give that to the overseer on
startup, but this is a bad idea when the finality is lagging for
example. There can be many of unfinalized leaves, we don't even need to
look at anymore. To solve this, the pr adds a maximum of 4 leaves we
forward to the overseer and the pr also checks that we only pass uncles
of the best block.

* yet another set of logging improvements (#2638)

* Polkadot companion for Substrate PR #7640 (Store multiple Justifications per block) (#2358)

* service: update for substrate PR #7640

* update substrate

* Add Pallet Babe to Integration Tests Runtime

Co-authored-by: André Silva <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Companion for Substrate#8386 (#2634)

* Companion for Substrate#8386

https://github.com/paritytech/substrate/pull/8386

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Request based collation fetching (#2621)

* Introduce collation fetching protocol

also move to mod.rs

* Allow `PeerId`s in requests to network bridge.

* Fix availability distribution tests.

* Move CompressedPoV to primitives.

* Request based collator protocol: validator side

- Missing: tests
- Collator side
- don't connect, if not connected

* Fixes.

* Basic request based collator side.

* Minor fix on collator side.

* Don't connect in requests in collation protocol.

Also some cleanup.

* Fix PoV distribution

* Bump substrate

* Add back metrics + whitespace fixes.

* Add back missing spans.

* More cleanup.

* Guide update.

* Fix tests

* Handle results in tests.

* Fix weird compilation issue.

* Add missing )

* Get rid of dead code.

* Get rid of redundant import.

* Fix runtime build.

* Cleanup.

* Fix wasm build.

* Format fixes.

Thanks @andronik !

* Companion for #8372 (Replace 'Module' with 'Pallet' in construct_runtime macro) (#2629)

* Replace 'Module' with 'Pallet'.

* "Update Substrate"

* fix babe usage

* fix benchmark

Co-authored-by: parity-processbot <>
Co-authored-by: thiolliere <[email protected]>

* integrate faster erasure code (#2608)

Breaks compatibility for distributing PoV and PersistentValidationData between validators.

Ref #2442

* Update for the new substrate client API (#2570)

* Update for the new substrate client API

* Code review suggestions

* Update substrate

* Improve the logging (#2645)

* Don't accept incoming connections for collators (#2644)

* Don't accept incoming connections for collators

on the `Collation` peer set.

* Better docs.

* fix reconstruct fuzzer name

* make script more robust

* REVERTME: test run

* REVERTME: test run II

* Revert "REVERTME: test run II"

This reverts commit 58375df3882b9dff4c207f58fd38ccb35bc90c50.

* Revert "REVERTME: test run"

This reverts commit 9759cb61da6169de27d8ee7be0edef75a11f0e7f.

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Guillaume Thiolliere <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Martin Pugh <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Jon Häggblad <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: Robert Klotzner <[email protected]>
Co-authored-by: Shaun Wang <[email protected]>
Co-authored-by: Bernhard Schuster <[email protected]>
Co-authored-by: Arkadiy Paronyan <[email protected]>

* Unify maximum supported PoV size a bit.

* Use MAX_POV_SIZE also in `HostConfiguration`.

* Fix types.

* Remove Parachains Stuff from Westend

* remove unused weights

* clean up genesis

* malicious reputation cost is fatal

* make ReportBad a malicious cost

* futures control-flow for cleaning up inactive collator peers

* guide: network bridge updates

* add `PeerDisconnected` message

* guide: update

* reverse order

* remember to match

* implement disconnect peer in network bridge

* implement disconnect_inactive_peers

* test

* remove println

* don't hardcore policy

* add fuse outside of loop

* use default eviction policy

* Bigger is better.

Made all request response sizes 10 times bigger.

* The smaller the better.

* Update comment.

* Ah, bigger is still better.

Max PoV size for rococo is around 50Meg, compression ratio is about 3.4.
With 30 Meg we should be fine, even with crypto kitties in the PoV.

* Additional logging for polkadot network protocols

* Additional log

* Update node/network/bitfield-distribution/src/lib.rs

Co-authored-by: Robert Habermeier <[email protected]>

* Update node/network/availability-distribution/src/responder.rs

* Added additional chunk info

* Added additional peer info

Co-authored-by: Robert Habermeier <[email protected]>

* initial

* add test

* Update lib.rs

* Update lib.rs

* Allow Root to Edit a Crowdloan Configuration

* Update crowdloan.rs

* add test

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Check fund depositor calls dissolve

* add auction cancel too

* use drain api rather than `iter` + `take`

* add test and benchmarks

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* CI: fix env file location

* Increase payable nominators per validator (#2637)

* inc

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Add test

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Add debug messaes to the bridge actions (#2668)

* more approval voting instrumentation (#2663)

* more approval voting instrumentation

* fix `unapproved_candidates`

* Update node/core/approval-voting/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Add Parachain Onboarding to Rococo (#2660)

* Add Parachain Onboarding Pallets to Rococo

* remove propose_parachain

* add back para sudo wrapper

* Revert "remove propose_parachain"

This reverts commit 38b7c33c817964c9f8ad48531a72aede83670291.

* turn propose_parachains into validator_manager

* Update runtime/rococo/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* update origin

Co-authored-by: Bastian Köcher <[email protected]>

* Improve logging (#2669)

* Improve logging

* Review feedback

* Fix some warning and some further logging changes

* Fix up CI for new labels (#2673)

* Fix up CI/CD for the new labels.

* Fix labels

* [CI] Require D*-audit labels for any runtime changes (#2617)

* update generate_release_text.rb

* add checks for audit labels for runtime changes

* fix check_labels.sh

* use has_runtime_changes in check_runtime.sh

* add D1-trivial label

* fix check_labels.sh

* Tweak Rococo Settings For Onboarding Testing (#2671)

* Tweak Rococo Settings

* rococo: use CurrentBlockRandomness wrapper in auctions

Co-authored-by: André Silva <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Fix check-labels.yml for forked repos (#2677)

* fix check-labels.yml for forked repos

* update labels

* CI: pass VERSION variable to SimNet

* CI: chore

* CI: fix env file location

* CI: pass VERSION variable to SimNet

* CI: chore

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>
Co-authored-by: Pierre Krieger <[email protected]>
Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>
Co-authored-by: Martin Pugh <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

* fix check-labels.yml for forked repos

* update labels

* Tweak Rococo Settings

* rococo: use CurrentBlockRandomness wrapper in auctions

Co-authored-by: André Silva <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* update generate_release_text.rb

* add checks for audit labels for runtime changes

* fix check_labels.sh

* use has_runtime_changes in check_runtime.sh

* add D1-trivial label

* fix check_labels.sh

* Fix up CI/CD for the new labels.

* Fix labels

* Improve logging

* Review feedback

* Fix some warning and some further logging changes

* Add Parachain Onboarding Pallets to Rococo

* remove propose_parachain

* add back para sudo wrapper

* Revert "remove propose_parachain"

This reverts commit 38b7c33c817964c9f8ad48531a72aede83670291.

* turn propose_parachains into validator_manager

* Update runtime/rococo/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* update origin

Co-authored-by: Bastian Köcher <[email protected]>