* guide: add candidate information to OccupiedCore

* add descriptor and hash to occupied core type

* guide: add candidate hash to inclusion

* runtime: return candidate info in core state

* bitfield signing: stop querying runtime as much

* minimize going to runtime in availability distribution

* fix availability distribution tests

* guide: remove para ID from Occupied core

* get all crates compiling

* adjust span names

* fix compile

* add better spans for unbacked candidates

* improve span names

* Fix bug and further optimizations in availability distribution

- There was a bug that resulted in only getting one candidate per block
as the candidates were put into the hashmap with the relay block hash as
key. The solution for this is to use the candidate hash and the relay
block hash as key.
- We stored received/sent messages with the candidate hash and chunk
index as key. The candidate hash wasn't required in this case, as the
messages are already stored per candidate.

* Update node/core/bitfield-signing/src/lib.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Remove the reverse map

* major refactor of receipts & query_live

* finish refactoring

remove ancestory mapping,

improve relay-parent cleanup & receipts-cache cleanup,
add descriptor to `PerCandidate`

* rename and rewrite query_pending_availability

* add a bunch of consistency tests

* Add some last changes

* xy

* fz

* Make it compile again

* Fix one test

* Fix logging

* Remove some buggy code

* Make tests work again

* Move stuff around

* Remove dbg

* Remove state from test_harness

* More refactor and new test

* New test and fixes

* Move metric

* Remove "duplicated code"

* Fix tests

* New test

* Change break to continue

* Update node/core/av-store/src/lib.rs

* Update node/core/av-store/src/lib.rs

* Update node/core/bitfield-signing/src/lib.rs

Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>

* update guide to match live_candidates changes

* add comment

* fix bitfield signing

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>

* refactor View to include finalized_number

* guide: update the NetworkBridge on BlockFinalized

* av-store: fix the tests

* actually fix tests

* grumbles

* ignore macro doctest

* use Hash::repeat_bytes more consistently

* broadcast empty leaves updates as well

* fix issuing view updates on empty leaves updates

* Improve Jaeger errors and debugging experience

* Bind on 0.0.0.0:0 instead

Right now if the collation is not happening one will have to sprinkle
log statements and then recompile the code. It's doubly annoying if that
happens when working with Cumulus: that means one has to resort to
.cargo/config's `paths` or `diener`, which both are not ideal.

This just adds some verbose logging to save the investigators some time
when looking why the collations are not happening

* wk2048 | D6 | issue-7143-treasury-refactor | integration

* wk2048 | D6 | issue-7143-treasury-refactor | integration | p2

* trait -> config

* fix weight files and import

* missed some

* fix import

* fix imports

* alphabetize

* fix config traits

* fix trait

* update traits

* update weights

* "Update Substrate"

* fix features

* Update runtime/kusama/src/lib.rs

* Update runtime/polkadot/src/lib.rs

* add bounties and tips in proxy filters

* remove unused

* remove unused

* remove unused

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: parity-processbot <>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>

* start approval types

* doc

* finish basic types

* add debug and codec impls to approval types

* grumbles

* remove unused AssignmentId

* remove aux-schema file

Bumps [indexmap](https://github.com/bluss/indexmap) from 1.6.0 to 1.6.1.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.6.0...1.6.1

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* guide: merge backing and approval keys

* bump substrate master & update primitives

* use new SessionInfo struct in session_info

* session keys upgrade for Polkadot

* kusama & westend runtimes

* bump westend, kusama, and polkadot versions

* add session key to rococo & test-runtime

* update prepare-test-net to latest subkey

* update chain specs to support new para_assignment session key

* get cargo.lock from master

* formatting

* update kill_storage based on substrate master

* fix test-service

* assgn -> asgn

* use session info module for assignment session key

* remove low information density error doc comments

* another round of error dancing

* fix compilation

* remove stale `None` argument

* adjust test, minor slip in command

* only add AvailabilityError for full node features

* another None where none shuld be

* adding missing param

* updating to latest master

* Adds consistency checks for the `HostConfiguration`

Besides that it fixes the chain specs to make the consistency checks happy.

* Update runtime/parachains/src/configuration.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Review feedback and test fixes etc

* Update

* More

* I'm an idiot

* Fix tests...

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Do not spam when we can not send a message to a job

There are legal reasons why a job ended. If a job failed, the error is
logged. So, we don't need to log an error when we can not send a message
to a job.

* Review feedback

* Rework `ConnectionsRequests`

Instead of implementing the `Stream` trait, this struct now provides a
function `next()`. This enables us to encode into the type system that
it will always return a value or block indefinitely.

* Review feedback

* Bump version from v0.8.26 to v0.8.27

* bump substrate

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.117 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.117...v1.0.118

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Because of a bug in the test script, we didn't stopped CI when the main
tests are failed.

Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.5.0...v1.5.1

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add an upper number of maximum parallel runtime api requests

Instead of spawning all runtime api requests in the background and using
all wasm instances. This pr adds a maximum number of parallel requests.

* Update node/core/runtime-api/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* Review feedback

* Increase instances

* Add warning

* Update node/core/runtime-api/src/lib.rs

Co-authored-by: Sergei Shulepov <sergei@parity.io>

Co-authored-by: Sergei Shulepov <sergei@parity.io>

* guide: non-semantic changes

* guide: update per the issue description

* GetBackedCandidates operates on multiple hashes now

* GetBackedCandidates still needs a relay parent

* implement changes specified in guide

* distinguish between various occasions for canceled oneshots

* add tracing info to getbackedcandidates

* REVERT ME: add tracing messages for GetBackedCandidates

Note that these messages are only sometimes actually passed on to the
candidate backing subsystem, with the consequence that it is
unexpectedly frequent that the provisioner fails to create its
provisionable data.

* REVERT ME: more tracing logging

* REVERT ME: log when CandidateBackingJob receives any message at all

* REVERT ME: log when send_msg sends a message to a job

* fix candidate-backing tests

* streamline GetBackedCandidates

This uses table.attested_candidate instead of table.get_candidate, because
it's not obvious how to get a BackedCandidate from just a
CommittedCandidateReceipt.

* REVERT ME: more logging tracing job lifespans

* promote warning about job premature demise

* don't terminate CandiateBackingJob::run_loop in event of failure to process message

* Revert "REVERT ME: more logging tracing job lifespans"

This reverts commit 7365f2fb.

* Revert "REVERT ME: log when send_msg sends a message to a job"

This reverts commit 58e46aad.

* Revert "REVERT ME: log when CandidateBackingJob receives any message at all"

This reverts commit 0d6f3841.

* Revert "REVERT ME: more tracing logging"

This reverts commit 675fd262.

* Revert "REVERT ME: add tracing messages for GetBackedCandidates"

This reverts commit e09e1564.

* formatting

* add logging message to CandidateBackingJob::run_loop start

* REVERT ME: add tracing to candidate-backing job creation

* run candidatebacking loop even if no assignment

* use unique error variants for each canceled oneshot

* Revert "REVERT ME: add tracing to candidate-backing job creation"

This reverts commit 8ce5f4f0.

* try_runtime_api more to reduce silent exits

* add sanity check that returned backed candidates preserve ordering

* remove redundant err attribute

* include new parameter in test `Configuration`

* update calls to `init_logger`

* "Update Substrate"

* cargo update -p sp-io

Co-authored-by: Matt <mattrutherford@users.noreply.github.com>
Co-authored-by: parity-processbot <>

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.59 to 1.0.60.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.59...v1.0.60

)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Companion PR for removing register notif protocol

* Missing .into()

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Cadidate selection check assignment

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Review fixes

* Punish collator for wrong announcements

* Update node/core/candidate-selection/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

This is not a big change since the user definition AbstractClient already is 'static.

* introduce a waiting period before selecting candidates and bitfields

* add network_bridge=debug tracing for rep

* change to 2.5s timeout in proposer

* pass timeout to proposer

* move timeout back to provisioner

* grumbles

* Update node/core/provisioner/src/lib.rs

* Fix nitpicks

* Fix bug

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

* refactor some functions to not rely on `self`

* factor out common elements of seconding and attesting

* Add Spawn to backing FromJob

* do candidate validation in background

* tests

* address grumbles

* Simplify subsystem jobs

This pr simplifies the subsystem jobs interface. Instead of requiring an
extra message that is used to signal that a job should be ended, a job
now ends when the receiver returns `None`. Besides that it changes the
interface to enforce that messages to a job provide a relay parent.

* Drop ToJobTrait

* Remove FromJob

We always convert this message to FromJobCommand anyway.

This pr changes how the runtime api subsystem processes runtime api
requests. Instead of answering all of them in the subsystem task and
thus, making all requests sequential, we now answer them in a background
task. This enables us to serve multiple requests at once.

* allow jobs to spawn sub-tasks

* fix fallout in subsytems