This deserves a bit of explanation, as the motivating issue explicitly
requested randomness. In short, it's hard to get randomness to compile
for `wasm32-unknown-unknown` because that is explicitly intended to be
as deterministic as practical. Additionally, even though it would never
be used for consensus purposes, it still felt offputting to intentionally
introduce randomness into a node's operations. Except, it wasn't really
random, either: it was a deterministic PRNG varying only in its state,
and getting the state to work right for that target would have required
initializing from a constant.

Given that it was a deterministic sequence anyway, it seemed much simpler
and more explicit to simply select one of each N messages instead of
attempting any kind of realistic randomness.

The previous impl using `from_entropy` depends on the `getrandom`
crate, which uses the system entropy source, and which does not
work on `wasm32-unknown-unknown` because it wants to fall back to
a JS implementation which we can't assume exists.

This impl depends only on `rand::thread_rng`, which has no documentation
stating that it's similarly limited.

This should be more accurate; it ensures that the timer runs
at least as long as that function. As `route_message` is async,
it may not actually run for some time after it is called (or ever).

This timer only exists / logs a fraction of the time (configurable
by `MESSAGE_TIMER_METRIC_CAPTURE_RATE`). When it exists, it tracks
the span between the `OverSubsystemContext` receiving the message
and its receipt in `Overseer::run`.

* utility functions for erasure-coding threshold

* add candidate-hash tag to candidate jaeger spans

* debug implementation for jaeger span

* add a span to each live candidate in availability dist.

* availability span covers only our piece

* fix tests

* keep span alive slightly longer

* remove spammy bitfield-gossip-received log

* Revert "remove spammy bitfield-gossip-received log"

This reverts commit 831a2db5

.

* add claimed validator to bitfield-gossip span

* add peer-id to handle-incoming span

* add peer-id to availability distribution span

* Update node/network/availability-distribution/src/lib.rs

Co-authored-by: Bernhard Schuster <[email protected]>

* Update erasure-coding/src/lib.rs

Co-authored-by: Bernhard Schuster <[email protected]>

* Update node/subsystem/src/jaeger.rs

Co-authored-by: Bernhard Schuster <[email protected]>

Co-authored-by: Bernhard Schuster <[email protected]>

* Cont.: Implement the state root obtaining during inclusion

During inclusion now we obtain the storage root by passing it through
the inclusion_inherent.

* Fix tests

* Bump rococo spec version

* Reorder the parent header into the end

of the inclusion inherent.

When the parent header is in the beginning, it shifts the other two
fields, so that a previous version won't be able to decode that. If
we put the parent header in the end, the other two fields will stay
at their positions, thus make it possible to decode with the previous
version.

That allows us to perform upgrade of rococo runtime without needing of
simultanuous upgrade of nodes and runtime, or restart of the network.

* Squash a stray tab

We need to make sure to drop the import-statement child span before the
parent span is dropped.

* Companion for #7810

* Added missing trait items for tests

* Add another missing trait item

* fixup

* "Update Substrate"

Co-authored-by: parity-processbot <>

* migrate claims to weightinfo

* fix up

* fix benchmark

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=claims --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* fix test runtime

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=claims --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* Update runtime/kusama/src/weights/claims.rs

* use path for pallet

* Update Cargo.lock

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_common::claims --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* Delete runtime_common::claims.rs

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_common::claims --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_common_claims.rs

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_common::claims --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_common_claims.rs

* use full automation file

* patch import

* consolidate benchmark logic

* fix

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_common::claims --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_common_claims.rs

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_common::claims --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_common_claims.rs

* update weight comments

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Companion for Substrate#7795

https://github.com/paritytech/substrate/pull/7795

* Fix missing test

* "Update Substrate"

Co-authored-by: parity-processbot <>

* wk2052 | D1 |Allow council to slash treasury tip | p1

* wk2052 | D1 | Allow council to slash treasury tip | p2

* Update Cargo.lock

* "Update Substrate"

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: parity-processbot <>

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.60 to 1.0.61.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.60...v1.0.61

)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Companion for Substrate#7775

https://github.com/paritytech/substrate/pull/7775

* "Update Substrate"

Co-authored-by: parity-processbot <>

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.22 to 1.0.23.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.22...1.0.23

)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* point to branch

* update babe API implementations

* build

* Revert "point to branch"

This reverts commit 669fde1e.

* update Cargo.lock

* bump

This pr adds support to change the session length of a Rococo chain at
genesis. This is rather useful because Rococo has a session length of
1 hour, while on rococo-local you will now get 1 minute. This improves
the dev experience, because a parachain is only going live at the
start of a new session.

* guide: ensure max-per-core leads to creation of extra, semi-useless cores

* alter behavior of max_validators_per_core

* guide fixes

* Don't initialize block when calling runtime APIs

* Adapt check_validation_outputs

We split the code path for the inclusion and for the commitments checking.

* Slap #[skip_initialize_block] on safe runtime APIs

That is, those that should not be affected by this attribute

* Make `Scheduled` not ephemeral

So that it is persisted in the storage and ready to be inspected
by the runtime APIs. This is in contrast to what was before, where we
would remove the storage entry and then rely on the scheduling performed
by `on_initialize` again.

* Add a big fat comment

* Typos

Co-authored-by: Robert Habermeier <[email protected]>

* Move session change to the end of the current block

Previously, it was the beginning of the next block. This allows us to
put #[skip_initialize_block]

* Update tests

* Fix a test in paras registrar

Also refactor it a bit so the next time there are more chances this kind
of issue is diagnosed quicker.

* Add for_runtime_api to inclusion's check_validation_outputs

Co-authored-by: Robert Habermeier <[email protected]>

* add candidate hash statement circulation span

* add relay-parent to hash-span

* Some typos and misspellings in docs I found, during my studies. (#2144)

* Fix stale link to overseer docs

* Some typos and mispellings in docs/comments

I found during studying how Polkadot works.

* Rococo V1 (#2141)

* Update to latest master and use 30 minutes sessions

* add bootnodes to chainspec

* Update Substrate

* Update chain-spec

* Update Cargo.lock

* GENESIS

* Change session length to one hour

* Bump spec_version to not fuck anything up ;)

Co-authored-by: Erin Grasmick <[email protected]>

* avoid creating duplicate unbacked spans when we see extra statements (#2145)

* improve jaeger spans for statement distribution

* tweak and add failing test for repropagation

* make a change that gets the test passing

* guide: clarify

* remove semicolon

Co-authored-by: Robert Klotzner <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Erin Grasmick <[email protected]>

* overseer: observe stalled subsystems and shut down

* notify on send_message failure as well

* Update to latest master and use 30 minutes sessions

* add bootnodes to chainspec

* Update Substrate

* Update chain-spec

* Update Cargo.lock

* GENESIS

* Change session length to one hour

* Bump spec_version to not fuck anything up ;)

Co-authored-by: Erin Grasmick <[email protected]>

* Fix stale link to overseer docs

* Some typos and mispellings in docs/comments

I found during studying how Polkadot works.

* fix bug where we over-eagerly remove backing spans for candidates we validate ourselves

* jaeger: watch importing of statements

Bumps [mick-jaeger](https://github.com/tomaka/mick-jaeger) from 0.1.1 to 0.1.2.
- [Release notes](https://github.com/tomaka/mick-jaeger/releases)
- [Commits](https://github.com/tomaka/mick-jaeger/commits

)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* guide: add candidate information to OccupiedCore

* add descriptor and hash to occupied core type

* guide: add candidate hash to inclusion

* runtime: return candidate info in core state

* bitfield signing: stop querying runtime as much

* minimize going to runtime in availability distribution

* fix availability distribution tests

* guide: remove para ID from Occupied core

* get all crates compiling

* adjust span names

* fix compile

* add better spans for unbacked candidates

* improve span names

* Fix bug and further optimizations in availability distribution

- There was a bug that resulted in only getting one candidate per block
as the candidates were put into the hashmap with the relay block hash as
key. The solution for this is to use the candidate hash and the relay
block hash as key.
- We stored received/sent messages with the candidate hash and chunk
index as key. The candidate hash wasn't required in this case, as the
messages are already stored per candidate.

* Update node/core/bitfield-signing/src/lib.rs

Co-authored-by: Robert Habermeier <[email protected]>

* Remove the reverse map

* major refactor of receipts & query_live

* finish refactoring

remove ancestory mapping,

improve relay-parent cleanup & receipts-cache cleanup,
add descriptor to `PerCandidate`

* rename and rewrite query_pending_availability

* add a bunch of consistency tests

* Add some last changes

* xy

* fz

* Make it compile again

* Fix one test

* Fix logging

* Remove some buggy code

* Make tests work again

* Move stuff around

* Remove dbg

* Remove state from test_harness

* More refactor and new test

* New test and fixes

* Move metric

* Remove "duplicated code"

* Fix tests

* New test

* Change break to continue

* Update node/core/av-store/src/lib.rs

* Update node/core/av-store/src/lib.rs

* Update node/core/bitfield-signing/src/lib.rs

Co-authored-by: Fedor Sakharov <[email protected]>

* update guide to match live_candidates changes

* add comment

* fix bitfield signing

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Bernhard Schuster <[email protected]>
Co-authored-by: Fedor Sakharov <[email protected]>