* remove para lock check for now

* fmt

* manual para lock

* expose schedule_code_upgrade and set_current_head

* extrinsics and benchmarks

* use zero

* add weights

* fix variable name

* add and fix comments

* fix weights

* add back default lock

Co-authored-by: parity-processbot <>

* Unblock offboarding for upgrading paras

* ".git/.scripts/fmt.sh" 1

Co-authored-by: command-bot <>

* Companion for #11981

* more renaming

* fmt

* fixes

* add generic type

* Companion for #11831

* fix

* revert changes

* Delete rename-outer-enum.diff

* revert

* Update run_benches_for_runtime.sh

* rename type Call & type Event

* passing tests

* fmt

* small fixes

* commit

* fix

* fmt

* commit

* error fixes

* fix

* small fix in test

* Update lib.rs

* Update lib.rs

* Update lib.rs

* Update lib.rs

* Update lib.rs

* Update lib.rs

* Update lib.rs

* remove RuntimeCall from pallet_grandpa

* last fix

* commit

* rename

* merge fix

* update lockfile for {"substrate"}

* cargo +nightly fmt

* fix

Co-authored-by: parity-processbot <>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Companion for paritytech/substrate#12183

* Fixes

* Fixes

* Fixes

* cargo fmt

* Fixes

* Fixes

* Fixes

* cargo fmt

* Update runtime/parachains/src/paras_inherent/mod.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>

* updates

* remove new

* fix up some stuff

* fix cargo files

* fix

* fix template

* update lockfile for {"substrate"}

* Update block_weights.rs

* remove unused

* remove unused

Co-authored-by: parity-processbot <>

* fix to latest substrate pr

* update weights

* cargo build -p polkadot-runtime-parachains

* fix xcm-builder

* fix import

* fix a bunch

* fix a bunch of weight stuff

* kusama compile

* unused

* builds

* maybe fix

* cargo test -p polkadot-runtime-parachains

* xcm simulator example

* fix tests

* xcm sim fuzz

* fix runtime tests

* remove unused

* fix integration tests

* scalar div

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Comanion for substrate#11136

Signed-off-by: koushiro <[email protected]>

* revert changes in bridge

Signed-off-by: koushiro <[email protected]>

* paras: `include_pvf_check_statement` rt bench

Resolves #4933

This PR adds a benchmark for the `include_pvf_check_statement`
dispatchable. This is a necessary step to make it work without
modifications. That enables us to proceed with testing on Versi.

This introduces 5 new benchmarks. Those measure performance of the
`include_pvf_check_statement` under 2 different conditions:

1. regular vote submission. That's the common case.
2. submission of the last vote. That happens only once and leads to a
   heavy finalization stage.

There are 2 different types of finalization (one for onboarding, one for
upgrading) and there are two outcomes: accepted and rejected. Those 4
are similar but I decided to cover them all and assign the maximum of
all 4. This is to avoid a situation when one of those paths becomes more
heavier than others and opens up an attack venue.

The regular vote submission weight is drastically different from the
submission last vote weight. That's why in case during runtime
finalization was not executed the weight consumed value will be lowered
down to the regular vote submission.

The finalization weight is proportional to the number of "causes", i.e.
the events that caused the PVF pre-checking vote in the first place, and
here we assume that the maximum number of causes is 100.

Theoretically, there is nothing that prevents an adversary to
register/upgrade to more than 100 parachains. In that case, the consumed
weight will be lower than the actual time consumed by the finalization
process. That can enable a DoS vector.

However, practically, it is not very possible. Right now it is very
expensive to call `schedule_para_initialize` because it requires a very
large lock up of funds. Moreover, finalizing a vote with 100 causes
leads to around 31ms time spent. Finalizing more will require more time.
However, finalizing with 200 causes will cause ≈62ms delay. This is not
that bad since even though we had a full block and the adversary tried
to finalize 200 causes it won't be able to even exceed the operational
extrinsic boundary of 250ms and even if so it won't make big difference.

That said, this should be addressed later on, esp. when we enable
parathreads, which will make creating causes easier. One of potential
solutions will be shifting the logic of finalization into
`on_initialize`/`on_finalize`. Another is to create a maximum number of
causes and then reject upgrades or onboardings if that was reached.

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --profile=production --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

* Fix import error

Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: Robert Klotzner <[email protected]>
Co-authored-by: Lldenaurois <[email protected]>

* remove v0 primitives from polkadot-primitives

* first pass: remove v0

* fix fallout in erasure-coding

* remove v1 primitives, consolidate to v2

* the great import update

* update runtime_api_impl_v1 to v2 as well

* guide: add `Version` request for runtime API

* add version query to runtime API

* reintroduce OldV1SessionInfo in a limited way

* Updating dependencies

* Adapting code to scale v3

* Upgrade bitvec to 1.0.0

* Fix bitvec arithmetics

* Update Cargo.lock

* Update sp-io

* Fixing the build

* Yanked scale-info 2.0.0

Co-authored-by: Bastian Köcher <[email protected]>

This commit adds a new primitive called `ParachainsCache` to manipulate
the `Parachains` storage entry in a more convenient way.

Then, on top of that, this commit changes the logic of
`initialize_para_now` so that it is identical to what is used for
initialization of onboarding.

the comments we're flipped between accepted and rejected events causing confusion, but usage of events is correct

* wip template for hrmp benchmarks

* add all of the benchmarks, first draft

* File was not saved :/

* cargo +nightly fmt

* Use configs

* add configs

* Fix rococo

* Final touches

* revert fmt changes, one last time

* Fix wrappings

* Fix a bunch of tests

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_hrmp.rs

* add to westend

* actually use everything

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_hrmp.rs

* Update runtime/parachains/src/hrmp.rs

Co-authored-by: Keith Yeung <[email protected]>

* use real weight in wnd

* reorg

* minor cleanup

* weigh some of the internal stuff as well

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_hrmp.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::hrmp --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_hrmp.rs

* add files

* Master.into()

* add validation

* fmt

* fmt

* final fixes

* all runtimes build

* undo formatting

* Update runtime/parachains/src/hrmp.rs

Co-authored-by: Zeke Mostov <[email protected]>

* non-controversial changes

* do it the parachain-way: use const instead of type configs for simplicity.

* borrow assert_storage_consistency_exhaustive

* move assert_storage_consistency_exhaustive to Pallet, so it can be reused for benchmarks as well.

* fix typo

Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Sergey Shulepov <[email protected]>
Co-authored-by: Zeke Mostov <[email protected]>

* Add `without_storage_info`

The MaxEncodedLen trait is now enforced by default in Substrate.
All pallets missing an implementation need to be marked with
`without_storage_info` now.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove `generate_storage_info`

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add more `without_storage_info`

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

if the PVF pre-checking is disabled the runtime dispatchable will reject
any attempts of submission. This is also concern the unsigned tx
validation.

Right now, the `include_pvf_check_statement` dispatchable is effectively
uncallable because of the weight set to the maximum value. If we were to
benchmark it, it would become includable in a block, but since there
will be no active votes, the dispatchable won't do anything.

However, it will execute some code, like signature validation and
querying some storage entries. To be completely safe, we can bail out
early if the `pvf_checking_enabled` config is disabled. That's what this
PR does.

In this PR, paras module emit runtime events on certain PVF pre-checking
related conditions.

Specifically, there are 3 new events in the paras module:

1. PvfCheckStarted
2. PvfCheckAccepted
3. PvfCheckRejected

All of those have identifiers for the parachain that triggered the PVF
pre-checking and the validation code that goes through the pre-checking.

The mechanics of those are as follows. Each time a new PVF is added, be
it due to onboarding or upgrading, the `PvfCheckStarted` will be
triggered. If another parachain triggers a pre-checking process for the
validation code which is already being pre-checked, another
`PvfCheckStarted` event will be triggered with the corresponding para
id.

When the PVF pre-checking voting for a PVF was finished, several
`PvfCheckAccepted/Rejected` events will be triggered: one for each para id that
was subscribed to this check (i.e. was a "cause" for it).

If the PVF pre-checking is disabled, then one can still expect these
events to be fired. Since insta PVF approval is syncronous, the
`PvfCheckStarted` will be followed by the `PvfCheckAccepted` with the
same validation code and para id.

If somebody is interested in following validation code changes for a PVF
of a parachain, they would need to subscribe to those events. I did not
supply the topics for the events, since I am not sure if that's needed
or will be used, but they can be added later if needed.

Looks like it is an artifact of prior code and now it is redundant.

This is only a module structure change: the tests module was promoted to
have its own file.

As was brought up in [here][og], we have some copy&paste comments. I
fixed them and also took liberty of fixing some other places.
Specifically, those that say "module" instead of contemporary "pallet".

[og]:
https://github.com/paritytech/polkadot/pull/4603#discussion_r776971291

Closes #3971

Read the linked issue.

Apart from that, this addresses the concern raised in this [comment] by
just adding a test. I couldn't find a clean way to reconcile a block
number delay with a PVF voting TTL, so I just resorted to rely on the
test. Should be fine for now.

[comment]: https://github.com/paritytech/polkadot/pull/4457#discussion_r770517562

This just renames a member of `HostConfiguration` from
validation_upgrade_frequency to -//-_cooldown.

As was already pointed out in #4460 the existing name is a misnomer, the
member actually represents a minimum time period between upgrades, which
is neatly expressed by a word cooldown.

I've been planning this rename already for some time and the term is
already used in paras module:

https://github.com/paritytech/polkadot/blob/1394b70d/runtime/parachains/src/paras.rs#L1568-L1574

* paras: add governance control dispatchables

Adds a couple of functions for governance control for the paras module
in the anticipation of PVF pre-checking enabling.

Specifically, this commit adds a function for pre-registering a PVF that
governance trusts enough. This function will come in handy in case there
is a parachain that does not follow the GoAhead signal. That is, does
not include https://github.com/paritytech/cumulus/pull/517.

This may be not an exhaustive list of the functions that may come in
handy. Any suggestions to add more are welcome.

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

Co-authored-by: Parity Bot <[email protected]>

Setting zero as weight may be a source of problems.

The problem is, a rogue validator can shove a lot of duplicated votes
and thus fill a block with work that may well exceed the weight limit.

As was suggested by Alexander Popiak [here][comment], this commit
checks the consistency of the configuration.

[comment]:
https://github.com/paritytech/polkadot/pull/4420#discussion_r764796519

* session_info: v2 + migration

* use primitives::v2

* use polkadot_primitives::v2

* impl primitives::v2

* fix approval-voting tests

* fix other tests

* hook storage migration up

* backwards compat (1)

* backwards compat (2)

* fmt

* fix tests

* FMT

* do not reexport v1 in v2

* fmt

* set storage version to 1

Co-authored-by: Javier Viola <[email protected]>

Basically updates the docs for the paras module.

Simply extract hardcoded log target into a const.

* pvf-precheck: Integrate PVF pre-checking into paras module

Closes #4009

This is the most of the runtime-side change needed for #3211.

Here is how it works.

The PVF pre-checking can be triggered either by an upgrade or by
onboarding (i.e. calling `schedule_para_initialize`). The PVF
pre-checking process is identified by the PVF code hash that is being
voted on. If there is already PVF pre-checking process running, then no
new PVF pre-checking process will be started. Instead, we just subscribe
to the existing one.

If there is no PVF pre-checking process running but the PVF code hash
was already saved in the storage, that necessarily means (I invite the
reviewers to double-check this invariant) that the PVF already passed
pre-checking. This is equivalent to instant approving of the PVF.

The pre-checking process can be concluded either by obtaining a
supermajority or if it expires.

Each validator checks the list of PVFs available for voting. The vote is
binary, i.e. accept or reject a given PVF. As soon as the supermajority
of votes are collected for one of the sides of the vote, the voting is
concluded in that direction and the effects of the voting are enacted.

Only validators from the active set can participate in the vote. The set
of active validators can change each session. That's why we reset the
votes each session. A voting that observed a certain number of sessions
will be rejected.

The effects of the PVF accepting depend on the operations requested it:

1. All onboardings subscribed to the approved PVF pre-checking process will
get scheduled and after passing 2 session boundaries they will be onboarded.
2. All upgrades subscribed to the approved PVF pre-checking process will
get scheduled very similarly to the existing process. Upgrades with
pre-checking are really the same process that is just delayed by the
time required for pre-checking voting. In case of instant approval the
mechanism is exactly the same. This is important from parachains
compatibility standpoint since following the delayed upgrade requires
the parachain to implement
https://github.com/paritytech/cumulus/pull/517.

In case, PVF pre-checking process was concluded with rejection, then all
the requesting operations get cancelled. For onboarding it means it gets
without movement: the lifecycle of such parachain is terminated on the
`Onboarding` state and after rejection the lifecycle is none. That in
turn means that the caller can attempt registering the parachain once
more. For upgrading it means that the upgrade process is aborted: that
flashes go-ahead signal with `Abort` flag.

Rejection leads to removing the allegedly bad validation code from the
chain storage. Among other things, this implies that the operation can
be re-requested. That allows for retrying an operation in case there was
some bug. At the same time it does not look as a DoS vector due to the
caching performed by the nodes.

PVF pre-checking can be enabled and disabled. Initially, according to
the changes in #4420, this mechanism is disabled. Triggering the PVF
pre-checking when it is disabled just means that we insta approve the
requesting operation. This should lead to the behavior being unchanged.

Follow-ups:

- expose runtime APIs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

* Review fixes

Co-authored-by: Parity Bot <[email protected]>

* remove Default from CandidateHash

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <[email protected]>

* chore: fmt

* remove backed candidate default

* Partial migration away from CandidateReceipt::default

* Remove more CandidateReceipt defaults

* fmt

* Mostly remove CommittedCandidateReceipt default usage

* Remove CommittedCandidateReceipt

* Remove more Defaults from polakdot primitives v1 + fmt

* Remove more Default from polkadot primites v1

* WIP trying to get overseer example + tests to compile

* feat: add primitives test helpers

* reduce deps of helper

* update primitive helpers

* make candidate validation compile

* fixup cargo lock

* make av-store compile

* fixup disputes coordinator tests

* test: fixup backing

* test: fixup approval voting

* fixup bitfield signing

* test: fixup runtime-api

* test: fixup availability dist

* foxi[ pverseer test]

* remove some Defaults, remove bounds from `dummy`

All `fn dummy` in primitives need to be removed anyways.
This aids in the transition.

* it's a test helper, so always use std

* test: fixup parachains runtime tests

Excluding benches.

* fix keyring

* fix paras runtime properly, no more default

* Remove fn dummy() usage from approval voting

* Move TestCandidateBuilder out of av store to test helpers

* Make candidate validation tests pass

* Make most dispute coirdinator tests pass

* Make provisioner tests work

* Make availability recovery tests work with test helpers

* Update polkadot-collator-protocol tests

* Update statement distribution tests

* Update polkadot overseer examples and tests

* Derive default for validation code so we don't break unrelated things

* Make para runtime test pass (no bench)

* Some more work

* chore: cargo fmt

* cargo fix

* avoid some Default::default

* fixup dispute coordinator test

* remove unused crate deps

* remove Default::default wherever possible, replace by dummy_* for the most part

* chore: cargo fmt

* Remove some warnings

* Remove CommittedCandidateReceipt dummy

* Remove CandidateReceipt dummy

* Remove CandidateDescriptor dummy

* Remove commented out code

* Fix para runtime tests

* chore: nightly

* Some updates to the builder

* Dynamically adjust mock head data size

* Make dispute cooridinator tests work

* Fix test candidate_backing_reorders_votes work

* +nightly-2021-10-29 fmt

* Spelling and remove a default use in builder

* Various clean up

* More small updates

* fmt

* More small updates

* Doc comments for test helpers

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs

* Update lib.rs

* review comments

* fix warnings

* fix test by using correct candidate receipt relay parent

Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: emostov <[email protected]>
Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

This PR is a part of
https://github.com/paritytech/polkadot/issues/3211.

This PR prepares ground for the following runtime changes required for
PVF pre-checking. Specifically, we do several changes here:

1. We remove `validation_code_at` and `validation_code_hash_at`. Those
   functions are not used. They were added in the early days with intent
   to use it later but turned out that we do not need them.
2. We replace `validation_code_hash_at` with just `current_code_hash`
   for the case of inclusion and candidate checking.
3. We also replace `last_code_upgrade` with a direct query into
   `FutureCodeHash` and `UpgradeRestrictionSignal`. Those in conjunction
   should replace the logic that was used for allowing/disallowing
   upgrades. This requires special attention of the reviewers.
4. Then we remove the machinery required to support those queries.
   Specifically the code related to `UseCodeAt`. We do not need it since
   we do not answer the historical queries. However, we still leave all
   the data on-chain. At some point we may clean it up, but that would
   be needed to be done with a dedicated migration which can be done as
   follow-up.
5. Some now irrelevant tests were removed and/or adapted.

407bf44a8a add missing license header (#1204)
9babb19810  Custom relay strategy (#1198)
c287872a11 fix clippy things (#1200)
3a40e62789 Expose some const value and type (#1186)
32b61476d1 increase sleep before connectingMillau (#1195)
aabe7041fa revert messages transactions mortality (#1194)
3651f4f909 Message transactions mortality (#1191)
364d6e155d Bump dependencies (#1180)
f0389acc08 cargo +nightly fmt --all (#1192)
b270b6a016 Unify error enums in substrate and ethereum clients with `thiserror` (#1094)
58c4946f74 Limit max call size of Rialto/Millau runtimes (#1187)
fd56a8cd56 Add UI to the deployment (#1047)
16f01dc736 Westend -> Millau alerts are pending before notifications are sent (#1184)
5628c11ece replace collective flip with babe randomness in Rialto (#1188)
1094a63b00 ignore another (pretty bad) RUSTSEC (#1185)
379fe323ea fix/ignore cargo deny issues (#1183)
92af5e6e64 additional log in finality relay + rephrase "failed" (#1182)
b996a3b681 Rialto parachain in test deployments (#1178)
28d9332b44 Resubmit transactions strategy for Polkadot/Kusama (#1175)
d0172c6847 Playing with CI (#1179)
fb6f42456d fix checks order when registering parachain (#1177)
ee828c005a Register-parachain subcommand of substrate-relay (#1170)
8cd2b1a112 Token swap pallet benchmarks (#1174)
bb811accb1 fix collision with westend bridge (#1172)
8d2fba70ed add token swaps to test deployments (#1169)
b6d1bdfe2c publish rialto parachain collator image (#1171)
834ae4a10a Fix OutboundLaneData types (#1159)
5ee0ea1626 copypasted -> copied (#1168)
c3bb835f18 fix spelling (#1167)
f90d041dc9 Upgrade `jsonrpsee` to v0.3 (#1051)
598c9b6d0d add some basic tests for swap tokens (#1164)
05e88c61f5 publish images when tag of specific format(e.g. v2021-09-27 + v2021-09-27-1) is published (#1166)
7f3f94a6e0 Fix CI again (#1165)
ff37de332f Move calculation relayer reward into `MessageDeliveryAndDispatchPayment` (#1153)
36fbba839b fix clippy warning (#1163)
16da44d018 explicit wasm build (#1158)
c9c8226449 Match substrate's fmt (#1148)
2fdd7f3e5e Fix/ignore clippy warnings (#1157)
43dfcc2686 Adding LookupAddress (#1156)
951eaa5582 Add rialto-parachain runtime and node (#1142)
803d266d61 Rename MessageId -> BridgeMessageId (#1152)
5f234484fc Box large arguments of GRANDPA pallet (#1154)
cf9abc1011 Fix spelling (#1150)
ab83ba2e58 Relay subcommand that performs token RLT <> MLAU token swap (#1141)
832536caf0 Polkadot <> Kusama relayers (#1122)
6d0daa8975 Add `OnMessageAccepted` callback (#1134)
5d03a20b3e Integrate token swap pallet into Millau runtime (#1099)
ea4cfa833e Adding MultiAddress type and ValidationCodeHash (#1139)
c20325a784 Add tests for `Raw` and `BridgeSendMessage` enum `Call` variants (#1125)
6d802416e2 increase pause before pining Rialto nodes (#1137)
b54fa56b62 calculate fee using full message payload (#1132)
ca5d8178f5 Add parachain pallets to rialto runtime (#1053)
9eaae4142e fix transaction resubmitter limits for Millau -> Rialto transactions (#1135)
9d4e17783c add --mandatory-headers-only cli option to complex relay (#1129)
1c5e0ec1cb Add local CI info to README (#1131)
a8e0929e14 chore: spellchecker fixes (#1130)
3b8e2118e3 set fee for importing mandatory headers to zero (#1127)
49bba9aa52 another bunch of words for spellchecker (#1128)
8a72eafef6 Increase pause before messages generation start (#1126)
1f0ba9a191 Move some associated types from relay_substrate_client::Chain to bp_runtime::Chain (#1087)
74bc1a5b54 Transactions resubmitter (#1083)
21ba001f26 log max balance drop when sending message (#1117)
638a7ddffa Code Cleaning (#1124)
be6555c51b Fix buildah logout (#1120)
87539c4a98 Format code work (#1116)
526fe7fdd7 fix spelling (#1119)
bd4ce7f241 Fix spelling (#1118)
3c1147858e added missing constants to Kusama/Polkadot primitives (#1114)
52093b22ab Fix delivery transaction estimation used by rational relayer (#1109)
77a2f2fbed Remove fund account checks from upgrade. (#1111)
824334802b Rename param and update comment (#1108)
d7784bfe06 Fix spellcheck (#1110)
0b18f5906a Refactor substrate messages source and substrate messages target (#1105)
b27240bbff fix compilation (#1107)
9697da4fe8 Emit mortal transactions from relay (#1073)
b29396c077 Change vault vars type to env vars (#1084)
35e0bbdc0c Make clippy mandatory. (#1103)
a517e8541f Remove unused deps (#1102)
873dae608a Remove unnessary deps (#1101)
13450b74ee Stored conversion rate updater (#1005)
74389829f3 [BREAKING] Migrate messages pallet to frame v2 (#1088)
424da938dd README fix (#1100)
865744c909 upgrade currency exchange pallet to frame v2 (#1097)
b5038148b3 Add missing docs (#1095)
0791e911c1 Common crate for substrate-relay (#1082)
3834c9d880 Update high-level-overview.md (#1093)
c93553face Increase the time window for messaging alerts. (#1092)
8b9cc3cecd migrate pallet-shift-session-manager to frame v2 (#1090)
dc91813c22 migrate eth PoA pallet to frame v2 (#1091)
f16bb098cc Migrate dispatch pallet  to frame v2 (#1089)
19f4325348 Bridge/This Chain Ids should be exposed as constants on pallet level. (#1085)
6381122df7 Change ChainSpec::from_genesis for Rialto and Millau chains to reflect the chain names. (#1079)
0f1d33e973 Make CI happy again (#1086)
238e65d96f fix typo (#1080)
fc008457b6 Token-swap-over-bridge pallet (#944)
3fb97fa5ef Fix full spellcheck (#1076)
eae4ed7170 fixed wrong trace (#1075)
219a0fad04 merge two weight-related loops in messages pallet (#1071)
fc85632fdb increase_message_fee depends on stored mesage size (#1066)
530f37a23b companion for https://github.com/paritytech/polkadot/pull/3507 (#1067)
53b8cba683 sc_basic_authorship=trace for millau nodes (#1074)
9874e05e98 Improve traces of message generator scripts (#1069)
7b5ee84fbb extract message_details impl into runtime common (#1070)
5a4aed5a8b refund weight for mot pruning messages (#1062)
90e3d1e111 Fix Westend -> Millau sync (#1064)
427d30ddfc When restarting client, also "restart" tokio runtime (#1065)
d47c05eeef Change get pipeline sensitive variables from Vault instead of GitLab settings (#1063)
d775a85415 use tokio reactor to execute jsonrpsee futures (#1061)
15c8cd61cb Use BABE to author blocks on Rialto (previously: Aura) (#1050)
5186293500 Allow reading suri && password override from file (#1059)
b506298262 Update jsonrpsee reference (#1049)
1734d00517 enable weight fee adjustent in Rialto/Millau (#1044)
607265afae Pay dispatch fee at target chain cli option (#1043)
ce79ef91be bump dependencies before start referencing polkadot repo (#1048)
924fa24f6d Cli option for greedy relayer + run no-losses relayer by default (#1042)
e21eba7b59 Yrong README Fixup + M1 Fixes (#1045)
20d08204a2 Confirm delivery detects when more than expected messages are confirmed (#1039)
994b846b52 pre and post dispatch weights of OnDeliveryConfirmed callback (#1040)
1dd5297e84 give real value to Rialto and Millau tokens (#1038)
035bee8715 Use real conversion rate in greedy relayer strategy (#1035)
9cfaecd0f7 fixed metrics prefix (#1037)
1d8d224937 Use kebab-case for bridge arguments (#1036)
f30a4c79a6 Shared reference to conversion rate metric value (#1034)
c34d7a5cbb estimate transaction fee (#1015)
93404b18bb change alert period from 2m to 10m for Westend -> Millau (GRANDPA or public node itself is lagging sometimes) (#1032)

git-subtree-dir: bridges
git-subtree-split: 407bf44a8a5f4e60aceef2dc755cd9ff09929ac3

* rococo-runtime: Switch to latest `construct_runtime!` syntax

Besides that it fixes pallet macro errors in other crates that popped up
because of this switch.

* FMT

* Impose new restrictions on paras init and cleanup

For upcoming PVF pre-checking feature we will need to impose a couple of
new restrictions for:

- `schedule_para_initialize`.
- `schedule_para_cleanup`.

Specifically, for the former we do not want to allow registration of
wasm blob that is empty, i.e. 0 bytes. While that currently already
does not make a lot of sense, it allows us to simplify the PVF
pre-checking logic: if this PR is deployed before the following changes
for PVF prechecking then we can be sure that no paras onboarding have to
have to go through the PVF pre-checking. In case, we deploy it
altogether this property will allow us to distingush paras that came in
before PVF pre-checking.

For `schedule_para_cleanup` we do not want to allow offboarding of paras
that are undergoing the upgrade process. While this is not a harsh
restriction this change allows us to avoid making the PVF prechecking
more complicated than it has to be.

* Add a test for schedule_para_initialize

* Link to `ParaLifecycle::is_stable` in docs.

* `schedule_para_{init,cleanup}` docs

Now they link to their original declarations in the pallet for more
details.

* move things around, add filter methods

* validator keys, modify availability bitfields according to disputes

* simplify, keep the filter -> sanitize generic for both usecases

* minor

* assure tests still work, reduce changeset

* integration

* start entropy passing

* fixins

* compile, 1 failing test

* filter with coverage

* fixins

* Update runtime/parachains/src/paras_inherent.rs

Co-authored-by: Robert Habermeier <[email protected]>

* slip of the pen

* improve test cases

* misc

* fix

* fixins

* test avoid extra into() calls in assert_noop!

* chores

* ff

* test fixup superfluous into call

* chore: pfmt

* improve apply_block_weight_limit to try to maximize the number of sufficiently backed

blocks and add extra bitfields in a round-robin fashion

* new code treats the lack of backed candidates as ok

* Use vrf based entropy

* fixup vrf random

* add warn

* slip of the pen

* fixup

* assure ordering

* rethink apply_weights

* mock

* use a closure as predicate check

* extract and use DisputedBitfield

* chore: simplify

* remove stray dbg

* chore: fmt

* address feedback

* fix test, halfway there

* stage1

* dbg stuff

* make group selection align

* fix session index

* fix wrongly returned candidates

* cleanup

* chore fmt

* fix ensure check

* make good case test work

* more tests for bitfields

* create sanitize_backed_candidates

* fixup tests

* update guide

* add check referenced in the guide

* improve weights code

* fmt

* fixins

* Update roadmap/implementers-guide/src/runtime/inclusion.md

Co-authored-by: Zeke Mostov <[email protected]>

* compiling + address review

* add comments

* fix weight calc

* address review comments and test failure

* fix

* fix: condition

* Fix random_sel function

* Fix overlength block check

* Zeke + Ladi commit for disputes filtering + integration test builder + runtime benchmarks + integration tests

* Add benchmarks for code upgrades

* Code upgrade bench; Feature gate TestWeightInfo

* Try and make CI happier

* Feature gate enter test to not(benchmarks)

* Make sure no unused imports/fn

* refactor, re-use, the beginning

* Fix issue with frame benchmarking dep compilation

* More precise feature gating for some derives

* integrate piece-wise

* foo

* fixins

* chore fmt

* fixins

* rename const generic

* Update runtime/parachains/src/paras_inherent.rs

Co-authored-by: Zeke Mostov <[email protected]>

* Fix compilation

* limit to test

* remove unused spam slots

* spellcheck

* remove a tick, fix a typo

* Add Code upgrade weights

* comment improvements + >=

Co-authored-by: Zeke Mostov <[email protected]>

* remove another tick

* Update runtime/parachains/src/paras_inherent/benchmarking.rs

Co-authored-by: Zeke Mostov <[email protected]>

* saturating fixins + some spaces

* fix

* benchmarking - preliminary results

* Add training wheels

* Refactor some early exit logic for enter

* Gracefully handle filtering bitfields & candidates (#4280)

This updates the logic for sanitize_bitfields and sanitize_backed_candidates to never error when there is an issue, but instead to simply skip the problematic items.

* Refactor inherent data weight limiting logic (#4287)

* Apply suggestions from code review

* Update runtime/parachains/src/builder.rs

Co-authored-by: Zeke Mostov <[email protected]>

* Update runtime/parachains/src/builder.rs

* Update runtime/parachains/src/paras_inherent.rs

* final pass

* Run cargo +nightly-2021-10-29 fmt

* Update implementors guide with `sanitize_*` & `enter` (#4294)

* Make spell check happier

* Make wasm runtimes compile with benchmarks enabled (#4303)

* comment stuff out, use old toml

* Seems to be working?

* Remove feature gating from builder

* Remove commented out stuff

* Remove generic from digest

* Update weight files for runtime

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Zeke Mostov <[email protected]>
Co-authored-by: Lldenaurois <[email protected]>
Co-authored-by: Zeke Mostov <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Introduce new Runtime API endpoint

`persisted_validation_data_with_code_hash` that will be used
by the candidate validation subsystem in order to decrease amount
of runtime API requests.

* Node-side part of new runtime API request

* Define code hash getter via macro

* Rename new endpoint to `assumed_validation_data`

* Docs for runtime API impl of new endpoint

* AssumedValidationData specialized request function

* fmt

* Add benchmarking to rococo; Remove weights from runtime_parachains

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs --header=./file_header.txt

* Impl TestWeightInfo for Paras and Configuration

* fmt

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::configuration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_configuration.rs --header=./file_header.txt

* Fix CI complaint: error: unused variable: `c`

* polkadot-runtime-common & polkadot-test-runtime compile

* xcm-simulator compile

* change TestWeightInfo to max_block

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs --header=./file_header.txt

* ordering

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs --header=./file_header.txt

* Replace max_block with Weight::MAX

Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>

* Crate basic barebones benchmarking infrastructure for paras

* Fill in benchmarking contents

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* Use autogenerated WeightInfos for kusama and westend

* cargo fmt

* Use saturating_sub

* Add missing import

* Try and hit the worst possible time complexity as much as possible

* cargo fmt

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* Add a MAX_HEAD_DATA_SIZE constant

* Prefill vectors with sample data for worst case complexity

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* Improve comment on SAMPLE_SIZE constant

Co-authored-by: Parity Bot <[email protected]>