This is only a module structure change: the tests module was promoted to
have its own file.

As was brought up in [here][og], we have some copy&paste comments. I
fixed them and also took liberty of fixing some other places.
Specifically, those that say "module" instead of contemporary "pallet".

[og]:
https://github.com/paritytech/polkadot/pull/4603#discussion_r776971291

Closes #3971

Read the linked issue.

Apart from that, this addresses the concern raised in this [comment] by
just adding a test. I couldn't find a clean way to reconcile a block
number delay with a PVF voting TTL, so I just resorted to rely on the
test. Should be fine for now.

[comment]: https://github.com/paritytech/polkadot/pull/4457#discussion_r770517562

This just renames a member of `HostConfiguration` from
validation_upgrade_frequency to -//-_cooldown.

As was already pointed out in #4460 the existing name is a misnomer, the
member actually represents a minimum time period between upgrades, which
is neatly expressed by a word cooldown.

I've been planning this rename already for some time and the term is
already used in paras module:

https://github.com/paritytech/polkadot/blob/1394b70d493a3b4bea7a9ae14af094081bc84456/runtime/parachains/src/paras.rs#L1568-L1574

* paras: add governance control dispatchables

Adds a couple of functions for governance control for the paras module
in the anticipation of PVF pre-checking enabling.

Specifically, this commit adds a function for pre-registering a PVF that
governance trusts enough. This function will come in handy in case there
is a parachain that does not follow the GoAhead signal. That is, does
not include https://github.com/paritytech/cumulus/pull/517

.

This may be not an exhaustive list of the functions that may come in
handy. Any suggestions to add more are welcome.

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

Co-authored-by: Parity Bot <[email protected]>

Setting zero as weight may be a source of problems.

The problem is, a rogue validator can shove a lot of duplicated votes
and thus fill a block with work that may well exceed the weight limit.

As was suggested by Alexander Popiak [here][comment], this commit
checks the consistency of the configuration.

[comment]:
https://github.com/paritytech/polkadot/pull/4420#discussion_r764796519

* session_info: v2 + migration

* use primitives::v2

* use polkadot_primitives::v2

* impl primitives::v2

* fix approval-voting tests

* fix other tests

* hook storage migration up

* backwards compat (1)

* backwards compat (2)

* fmt

* fix tests

* FMT

* do not reexport v1 in v2

* fmt

* set storage version to 1

Co-authored-by: Javier Viola <[email protected]>

Basically updates the docs for the paras module.

Simply extract hardcoded log target into a const.

* pvf-precheck: Integrate PVF pre-checking into paras module

Closes #4009

This is the most of the runtime-side change needed for #3211.

Here is how it works.

The PVF pre-checking can be triggered either by an upgrade or by
onboarding (i.e. calling `schedule_para_initialize`). The PVF
pre-checking process is identified by the PVF code hash that is being
voted on. If there is already PVF pre-checking process running, then no
new PVF pre-checking process will be started. Instead, we just subscribe
to the existing one.

If there is no PVF pre-checking process running but the PVF code hash
was already saved in the storage, that necessarily means (I invite the
reviewers to double-check this invariant) that the PVF already passed
pre-checking. This is equivalent to instant approving of the PVF.

The pre-checking process can be concluded either by obtaining a
supermajority or if it expires.

Each validator checks the list of PVFs available for voting. The vote is
binary, i.e. accept or reject a given PVF. As soon as the supermajority
of votes are collected for one of the sides of the vote, the voting is
concluded in that direction and the effects of the voting are enacted.

Only validators from the active set can participate in the vote. The set
of active validators can change each session. That's why we reset the
votes each session. A voting that observed a certain number of sessions
will be rejected.

The effects of the PVF accepting depend on the operations requested it:

1. All onboardings subscribed to the approved PVF pre-checking process will
get scheduled and after passing 2 session boundaries they will be onboarded.
2. All upgrades subscribed to the approved PVF pre-checking process will
get scheduled very similarly to the existing process. Upgrades with
pre-checking are really the same process that is just delayed by the
time required for pre-checking voting. In case of instant approval the
mechanism is exactly the same. This is important from parachains
compatibility standpoint since following the delayed upgrade requires
the parachain to implement
https://github.com/paritytech/cumulus/pull/517

.

In case, PVF pre-checking process was concluded with rejection, then all
the requesting operations get cancelled. For onboarding it means it gets
without movement: the lifecycle of such parachain is terminated on the
`Onboarding` state and after rejection the lifecycle is none. That in
turn means that the caller can attempt registering the parachain once
more. For upgrading it means that the upgrade process is aborted: that
flashes go-ahead signal with `Abort` flag.

Rejection leads to removing the allegedly bad validation code from the
chain storage. Among other things, this implies that the operation can
be re-requested. That allows for retrying an operation in case there was
some bug. At the same time it does not look as a DoS vector due to the
caching performed by the nodes.

PVF pre-checking can be enabled and disabled. Initially, according to
the changes in #4420, this mechanism is disabled. Triggering the PVF
pre-checking when it is disabled just means that we insta approve the
requesting operation. This should lead to the behavior being unchanged.

Follow-ups:

- expose runtime APIs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

* Review fixes

Co-authored-by: Parity Bot <[email protected]>

* remove Default from CandidateHash

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <[email protected]>

* chore: fmt

* remove backed candidate default

* Partial migration away from CandidateReceipt::default

* Remove more CandidateReceipt defaults

* fmt

* Mostly remove CommittedCandidateReceipt default usage

* Remove CommittedCandidateReceipt

* Remove more Defaults from polakdot primitives v1 + fmt

* Remove more Default from polkadot primites v1

* WIP trying to get overseer example + tests to compile

* feat: add primitives test helpers

* reduce deps of helper

* update primitive helpers

* make candidate validation compile

* fixup cargo lock

* make av-store compile

* fixup disputes coordinator tests

* test: fixup backing

* test: fixup approval voting

* fixup bitfield signing

* test: fixup runtime-api

* test: fixup availability dist

* foxi[ pverseer test]

* remove some Defaults, remove bounds from `dummy`

All `fn dummy` in primitives need to be removed anyways.
This aids in the transition.

* it's a test helper, so always use std

* test: fixup parachains runtime tests

Excluding benches.

* fix keyring

* fix paras runtime properly, no more default

* Remove fn dummy() usage from approval voting

* Move TestCandidateBuilder out of av store to test helpers

* Make candidate validation tests pass

* Make most dispute coirdinator tests pass

* Make provisioner tests work

* Make availability recovery tests work with test helpers

* Update polkadot-collator-protocol tests

* Update statement distribution tests

* Update polkadot overseer examples and tests

* Derive default for validation code so we don't break unrelated things

* Make para runtime test pass (no bench)

* Some more work

* chore: cargo fmt

* cargo fix

* avoid some Default::default

* fixup dispute coordinator test

* remove unused crate deps

* remove Default::default wherever possible, replace by dummy_* for the most part

* chore: cargo fmt

* Remove some warnings

* Remove CommittedCandidateReceipt dummy

* Remove CandidateReceipt dummy

* Remove CandidateDescriptor dummy

* Remove commented out code

* Fix para runtime tests

* chore: nightly

* Some updates to the builder

* Dynamically adjust mock head data size

* Make dispute cooridinator tests work

* Fix test candidate_backing_reorders_votes work

* +nightly-2021-10-29 fmt

* Spelling and remove a default use in builder

* Various clean up

* More small updates

* fmt

* More small updates

* Doc comments for test helpers

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs

* Update lib.rs

* review comments

* fix warnings

* fix test by using correct candidate receipt relay parent

Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: emostov <[email protected]>
Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

This PR is a part of
https://github.com/paritytech/polkadot/issues/3211.

This PR prepares ground for the following runtime changes required for
PVF pre-checking. Specifically, we do several changes here:

1. We remove `validation_code_at` and `validation_code_hash_at`. Those
   functions are not used. They were added in the early days with intent
   to use it later but turned out that we do not need them.
2. We replace `validation_code_hash_at` with just `current_code_hash`
   for the case of inclusion and candidate checking.
3. We also replace `last_code_upgrade` with a direct query into
   `FutureCodeHash` and `UpgradeRestrictionSignal`. Those in conjunction
   should replace the logic that was used for allowing/disallowing
   upgrades. This requires special attention of the reviewers.
4. Then we remove the machinery required to support those queries.
   Specifically the code related to `UseCodeAt`. We do not need it since
   we do not answer the historical queries. However, we still leave all
   the data on-chain. At some point we may clean it up, but that would
   be needed to be done with a dedicated migration which can be done as
   follow-up.
5. Some now irrelevant tests were removed and/or adapted.

407bf44a8a add missing license header (#1204)
9babb19810  Custom relay strategy (#1198)
c287872a11 fix clippy things (#1200)
3a40e62789 Expose some const value and type (#1186)
32b61476d1 increase sleep before connectingMillau (#1195)
aabe7041fa revert messages transactions mortality (#1194)
3651f4f909 Message transactions mortality (#1191)
364d6e155d Bump dependencies (#1180)
f0389acc08 cargo +nightly fmt --all (#1192)
b270b6a016 Unify error enums in substrate and ethereum clients with `thiserror` (#1094)
58c4946f74 Limit max call size of Rialto/Millau runtimes (#1187)
fd56a8cd56 Add UI to the deployment (#1047)
16f01dc736 Westend -> Millau alerts are pending before notifications are sent (#1184)
5628c11ece replace collective flip with babe randomness in Rialto (#1188)
1094a63b00 ignore another (pretty bad) RUSTSEC (#1185)
379fe323ea fix/ignore cargo deny issues (#1183)
92af5e6e64 additional log in finality relay + rephrase "failed" (#1182)
b996a3b681 Rialto parachain in test deployments (#1178)
28d9332b44 Resubmit transactions strategy for Polkadot/Kusama (#1175)
d0172c6847 Playing with CI (#1179)
fb6f42456d fix checks order when registering parachain (#1177)
ee828c005a Register-parachain subcommand of substrate-relay (#1170)
8cd2b1a112 Token swap pallet benchmarks (#1174)
bb811accb1 fix collision with westend bridge (#1172)
8d2fba70ed add token swaps to test deployments (#1169)
b6d1bdfe2c publish rialto parachain collator image (#1171)
834ae4a10a Fix OutboundLaneData types (#1159)
5ee0ea1626 copypasted -> copied (#1168)
c3bb835f18 fix spelling (#1167)
f90d041dc9 Upgrade `jsonrpsee` to v0.3 (#1051)
598c9b6d0d add some basic tests for swap tokens (#1164)
05e88c61f5 publish images when tag of specific format(e.g. v2021-09-27 + v2021-09-27-1) is published (#1166)
7f3f94a6e0 Fix CI again (#1165)
ff37de332f Move calculation relayer reward into `MessageDeliveryAndDispatchPayment` (#1153)
36fbba839b fix clippy warning (#1163)
16da44d018 explicit wasm build (#1158)
c9c8226449 Match substrate's fmt (#1148)
2fdd7f3e5e Fix/ignore clippy warnings (#1157)
43dfcc2686 Adding LookupAddress (#1156)
951eaa5582 Add rialto-parachain runtime and node (#1142)
803d266d61 Rename MessageId -> BridgeMessageId (#1152)
5f234484fc Box large arguments of GRANDPA pallet (#1154)
cf9abc1011 Fix spelling (#1150)
ab83ba2e58 Relay subcommand that performs token RLT <> MLAU token swap (#1141)
832536caf0 Polkadot <> Kusama relayers (#1122)
6d0daa8975 Add `OnMessageAccepted` callback (#1134)
5d03a20b3e Integrate token swap pallet into Millau runtime (#1099)
ea4cfa833e Adding MultiAddress type and ValidationCodeHash (#1139)
c20325a784 Add tests for `Raw` and `BridgeSendMessage` enum `Call` variants (#1125)
6d802416e2 increase pause before pining Rialto nodes (#1137)
b54fa56b62 calculate fee using full message payload (#1132)
ca5d8178f5 Add parachain pallets to rialto runtime (#1053)
9eaae4142e fix transaction resubmitter limits for Millau -> Rialto transactions (#1135)
9d4e17783c add --mandatory-headers-only cli option to complex relay (#1129)
1c5e0ec1cb Add local CI info to README (#1131)
a8e0929e14 chore: spellchecker fixes (#1130)
3b8e2118e3 set fee for importing mandatory headers to zero (#1127)
49bba9aa52 another bunch of words for spellchecker (#1128)
8a72eafef6 Increase pause before messages generation start (#1126)
1f0ba9a191 Move some associated types from relay_substrate_client::Chain to bp_runtime::Chain (#1087)
74bc1a5b54 Transactions resubmitter (#1083)
21ba001f26 log max balance drop when sending message (#1117)
638a7ddffa Code Cleaning (#1124)
be6555c51b Fix buildah logout (#1120)
87539c4a98 Format code work (#1116)
526fe7fdd7 fix spelling (#1119)
bd4ce7f241 Fix spelling (#1118)
3c1147858e added missing constants to Kusama/Polkadot primitives (#1114)
52093b22ab Fix delivery transaction estimation used by rational relayer (#1109)
77a2f2fbed Remove fund account checks from upgrade. (#1111)
824334802b Rename param and update comment (#1108)
d7784bfe06 Fix spellcheck (#1110)
0b18f5906a Refactor substrate messages source and substrate messages target (#1105)
b27240bbff fix compilation (#1107)
9697da4fe8 Emit mortal transactions from relay (#1073)
b29396c077 Change vault vars type to env vars (#1084)
35e0bbdc0c Make clippy mandatory. (#1103)
a517e8541f Remove unused deps (#1102)
873dae608a Remove unnessary deps (#1101)
13450b74ee Stored conversion rate updater (#1005)
74389829f3 [BREAKING] Migrate messages pallet to frame v2 (#1088)
424da938dd README fix (#1100)
865744c909 upgrade currency exchange pallet to frame v2 (#1097)
b5038148b3 Add missing docs (#1095)
0791e911c1 Common crate for substrate-relay (#1082)
3834c9d880 Update high-level-overview.md (#1093)
c93553face Increase the time window for messaging alerts. (#1092)
8b9cc3cecd migrate pallet-shift-session-manager to frame v2 (#1090)
dc91813c22 migrate eth PoA pallet to frame v2 (#1091)
f16bb098cc Migrate dispatch pallet  to frame v2 (#1089)
19f4325348 Bridge/This Chain Ids should be exposed as constants on pallet level. (#1085)
6381122df7 Change ChainSpec::from_genesis for Rialto and Millau chains to reflect the chain names. (#1079)
0f1d33e973 Make CI happy again (#1086)
238e65d96f fix typo (#1080)
fc008457b6 Token-swap-over-bridge pallet (#944)
3fb97fa5ef Fix full spellcheck (#1076)
eae4ed7170 fixed wrong trace (#1075)
219a0fad04 merge two weight-related loops in messages pallet (#1071)
fc85632fdb increase_message_fee depends on stored mesage size (#1066)
530f37a23b companion for https://github.com/paritytech/polkadot/pull/3507 (#1067)
53b8cba683 sc_basic_authorship=trace for millau nodes (#1074)
9874e05e98 Improve traces of message generator scripts (#1069)
7b5ee84fbb extract message_details impl into runtime common (#1070)
5a4aed5a8b refund weight for mot pruning messages (#1062)
90e3d1e111 Fix Westend -> Millau sync (#1064)
427d30ddfc When restarting client, also "restart" tokio runtime (#1065)
d47c05eeef Change get pipeline sensitive variables from Vault instead of GitLab settings (#1063)
d775a85415 use tokio reactor to execute jsonrpsee futures (#1061)
15c8cd61cb Use BABE to author blocks on Rialto (previously: Aura) (#1050)
5186293500 Allow reading suri && password override from file (#1059)
b506298262 Update jsonrpsee reference (#1049)
1734d00517 enable weight fee adjustent in Rialto/Millau (#1044)
607265afae Pay dispatch fee at target chain cli option (#1043)
ce79ef91be bump dependencies before start referencing polkadot repo (#1048)
924fa24f6d Cli option for greedy relayer + run no-losses relayer by default (#1042)
e21eba7b59 Yrong README Fixup + M1 Fixes (#1045)
20d08204a2 Confirm delivery detects when more than expected messages are confirmed (#1039)
994b846b52 pre and post dispatch weights of OnDeliveryConfirmed callback (#1040)
1dd5297e84 give real value to Rialto and Millau tokens (#1038)
035bee8715 Use real conversion rate in greedy relayer strategy (#1035)
9cfaecd0f7 fixed metrics prefix (#1037)
1d8d224937 Use kebab-case for bridge arguments (#1036)
f30a4c79a6 Shared reference to conversion rate metric value (#1034)
c34d7a5cbb estimate transaction fee (#1015)
93404b18bb change alert period from 2m to 10m for Westend -> Millau (GRANDPA or public node itself is lagging sometimes) (#1032)

git-subtree-dir: bridges
git-subtree-split: 407bf44a8a5f4e60aceef2dc755cd9ff09929ac3

* rococo-runtime: Switch to latest `construct_runtime!` syntax

Besides that it fixes pallet macro errors in other crates that popped up
because of this switch.

* FMT

* Impose new restrictions on paras init and cleanup

For upcoming PVF pre-checking feature we will need to impose a couple of
new restrictions for:

- `schedule_para_initialize`.
- `schedule_para_cleanup`.

Specifically, for the former we do not want to allow registration of
wasm blob that is empty, i.e. 0 bytes. While that currently already
does not make a lot of sense, it allows us to simplify the PVF
pre-checking logic: if this PR is deployed before the following changes
for PVF prechecking then we can be sure that no paras onboarding have to
have to go through the PVF pre-checking. In case, we deploy it
altogether this property will allow us to distingush paras that came in
before PVF pre-checking.

For `schedule_para_cleanup` we do not want to allow offboarding of paras
that are undergoing the upgrade process. While this is not a harsh
restriction this change allows us to avoid making the PVF prechecking
more complicated than it has to be.

* Add a test for schedule_para_initialize

* Link to `ParaLifecycle::is_stable` in docs.

* `schedule_para_{init,cleanup}` docs

Now they link to their original declarations in the pallet for more
details.

* move things around, add filter methods

* validator keys, modify availability bitfields according to disputes

* simplify, keep the filter -> sanitize generic for both usecases

* minor

* assure tests still work, reduce changeset

* integration

* start entropy passing

* fixins

* compile, 1 failing test

* filter with coverage

* fixins

* Update runtime/parachains/src/paras_inherent.rs

Co-authored-by: Robert Habermeier <[email protected]>

* slip of the pen

* improve test cases

* misc

* fix

* fixins

* test avoid extra into() calls in assert_noop!

* chores

* ff

* test fixup superfluous into call

* chore: pfmt

* improve apply_block_weight_limit to try to maximize the number of sufficiently backed

blocks and add extra bitfields in a round-robin fashion

* new code treats the lack of backed candidates as ok

* Use vrf based entropy

* fixup vrf random

* add warn

* slip of the pen

* fixup

* assure ordering

* rethink apply_weights

* mock

* use a closure as predicate check

* extract and use DisputedBitfield

* chore: simplify

* remove stray dbg

* chore: fmt

* address feedback

* fix test, halfway there

* stage1

* dbg stuff

* make group selection align

* fix session index

* fix wrongly returned candidates

* cleanup

* chore fmt

* fix ensure check

* make good case test work

* more tests for bitfields

* create sanitize_backed_candidates

* fixup tests

* update guide

* add check referenced in the guide

* improve weights code

* fmt

* fixins

* Update roadmap/implementers-guide/src/runtime/inclusion.md

Co-authored-by: Zeke Mostov <[email protected]>

* compiling + address review

* add comments

* fix weight calc

* address review comments and test failure

* fix

* fix: condition

* Fix random_sel function

* Fix overlength block check

* Zeke + Ladi commit for disputes filtering + integration test builder + runtime benchmarks + integration tests

* Add benchmarks for code upgrades

* Code upgrade bench; Feature gate TestWeightInfo

* Try and make CI happier

* Feature gate enter test to not(benchmarks)

* Make sure no unused imports/fn

* refactor, re-use, the beginning

* Fix issue with frame benchmarking dep compilation

* More precise feature gating for some derives

* integrate piece-wise

* foo

* fixins

* chore fmt

* fixins

* rename const generic

* Update runtime/parachains/src/paras_inherent.rs

Co-authored-by: Zeke Mostov <[email protected]>

* Fix compilation

* limit to test

* remove unused spam slots

* spellcheck

* remove a tick, fix a typo

* Add Code upgrade weights

* comment improvements + >=

Co-authored-by: Zeke Mostov <[email protected]>

* remove another tick

* Update runtime/parachains/src/paras_inherent/benchmarking.rs

Co-authored-by: Zeke Mostov <[email protected]>

* saturating fixins + some spaces

* fix

* benchmarking - preliminary results

* Add training wheels

* Refactor some early exit logic for enter

* Gracefully handle filtering bitfields & candidates (#4280)

This updates the logic for sanitize_bitfields and sanitize_backed_candidates to never error when there is an issue, but instead to simply skip the problematic items.

* Refactor inherent data weight limiting logic (#4287)

* Apply suggestions from code review

* Update runtime/parachains/src/builder.rs

Co-authored-by: Zeke Mostov <[email protected]>

* Update runtime/parachains/src/builder.rs

* Update runtime/parachains/src/paras_inherent.rs

* final pass

* Run cargo +nightly-2021-10-29 fmt

* Update implementors guide with `sanitize_*` & `enter` (#4294)

* Make spell check happier

* Make wasm runtimes compile with benchmarks enabled (#4303)

* comment stuff out, use old toml

* Seems to be working?

* Remove feature gating from builder

* Remove commented out stuff

* Remove generic from digest

* Update weight files for runtime

Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Zeke Mostov <[email protected]>
Co-authored-by: Lldenaurois <[email protected]>
Co-authored-by: Zeke Mostov <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Introduce new Runtime API endpoint

`persisted_validation_data_with_code_hash` that will be used
by the candidate validation subsystem in order to decrease amount
of runtime API requests.

* Node-side part of new runtime API request

* Define code hash getter via macro

* Rename new endpoint to `assumed_validation_data`

* Docs for runtime API impl of new endpoint

* AssumedValidationData specialized request function

* fmt

* Add benchmarking to rococo; Remove weights from runtime_parachains

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs --header=./file_header.txt

* Impl TestWeightInfo for Paras and Configuration

* fmt

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::configuration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_configuration.rs --header=./file_header.txt

* Fix CI complaint: error: unused variable: `c`

* polkadot-runtime-common & polkadot-test-runtime compile

* xcm-simulator compile

* change TestWeightInfo to max_block

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs --header=./file_header.txt

* ordering

* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs --header=./file_header.txt

* Replace max_block with Weight::MAX

Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>

* Crate basic barebones benchmarking infrastructure for paras

* Fill in benchmarking contents

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* Use autogenerated WeightInfos for kusama and westend

* cargo fmt

* Use saturating_sub

* Add missing import

* Try and hit the worst possible time complexity as much as possible

* cargo fmt

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* Add a MAX_HEAD_DATA_SIZE constant

* Prefill vectors with sample data for worst case complexity

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

* Improve comment on SAMPLE_SIZE constant

Co-authored-by: Parity Bot <[email protected]>

* Use beefy branch with scale-info

* Add patches

* Sprinkle some TypeInfo derives

* Add some TypeInfo deriv

* Cargo.lock

* Derive TypeInfo and skip type params for Xcm types

* Cargo.lock

* Fix up scale_info bounds attributes

* Fix up dependencies

* Use my own beefy-primitives branch

* Bump BEEFY

* Update patches

* Add some scale-info dependencies and TypeInfo derives

* More TypeInfo decoration

* Update scale-info

* Some TypeInfos and remove more Event pallet::metadata

* Moar TypeInfos

* TypeInfos galore, fix up metadata runtime API

* TypeInfo

* TypeInfos, update other runtime metadata APIs

* Fix up Kusama, comment out some `usize` QueueSize parameter types

* Remove local diener patches

* Cargo.lock

* Cargo.lock

* Update to scale-info crates.io release

* Update primitive-types branch

* Update pallet-beefy to use custom branch

* Update other parity-common deps

* Update parity-common patches

* bump a bunch of deps in parity-common

* Remove parity-common patches

* Bump finality-grandpa version

* Cargo.lock

* Update scale-info to 0.9.1

* Add recursion_limit for runtime-parachains

* Add some scale_info attributes

* Cargo.lock

* Revert finality-grandpa bump

* Cargo.lock, scale-info update

* cargo update

* Make sure using patched version of finality-grandpa

* Use patched scale-info

* Update to scale-info 0.10.0

* Update finality-grandpa

* Cargo.lock

* Update beefy deps

* Update beefy deps again

* Add scale-info dependency

* Remove deprecated pallet::metadata attributes.

* Add some missing scale-info deps and derives

* Use some variant struct call syntax

* Add missing TypeInfo impl

* Add some more TypeInfo impls

* Convert some call enum struct variant constructors

* More scale-info deps and derives

* Call enum struct variants

* TypeInfo derives

* Call enum variant structs

* scale-info deps and derives

* Call enum variant struct constructors

* Use beefy-primitives scale-info feature

* Use grandpa-bridge-gadget master branch

* Remove finality-grandpa patch

* Add missing scale_info dependency and derive

* Fix up some call variant constructors

* Add missing scale_info dependency

* Fix some test errors

* More TypeInfo derives

* More call variant structs

* Call variant structs in tests

* Cargo.lock

* Fmt

* Fix more call struct variants

* Another call struct variant

* add scale-info/std features explicitly

* More call struct variants

* Add missing scale-info dependency

* Fmt

* review: activate scale-info/std where missing

* Remove some duplicate std feature activation

* review: add scale_info bounds() attr

* More call variant structs

* Remove recursion limit

* Update beefy-primitives

* Update beefy-primitives

* Fix simnet call variant struct errors

* Fmt

* cargo update -p beefy-primitives

* Add some missing TypeInfo derives

* Fix some call variants

* Fix some call variant underscores

* Cargo.lock

* Cargo.lock

* Add missing TypeInfo derive

* Add some more missing TypeInfo derives

* Even more missing TypeInfo derives

* Add TypeInfo derives to new xcm types

* Fmt

* Cargo.lock

* Add missing TypeInfo impls

* Cargo.lock

* More missing TypeInfos

* Fixes

* Cargo.lock

* Cargo.lock

* Add TypeInfo impls to xcm v2

* Update to scale-info 1.0

* Update finality-grandpa 0.14.4, patch for now

* Update beefy

* Remove patched finality-grandpa

* Add TypeInfo impl to Outcome

* Fixes

* Call variant struct

* Call variant struct

* Fix test

* Add TypeInfo impl

* Cargo.lock

* Cargo.lock

* Cargo.lock

* git checkout master Cargo.lock

* update Substrate

* Add missing scale-info features for beefy-primitives

* Fmt

* Remove check for now

* Update beefy-primitives, removes scale-info feature

* Update beefy-primitives again

Co-authored-by: adoerr <[email protected]>
Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: thiolliere <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <[email protected]>

This is an artifact left by rustfmt which is not dare to remove the
comma being conservative.

* Introduce upgrade goahead and upgrade restriction signals

* Explicit encoding indicies for exposed enums

* typo: abscent -> absent

* Prune cooldowns as well

* Please hunspell

* cargo +nightly fmt

* add cargo-fmt check to ci

* update ci

* fmt

* fmt

* skip macro

* ignore bridges

* gotta migrate them all

* migrate rococo construct_runtime

* trigger ci

* fix warnings

* get mocks to work

* add pallet to test runtime

* comments

* calm down mr tabrizi lol

* CI: add spellcheck

* revert me

* CI: explicit command for spellchecker

* spellcheck: edit misspells

* CI: run spellcheck on diff

* spellcheck: edits

* spellcheck: edit misspells

* spellcheck: add rules

* spellcheck: mv configs

* spellcheck: more edits

* spellcheck: chore

* spellcheck: one more thing

* spellcheck: and another one

* spellcheck: seems like it doesn't get to an end

* spellcheck: new words after rebase

* spellcheck: new words appearing out of nowhere

* chore

* review edits

* more review edits

* more edits

* wonky behavior

* wonky behavior 2

* wonky behavior 3

* change git behavior

* spellcheck: another bunch of new edits

* spellcheck: new words are koming out of nowhere

* CI: finding the master

* CI: fetching master implicitly

* CI: undebug

* new errors

* a bunch of new edits

* and some more

* Update node/core/approval-voting/src/approval_db/v1/mod.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Update xcm/xcm-executor/src/assets.rs

Co-authored-by: Andronik Ordian <[email protected]>

* Apply suggestions from code review

Co-authored-by: Andronik Ordian <[email protected]>

* Suggestions from the code review

* CI: scan only changed files

Co-authored-by: Andronik Ordian <[email protected]>

* to v2 macro

* line width

* fix mock

* actually compile moxk

* compile tests

* add hooks

* origin back in mocks

* assimilate_storage

* add generic

* maybe mock compiles now

* origin between parachain and system causing problem

* change origin

* type alias as origin

* keep origin

* add aliases to tests

* remove unnecessary imports

* Parachain -> Paras

* paras test

* nvm

* use genesis build in mock

* create newtype for ValidationCodeHash

* pvf: fix tests compilation

* primitives: fix test compilation

* replace HistoricalValidationCode usages with ValidationCodeByHash

* runtime-api: tabify tests file

* update implementers guide

* remove some 'allow(unused)'

* fix historical validation code runtime API

* Relax Origin Checks in Registrar

* fix tests

* Update runtime/common/src/paras_registrar.rs

* introduce para locks

* apply a lock after upgrade

* add test

* add lock when creating crowdloan

* add disputes members to `HostConfiguration`

* revert bad rename

* code stored in para + modify CandidateDescriptor.

* WIP: digest + some more impl

* validation_code_hash in payload + check in inclusion

* check in client + refator

* tests

* fix encoding indices

* remove old todos

* fix test

* fix test

* add test

* fetch validation code inside collation-generation from the relay-chain

* HashMismatch -> PoVHashMismatch + miscompilation

* refactor, store hash when needed

* storage rename: more specific but slightly too verbose

* do not hash on candidate validation, fetch hash instead

* better test

* fix test

* guide updates

* don't panic in runtime

Co-authored-by: Robert Habermeier <[email protected]>

* Add Root functions to Paras

* note past code

* fix build

* fix test build

* compile fix and force_queue_action

* Replace 'Module' with 'Pallet'.

* "Update Substrate"

* fix babe usage

* fix benchmark

Co-authored-by: parity-processbot <>
Co-authored-by: thiolliere <[email protected]>

* TODOs

* Add auctions.rs, comment on changes needed.

* Remove cruft from slots

* Remove more from auctions.rs

* More logic drafting in slots.

* More logic in slots.rs

* patch some errors

* more fixes

* last nit

* Cleanups in slots.rs

* Cleanups in slots.rs

* patches

* make build

* crowdloan to new api

* auction compile

* Use ParaId instead of FundIndex in Crowdloan (#2303)

* use paraid instead of fundindex

* Update crowdloan.rs

* check caller is manager

* Auction tests and fix build warnings.

* Configurable origin for initiating auctions

* Remove on_finalize

* #2303 (manual merge)

* Tests for Slots

* some registrar tests

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Slots uses Registrar for CurrentChains

* swap works test

* on swap impl

* traitify parachain cleanup

* explicit lifecycle tracking for paras

* initial implementation of lifecycles and upgrades

* clean up a bit

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* Revert "explicit lifecycle tracking for paras"

This reverts commit 4177af7ba473bbd9c26bccd861793f25265b6657.

* remove lifecycle tracking from registrar

* do not include transitioning paras in identifier

* Update paras_registrar.rs

* final patches to registrar

* Fix test

* use noop in test

* clean up pending swap on deregistration

* finish registrar tests

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* Remove direct need for Slots and Registrar from Crowdloan

* Rejig things slightly

* actions queue

* Revert "actions queue"

This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.

* Traitify Auction interface.

* Mockups and initial code for Crowdloan testing

* One test...

* collapse onboarding state

* fix some crowdloan tests

* one more

* start benchmarks for auctions

* benchmark bid

* fix more crowdloan tests

* onboard and begin retirement no longer exist

* Revert "onboard and begin retirement no longer exist"

This reverts commit 2e100fd94e3540bff5f172328b5d917896f1c6fc.

* Simplify crowdloan and make it work.

* Fixes

* fix some

* finish merge fixes

* fix refund bug in auctions

* Add traits to Registrar for tests and benchmarks

* fix more auction benchmarks

* Fix TestAuctioneer

* finish crowdloan benchmarks

* start setting up full integration tests

* expand integration tests

* finish basic integration test

* add more integration tests

* begin slots benchmarks

* start paras registrar benchmarks

* fix merge

* fix tests

* clean up paras registrar

* remove println

* remove outdated cleanup config

* update benchmarks

* Add WeightInfo

* enable runtime-benchmarks feature flag

* complete swap benchmark

* add parachains and onboarding into westend

* add benchmarks and genesis

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix benchmark execution

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Use `new_raise_len` in crowdloan on_initialize

* Update paras_registrar.rs

* fix westend merge

* impl on_swap for crowdloan

* Check fund exists before create

* update for crowdloan sig

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* slots on_initialize

* use integration tests environment for benchmarks

* fix hrmp event

* auction on_initialize

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix storage name in auctions

* add auction_index to winning data

* winning data takes into account current auction index

* remove println

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Revert "add auction_index to winning data"

* PastRandomness.

* Fixes

* Use new randomness

* fix use of randomness in auctions and runtime config

* expose consts

* fix auction test

* add deposit per byte for para registration

* basic swap integration test

* make swap test more comprehensive

* Add WinningVec for easier retrieval in the front-end.

* clean up `WinningVec` at the end

* Add event for when a new best bid comes in

* Fix propagation of winners in ending period

* fix benchmarks, refund weight in dissolve

* fix unused

* remove some TODOs

* setup opaque keys for paras in westend

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* remove unused

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* back to regular runtime config

* use saturating math where user input can be

* better first slot check

* Update runtime/common/src/claims.rs

* update westend onswap impl

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Guillaume Thiolliere <[email protected]>
Co-authored-by: Parity Benchmarking Bot <[email protected]>

* guide: max_validators

* guide: new approach

* restrict validators based on configurable max

* add some tests

* fix wasm build (Vec)

* clean up warnings

* add logging

* set validator indices in tests as well

* fix common tests

* Update runtime/parachains/src/util.rs

Co-authored-by: Andronik Ordian <[email protected]>

Co-authored-by: Andronik Ordian <[email protected]>