- Mar 21, 2024
-
-
gupnik authored
Step in https://github.com/paritytech/polkadot-sdk/issues/3688
-
- Mar 20, 2024
-
-
eskimor authored
We witnessed really poor performance on Rococo, where we ended up with 50 on-demand cores. This was due to the fact that for each core the full queue was processed. With this change full queue processing will happen way less often (most of the time complexity is O(1) or O(log(n))) and if it happens then only for one core (in expectation). Also spot price is now updated before each order to ensure economic back pressure. TODO: - [x] Implement - [x] Basic tests - [x] Add more tests (see todos) - [x] Run benchmark to confirm better performance, first results suggest > 100x faster. - [x] Write migrations - [x] Bump scale-info version and remove patch in Cargo.toml - [x] Write PR docs: on-demand performance improved, more on-demand cores are now non problematic anymore. If need by also the max queue size can be increased again. (Maybe not to 10k) Optional: Performance can be improved even more, if we called `pop_assignment_for_core()`, before calling `report_processed` (Avoid needless affinity drops). The effect gets smaller the larger the claim queue and I would only go for it, if it does not add complexity to the scheduler. --------- Co-authored-by: eskimor <[email protected]> Co-authored-by: antonva <[email protected]> Co-authored-by: command-bot <> Co-authored-by: Anton Vilhelm Ásgeirsson <[email protected]> Co-authored-by: ordian <[email protected]>
-
Tsvetomir Dimitrov authored
The PR adds two things: 1. Runtime API exposing the whole claim queue 2. Consumes the API in `collation-generation` to fetch the next scheduled `ParaEntry` for an occupied core. Related to https://github.com/paritytech/polkadot-sdk/issues/1797
-
- Mar 19, 2024
-
-
Davide Galassi authored
Introduces `CryptoBytes` type defined as: ```rust pub struct CryptoBytes<const N: usize, Tag = ()>(pub [u8; N], PhantomData<fn() -> Tag>); ``` The type implements a bunch of methods and traits which are typically expected from a byte array newtype (NOTE: some of the methods and trait implementations IMO are a bit redundant, but I decided to maintain them all to not change too much stuff in this PR) It also introduces two (generic) typical consumers of `CryptoBytes`: `PublicBytes` and `SignatureBytes`. ```rust pub struct PublicTag; pub PublicBytes<const N: usize, CryptoTag> = CryptoBytes<N, (PublicTag, CryptoTag)>; pub struct SignatureTag; pub SignatureBytes<const N: usize, CryptoTag> = CryptoBytes<N, (SignatureTag, CryptoTag)>; ``` Both of them use a tag to differentiate the two types at a higher level. Downstream specializations will further specialize using a dedicated crypto tag. For example in ECDSA: ```rust pub struct EcdsaTag; pub type Public = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>; pub type Signature = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>; ``` Overall we have a cleaner and most importantly **consistent** code for all the types involved All these details are opaque to the end user which can use `Public` and `Signature` for the cryptos as before
-
ordian authored
On top of #3302. We want the validators to upgrade first before we add changes to the collation side to send the new variants, which is why this part is extracted into a separate PR. The detection of when to send the parent head is based on the core assignments at the relay parent of the candidate. We probably want to make it more flexible in the future, but for now, it will work for a simple use case when a para always has multiple cores assigned to it. --------- Signed-off-by: Matteo Muraca <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Matteo Muraca <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Juan Ignacio Rios <[email protected]> Co-authored-by: Branislav Kontur <[email protected]> Co-authored-by: Bastian Köcher <[email protected]>
-
Matteo Muraca authored
Part of #3326 cc @Kianenigma @ggwpez @liamaharon polkadot address: 12poSUQPtcF1HUPQGY3zZu2P8emuW9YnsPduA4XG3oCEfJVp --------- Signed-off-by: Matteo Muraca <[email protected]>
-
Bastian Köcher authored
Closes: https://github.com/paritytech/polkadot-sdk/issues/3704
-
Juan Ignacio Rios authored
Currently the xcm-executor returns an `Unimplemented` error if it receives any HRMP-related instruction. What I propose here, which is what we are currently doing in our forked executor at polimec, is to introduce a trait implemented by the executor which will handle those instructions. This way, if parachains want to keep the default behavior, they just use `()` and it will return unimplemented, but they can also implement their own logic to establish HRMP channels with other chains in an automated fashion, without requiring to go through governance. Our implementation is mentioned in the [polkadot HRMP docs](https://arc.net/l/quote/hduiivbu), and it was suggested to us to submit a PR to add these changes to polkadot-sdk. --------- Co-authored-by: Branislav Kontur <[email protected]> Co-authored-by: command-bot <>
-
- Mar 18, 2024
-
-
jokess123 authored
-
- Mar 17, 2024
-
-
dependabot[bot] authored
Bumps the known_good_semver group with 3 updates: [log](https://github.com/rust-lang/log), [syn](https://github.com/dtolnay/syn) and [clap](https://github.com/clap-rs/clap). Updates `log` from 0.4.20 to 0.4.21 <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/rust-lang/log/blob/master/CHANGELOG.md">log's changelog</a>.</em></p> <blockquote> <h2>[0.4.21] - 2024-02-27</h2> <h2>What's Changed</h2> <ul> <li>Minor clippy nits by <a href="https://github.com/nyurik"><code>@nyurik</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/578">rust-lang/log#578</a></li> <li>Simplify Display impl by <a href="https://github.com/nyurik"><code>@nyurik</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/579">rust-lang/log#579</a></li> <li>Set all crates to 2021 edition by <a href="https://github.com/nyurik"><code>@nyurik</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/580">rust-lang/log#580</a></li> <li>Various changes based on review by <a href="https://github.com/Thomasdezeeuw"><code>@Thomasdezeeuw</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/583">rust-lang/log#583</a></li> <li>Fix typo in file_static() method doc by <a href="https://github.com/dimo414"><code>@dimo414</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/590">rust-lang/log#590</a></li> <li>Specialize empty key value pairs by <a href="https://github.com/EFanZh"><code>@EFanZh</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/576">rust-lang/log#576</a></li> <li>Fix incorrect lifetime in Value::to_str() by <a href="https://github.com/peterjoel"><code>@peterjoel</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/587">rust-lang/log#587</a></li> <li>Remove some API of the key-value feature by <a href="https://github.com/Thomasdezeeuw"><code>@Thomasdezeeuw</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/585">rust-lang/log#585</a></li> <li>Add logcontrol-log and log-reload by <a href="https://github.com/swsnr"><code>@swsnr</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/595">rust-lang/log#595</a></li> <li>Add Serialization section to kv::Value docs by <a href="https://github.com/Thomasdezeeuw"><code>@Thomasdezeeuw</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/593">rust-lang/log#593</a></li> <li>Rename Value::to_str to to_cow_str by <a href="https://github.com/Thomasdezeeuw"><code>@Thomasdezeeuw</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/592">rust-lang/log#592</a></li> <li>Clarify documentation and simplify initialization of <code>STATIC_MAX_LEVEL</code> by <a href="https://github.com/ptosi"><code>@ptosi</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/594">rust-lang/log#594</a></li> <li>Update docs to 2021 edition, test by <a href="https://github.com/nyurik"><code>@nyurik</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/577">rust-lang/log#577</a></li> <li>Add "alterable_logger" link to README.md by <a href="https://github.com/brummer-simon"><code>@brummer-simon</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/589">rust-lang/log#589</a></li> <li>Normalize line ending by <a href="https://github.com/EFanZh"><code>@EFanZh</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/602">rust-lang/log#602</a></li> <li>Remove <code>ok_or</code> in favor of <code>Option::ok_or</code> by <a href="https://github.com/AngelicosPhosphoros"><code>@AngelicosPhosphoros</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/607">rust-lang/log#607</a></li> <li>Use <code>Acquire</code> ordering for initialization check by <a href="https://github.com/AngelicosPhosphoros"><code>@AngelicosPhosphoros</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/610">rust-lang/log#610</a></li> <li>Get structured logging API ready for stabilization by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/rust-lang/log/pull/613">rust-lang/log#613</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/nyurik"><code>@nyurik</code></a> made their first contribution in <a href="https://redirect.github.com/rust-lang/log/pull/578">rust-lang/log#578</a></li> <li><a href="https://github.com/dimo414"><code>@dimo414</code></a> made their first contribution in <a href="https://redirect.github.com/rust-lang/log/pull/590">rust-lang/log#590</a></li> <li><a href="https://github.com/peterjoel"><code>@peterjoel</code></a> made their first contribution in <a href="https://redirect.github.com/rust-lang/log/pull/587">rust-lang/log#587</a></li> <li><a href="https://github.com/ptosi"><code>@ptosi</code></a> made their first contribution in <a href="https://redirect.github.com/rust-lang/log/pull/594">rust-lang/log#594</a></li> <li><a href="https://github.com/brummer-simon"><code>@brummer-simon</code></a> made their first contribution in <a href="https://redirect.github.com/rust-lang/log/pull/589">rust-lang/log#589</a></li> <li><a href="https://github.com/AngelicosPhosphoros"><code>@AngelicosPhosphoros</code></a> made their first contribution in <a href="https://redirect.github.com/rust-lang/log/pull/607">rust-lang/log#607</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/rust-lang/log/commit/3ccdc286fef3076747fe18a2a93658ea4d4ae012"><code>3ccdc28</code></a> Merge pull request <a href="https://redirect.github.com/rust-lang/log/issues/617">#617</a> from rust-lang/cargo/0.4.21</li> <li><a href="https://github.com/rust-lang/log/commit/6153cb289f0e7b80f00ae07dbe5ee41cf3d3fcb0"><code>6153cb2</code></a> prepare for 0.4.21 release</li> <li><a href="https://github.com/rust-lang/log/commit/f0f74946a4bfb02cfc407795a3499c4b69d7a290"><code>f0f7494</code></a> Merge pull request <a href="https://redirect.github.com/rust-lang/log/issues/613">#613</a> from rust-lang/feat/kv-cleanup</li> <li><a href="https://github.com/rust-lang/log/commit/2b220bf3b705f2abc0ee591c7eb17972a979da3a"><code>2b220bf</code></a> clean up structured logging example</li> <li><a href="https://github.com/rust-lang/log/commit/646e9ab9917fb79e44b6b36b8375106a1a09766c"><code>646e9ab</code></a> use original Visitor name for VisitValue</li> <li><a href="https://github.com/rust-lang/log/commit/cf85c38d3519745d60e7b891c4b2025050a8389f"><code>cf85c38</code></a> add needed subfeatures to kv_unstable</li> <li><a href="https://github.com/rust-lang/log/commit/73e953905b970ef765a86bf6cbd69bc2c5e2bac4"><code>73e9539</code></a> fix up capturing of :err</li> <li><a href="https://github.com/rust-lang/log/commit/31bb4b0ff36e458c6bef304a336b71f6342ddcc7"><code>31bb4b0</code></a> move error macros together</li> <li><a href="https://github.com/rust-lang/log/commit/ad917118a5e781d0dd60b3a75ba519ce9839ba70"><code>ad91711</code></a> support field shorthand in macros</li> <li><a href="https://github.com/rust-lang/log/commit/90a347bd836873264a393a35bfd90fe478fadae2"><code>90a347b</code></a> restore removed APIs as deprecated</li> <li>Additional commits viewable in <a href="https://github.com/rust-lang/log/compare/0.4.20...0.4.21">compare view</a></li> </ul> </details> <br /> Updates `syn` from 2.0.50 to 2.0.52 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/dtolnay/syn/releases">syn's releases</a>.</em></p> <blockquote> <h2>2.0.52</h2> <ul> <li>Add an expression parser that uses match-arm's boundary rules (<a href="https://redirect.github.com/dtolnay/syn/issues/1593">#1593</a>)</li> </ul> <h2>2.0.51</h2> <ul> <li>Resolve non_local_definitions warnings in generated code under rustc 1.78-nightly</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/dtolnay/syn/commit/07ede6a6b31adeb3a18899ada1f352f63b3a36b9"><code>07ede6a</code></a> Release 2.0.52</li> <li><a href="https://github.com/dtolnay/syn/commit/acbcfbc8c113fa1603469c9ad329d061ee74662e"><code>acbcfbc</code></a> Merge pull request <a href="https://redirect.github.com/dtolnay/syn/issues/1593">#1593</a> from dtolnay/boundary</li> <li><a href="https://github.com/dtolnay/syn/commit/4924a993dce23abe65128ac318dd662d1e2ceef2"><code>4924a99</code></a> Add an expression parser that uses match-arm's boundary rules</li> <li><a href="https://github.com/dtolnay/syn/commit/e06122bf2cfd31bd7f70304694477dd292fe7e1e"><code>e06122b</code></a> Resolve unnecessary_get_then_check clippy lint</li> <li><a href="https://github.com/dtolnay/syn/commit/018fc5a6298491525387910cb359a9ec618abe54"><code>018fc5a</code></a> Update test suite to nightly-2024-02-27</li> <li><a href="https://github.com/dtolnay/syn/commit/5e15a9b412cb1e2df481e3470e1be8defaee4495"><code>5e15a9b</code></a> Release 2.0.51</li> <li><a href="https://github.com/dtolnay/syn/commit/7e0d4e1f43a879078595f0a3876484a1920ab8f8"><code>7e0d4e1</code></a> Resolve non_local_definitions warning in debug impls</li> <li><a href="https://github.com/dtolnay/syn/commit/8667ad97c1d4e75ac1bb323fb5c7849269814145"><code>8667ad9</code></a> Ignore module_name_repetitions pedantic clippy lint in codegen</li> <li><a href="https://github.com/dtolnay/syn/commit/1fc32000e25bf8fda7371071073f91e012ddf808"><code>1fc3200</code></a> Update test suite to nightly-2024-02-26</li> <li><a href="https://github.com/dtolnay/syn/commit/07a2065576b27dcf0c104f56379cc446d2f3824b"><code>07a2065</code></a> Update test suite to nightly-2024-02-23</li> <li>See full diff in <a href="https://github.com/dtolnay/syn/compare/2.0.50...2.0.52">compare view</a></li> </ul> </details> <br /> Updates `clap` from 4.5.1 to 4.5.3 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/clap-rs/clap/releases">clap's releases</a>.</em></p> <blockquote> <h2>v4.5.3</h2> <h2>[4.5.3] - 2024-03-15</h2> <h3>Internal</h3> <ul> <li><em>(derive)</em> Update <code>heck</code></li> </ul> <h2>v4.5.2</h2> <h2>[4.5.2] - 2024-03-06</h2> <h3>Fixes</h3> <ul> <li><em>(macros)</em> Silence a warning</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's changelog</a>.</em></p> <blockquote> <h2>[4.5.3] - 2024-03-15</h2> <h3>Internal</h3> <ul> <li><em>(derive)</em> Update <code>heck</code></li> </ul> <h2>[4.5.2] - 2024-03-06</h2> <h3>Fixes</h3> <ul> <li><em>(macros)</em> Silence a warning</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/clap-rs/clap/commit/4e07b438584bb8a19e37599d4c5b11797bec5579"><code>4e07b43</code></a> chore: Release</li> <li><a href="https://github.com/clap-rs/clap/commit/8247c7ddf05d8023729ac180d8e8df260f1da5ff"><code>8247c7d</code></a> docs: Update changelog</li> <li><a href="https://github.com/clap-rs/clap/commit/677c52ce0870115845a4c42e204f6c049b81a1e7"><code>677c52c</code></a> chore: Update <code>heck</code> requirement (<a href="https://redirect.github.com/clap-rs/clap/issues/5396">#5396</a>)</li> <li><a href="https://github.com/clap-rs/clap/commit/f65d421607ba16c3175ffe76a20820f123b6c4cb"><code>f65d421</code></a> chore: Release</li> <li><a href="https://github.com/clap-rs/clap/commit/886b2729e419114bf42f1a92c66d346c81aa8f33"><code>886b272</code></a> docs: Update changelog</li> <li><a href="https://github.com/clap-rs/clap/commit/3ba429752fdb19b7a1c2e151c41d5141ad5b9295"><code>3ba4297</code></a> Merge pull request <a href="https://redirect.github.com/clap-rs/clap/issues/5386">#5386</a> from amaanq/static-var-name</li> <li><a href="https://github.com/clap-rs/clap/commit/2aea9504c4894b3bddf9cd4d2d6cba889307c157"><code>2aea950</code></a> fix: Use SCREAMING_SNAKE_CASE for static variable <code>authors</code></li> <li><a href="https://github.com/clap-rs/clap/commit/690f5557d7f25904c31ec9f2a3c3657cbb68c98e"><code>690f555</code></a> Merge pull request <a href="https://redirect.github.com/clap-rs/clap/issues/5382">#5382</a> from clap-rs/renovate/pre-commit-action-3.x</li> <li><a href="https://github.com/clap-rs/clap/commit/a2aa644368ec19026b16b870ec32dc57b325ba9b"><code>a2aa644</code></a> chore(deps): update compatible (dev) (<a href="https://redirect.github.com/clap-rs/clap/issues/5381">#5381</a>)</li> <li><a href="https://github.com/clap-rs/clap/commit/c233de53c0cca4281f444cf16d16d161bc9c3cab"><code>c233de5</code></a> chore(deps): update pre-commit/action action to v3.0.1</li> <li>Additional commits viewable in <a href="https://github.com/clap-rs/clap/compare/clap_complete-v4.5.1...v4.5.3">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
Oliver Tale-Yazdi authored
Crowdloan account should burn all funds after a crowd loan got dissolved to ensure that the account is reaped correctly. --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Francisco Aguirre <[email protected]>
-
- Mar 15, 2024
-
-
ordian authored
Fixes #3128. This introduces a new variant for the collation response from the collator that includes the parent head data. For now, collators won't send this new variant. We'll need to change the collator side of the collator protocol to detect all the cores assigned to a para and send the parent head data in the case when it's more than 1 core. - [x] validate approach - [x] check head data hash
-
Andrei Sandu authored
Extracted Benchbuilder enhancements used in https://github.com/paritytech/polkadot-sdk/pull/3644 . Might still require some work to fully support all scenarios when disputing elastic scaling parachains, but it should be useful in writing elastic scaling runtime tests. --------- Signed-off-by: Andrei Sandu <[email protected]>
-
gupnik authored
Step in https://github.com/paritytech/polkadot-sdk/issues/171 This PR removes `as [disambiguation_path]` syntax from `derive_impl` usage across the polkadot-sdk as introduced in https://github.com/paritytech/polkadot-sdk/pull/3505
-
- Mar 14, 2024
-
-
Alejandro Martinez Andres authored
This change includes new bootnodes for Paseo. _This is meant to be a silent PR_
-
Dino Pačandi authored
Make Rococo & Westend XCM's location converter `HashedDescription` more in line with Polkadot/Kusama runtimes. Co-authored-by: Muharem <[email protected]>
-
- Mar 13, 2024
-
-
Andrei Eres authored
Fixes https://github.com/paritytech/polkadot-sdk/issues/3530
-
Alexandru Gheorghe authored
This is printed every 10 minutes, I see no reason why it shouldn't be in all the logs, it would give us valuable information about what is going on with node connectivity when validators come-back to us to report issues. Signed-off-by: Alexandru Gheorghe <[email protected]>
-
georgepisaltu authored
Revert "FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280)" (#3665) This PR reverts #2280 which introduced `TransactionExtension` to replace `SignedExtension`. As a result of the discussion [here](https://github.com/paritytech/polkadot-sdk/pull/3623#issuecomment-1986789700), the changes will be reverted for now with plans to reintroduce the concept in the future. --------- Signed-off-by: georgepisaltu <[email protected]>
-
- Mar 12, 2024
-
-
Alexandru Gheorghe authored
Sometimes we see nodes printing this warning: ``` cannot query the runtime API version: Api called for an unknown Block: State already discarded for ``` The log is harmless, but let's print the api we got this for, so that we can track its call site and truly confirm it is harmless or fix it. Signed-off-by: Alexandru Gheorghe <[email protected]>
-
Koute authored
This PR adds a new PolkaVM-based executor to Substrate. - The executor can now be used to actually run a PolkaVM-based runtime, and successfully produces blocks. - The executor is always compiled-in, but is disabled by default. - The `SUBSTRATE_ENABLE_POLKAVM` environment variable must be set to `1` to enable the executor, in which case the node will accept both WASM and PolkaVM program blobs (otherwise it'll default to WASM-only). This is deliberately undocumented and not explicitly exposed anywhere (e.g. in the command line arguments, or in the API) to disincentivize anyone from enabling it in production. If/when we'll move this into production usage I'll remove the environment variable and do it "properly". - I did not use our legacy runtime allocator for the PolkaVM executor, so currently every allocation inside of the runtime will leak guest memory until that particular instance is destroyed. The idea here is that I will work on the https://github.com/polkadot-fellows/RFCs/pull/4 which will remove the need for the legacy allocator under WASM, and that will also allow us to use a proper non-leaking allocator under PolkaVM. - I also did some minor cleanups of the WASM executor and deleted some dead code. No prdocs included since this is not intended to be an end-user feature, but an unofficial experiment, and shouldn't affect any current production user. Once this is production-ready a full Polkadot Fellowship RFC will be necessary anyway.
-
- Mar 11, 2024
-
-
Andrei Eres authored
Fixes https://github.com/paritytech/polkadot-sdk/issues/3528 ```rust latency: mean_latency_ms = 30 // common sense std_dev = 2.0 // common sense n_validators = 300 // max number of validators, from chain config n_cores = 60 // 300/5 max_validators_per_core = 5 // default min_pov_size = 5120 // max max_pov_size = 5120 // max peer_bandwidth = 44040192 // from the Parity's kusama validators bandwidth = 44040192 // from the Parity's kusama validators connectivity = 90 // we need to be connected to 90-95% of peers ```
-
Tsvetomir Dimitrov authored
Fixes some typos, outdated comments and test asserts. Also uses safe math and `defensive` for arithmetic operations.
-
eskimor authored
Currently redesigning candidate data structures, noticed that this one seems dead. Co-authored-by: eskimor <[email protected]>
-
- Mar 08, 2024
-
-
cuinix authored
Signed-off-by: cuinix <[email protected]> Co-authored-by: Bastian Köcher <[email protected]>
-
Alexandru Gheorghe authored
Looking at rococo-asset-hub https://github.com/paritytech/polkadot-sdk/issues/3519 there seems to be a lot of instances where collator did not advertise their collations, while there are multiple problems there, one of it is that we are connecting and disconnecting to our assigned validators every block, because on reconnect_timeout every 4s we call connect_to_validators and that will produce 0 validators when all went well, so set_reseverd_peers called from validator discovery will disconnect all our peers. More details here: https://github.com/paritytech/polkadot-sdk/issues/3519#issuecomment-1972667343 Now, this shouldn't be a problem, but it stacks with an existing bug in our network stack where if disconnect from a peer the peer might not notice it, so it won't detect the reconnect either and it won't send us the necessary view updates, so we won't advertise the collation to it more details here: https://github.com/paritytech/polkadot-sdk/issues/3519#issuecomment-1972958276 To avoid hitting this condition that often, let's keep the peers in the reserved set for the entire duration we are allocated to a backing group. Backing group sizes(1 rococo, 3 kusama, 5 polkadot) are really small, so this shouldn't lead to that many connections. Additionally, the validators would disconnect us any way if we don't advertise anything for 4 blocks. ## TODO - [x] More testing. - [x] Confirm on rococo that this is improving the situation. (It doesn't but just because other things are going wrong there). --------- Signed-off-by: Alexandru Gheorghe <[email protected]>
-
- Mar 07, 2024
-
-
Pablo Andrés Dorado Suárez authored
Closes #376 --------- Co-authored-by: command-bot <>
-
Muharem Ismailov authored
Deprecate the `xcm::body::TREASURER_INDEX` constant and use the standard Treasury variant from the `xcm::BodyId` type instead. To align with the production runtimes: https://github.com/polkadot-fellows/runtimes/pull/149
-
André Silva authored
Moves [substrate-bip39](https://github.com/paritytech/substrate-bip39) into substrate. All git history is preserved. Dependencies have been updated to use the same version as the rest of the repo. Fixes https://github.com/paritytech/polkadot-sdk/issues/1934. --------- Co-authored-by: Maciej Hirsz <[email protected]> Co-authored-by: Maciej Hirsz <[email protected]> Co-authored-by: Gav Wood <[email protected]> Co-authored-by: Stanislav Tkach <[email protected]> Co-authored-by: Robert Habermeier <[email protected]> Co-authored-by: Pierre Krieger <[email protected]> Co-authored-by: Demi M. Obenour <[email protected]> Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: NikVolf <[email protected]> Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Benjamin Kampmann <[email protected]> Co-authored-by: Maciej Hirsz <[email protected]> Co-authored-by: cheme <[email protected]> Co-authored-by: adoerr <[email protected]> Co-authored-by: Jun Jiang <[email protected]> Co-authored-by: Dan Shields <[email protected]> Co-authored-by: Michal Kucharczyk <[email protected]>
-
Andrei Sandu authored
Changes the way we perform the random selection of backed candidates when there isn't enough room for all of them. Instead of picking individual backed candidates `apply_weight` now operates on chains of candidates. This is fully backwards compatible and relies on the node side (provisioner/prospective parachains) doing the heavy lifting and providing the candidates in the order they form a chain. The same approach can be implemented for bitfields random selection once https://github.com/paritytech/polkadot-sdk/pull/3479 is merged. The approach taken in this PR aims for reduced additional complexity at the cost of being less fair wrt how many backed candidates from each chain are picked. It favors elastic scaling parachains vs parachains not using elastic scaling, but from my perspective it should be fine as this should happen under exceptional circumstances like dispute storms. Note: to make things more fair we can consider specializing `random_sel` such that it will try to pick candidates one by one in the order provided by the provisioner such that non elastic scaling parachains have the same chance of getting a candidate backed. --------- Signed-off-by: Andrei Sandu <[email protected]>
-
- Mar 05, 2024
-
-
philoniare authored
# Description *Deletes `testing.md` file in accordance with the discussion on issue #2527.* Old references to Gurke or simnet have been removed. Fixes #2527 --------- Co-authored-by: Bastian Köcher <[email protected]>
-
- Mar 04, 2024
-
-
Gavin Wood authored
Closes #2160 First part of [Extrinsic Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415) Introduces a new trait `TransactionExtension` to replace `SignedExtension`. Introduce the idea of transactions which obey the runtime's extensions and have according Extension data (né Extra data) yet do not have hard-coded signatures. Deprecate the terminology of "Unsigned" when used for transactions/extrinsics owing to there now being "proper" unsigned transactions which obey the extension framework and "old-style" unsigned which do not. Instead we have __*General*__ for the former and __*Bare*__ for the latter. (Ultimately, the latter will be phased out as a type of transaction, and Bare will only be used for Inherents.) Types of extrinsic are now therefore: - Bare (no hardcoded signature, no Extra data; used to be known as "Unsigned") - Bare transactions (deprecated): Gossiped, validated with `ValidateUnsigned` (deprecated) and the `_bare_compat` bits of `TransactionExtension` (deprecated). - Inherents: Not gossiped, validated with `ProvideInherent`. - Extended (Extra data): Gossiped, validated via `TransactionExtension`. - Signed transactions (with a hardcoded signature). - General transactions (without a hardcoded signature). `TransactionExtension` differs from `SignedExtension` because: - A signature on the underlying transaction may validly not be present. - It may alter the origin during validation. - `pre_dispatch` is renamed to `prepare` and need not contain the checks present in `validate`. - `validate` and `prepare` is passed an `Origin` rather than a `AccountId`. - `validate` may pass arbitrary information into `prepare` via a new user-specifiable type `Val`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. It is encoded *for the entire transaction* and passed in to each extension as a new argument to `validate`. This facilitates the ability of extensions to acts as underlying crypto. There is a new `DispatchTransaction` trait which contains only default function impls and is impl'ed for any `TransactionExtension` impler. It provides several utility functions which reduce some of the tedium from using `TransactionExtension` (indeed, none of its regular functions should now need to be called directly). Three transaction version discriminator ("versions") are now permissible: - 0b000000100: Bare (used to be called "Unsigned"): contains Signature or Extra (extension data). After bare transactions are no longer supported, this will strictly identify an Inherents only. - 0b100000100: Old-school "Signed" Transaction: contains Signature and Extra (extension data). - 0b010000100: New-school "General" Transaction: contains Extra (extension data), but no Signature. For the New-school General Transaction, it becomes trivial for authors to publish extensions to the mechanism for authorizing an Origin, e.g. through new kinds of key-signing schemes, ZK proofs, pallet state, mutations over pre-authenticated origins or any combination of the above. ## Code Migration ### NOW: Getting it to build Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be accompanied by renaming your aggregate type in line with the new terminology. E.g. Before: ```rust /// The SignedExtension to the basic transaction logic. pub type SignedExtra = ( /* snip */ MySpecialSignedExtension, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>; ``` After: ```rust /// The extension to the basic transaction logic. pub type TxExtension = ( /* snip */ AsTransactionExtension<MySpecialSignedExtension>, ); /// Unchecked extrinsic type as expected by this runtime. pub type UncheckedExtrinsic = generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>; ``` You'll also need to alter any transaction building logic to add a `.into()` to make the conversion happen. E.g. Before: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let extra: SignedExtra = ( /* snip */ MySpecialSignedExtension::new(/* snip */), ); let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), extra, ) } ``` After: ```rust fn construct_extrinsic( /* snip */ ) -> UncheckedExtrinsic { let tx_ext: TxExtension = ( /* snip */ MySpecialSignedExtension::new(/* snip */).into(), ); let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap(); let signature = payload.using_encoded(|e| sender.sign(e)); UncheckedExtrinsic::new_signed( /* snip */ Signature::Sr25519(signature), tx_ext, ) } ``` ### SOON: Migrating to `TransactionExtension` Most `SignedExtension`s can be trivially converted to become a `TransactionExtension`. There are a few things to know. - Instead of a single trait like `SignedExtension`, you should now implement two traits individually: `TransactionExtensionBase` and `TransactionExtension`. - Weights are now a thing and must be provided via the new function `fn weight`. #### `TransactionExtensionBase` This trait takes care of anything which is not dependent on types specific to your runtime, most notably `Call`. - `AdditionalSigned`/`additional_signed` is renamed to `Implicit`/`implicit`. - Weight must be returned by implementing the `weight` function. If your extension is associated with a pallet, you'll probably want to do this via the pallet's existing benchmarking infrastructure. #### `TransactionExtension` Generally: - `pre_dispatch` is now `prepare` and you *should not reexecute the `validate` functionality in there*! - You don't get an account ID any more; you get an origin instead. If you need to presume an account ID, then you can use the trait function `AsSystemOriginSigner::as_system_origin_signer`. - You get an additional ticket, similar to `Pre`, called `Val`. This defines data which is passed from `validate` into `prepare`. This is important since you should not be duplicating logic from `validate` to `prepare`, you need a way of passing your working from the former into the latter. This is it. - This trait takes two type parameters: `Call` and `Context`. `Call` is the runtime call type which used to be an associated type; you can just move it to become a type parameter for your trait impl. `Context` is not currently used and you can safely implement over it as an unbounded type. - There's no `AccountId` associated type any more. Just remove it. Regarding `validate`: - You get three new parameters in `validate`; all can be ignored when migrating from `SignedExtension`. - `validate` returns a tuple on success; the second item in the tuple is the new ticket type `Self::Val` which gets passed in to `prepare`. If you use any information extracted during `validate` (off-chain and on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can pass it through with this. For the tuple's last item, just return the `origin` argument. Regarding `prepare`: - This is renamed from `pre_dispatch`, but there is one change: - FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM `validate`!! - (This is different to `SignedExtension` which was required to run the same checks in `pre_dispatch` as in `validate`.) Regarding `post_dispatch`: - Since there are no unsigned transactions handled by `TransactionExtension`, `Pre` is always defined, so the first parameter is `Self::Pre` rather than `Option<Self::Pre>`. If you make use of `SignedExtension::validate_unsigned` or `SignedExtension::pre_dispatch_unsigned`, then: - Just use the regular versions of these functions instead. - Have your logic execute in the case that the `origin` is `None`. - Ensure your transaction creation logic creates a General Transaction rather than a Bare Transaction; this means having to include all `TransactionExtension`s' data. - `ValidateUnsigned` can still be used (for now) if you need to be able to construct transactions which contain none of the extension data, however these will be phased out in stage 2 of the Transactions Horizon, so you should consider moving to an extension-centric design. ## TODO - [x] Introduce `CheckSignature` impl of `TransactionExtension` to ensure it's possible to have crypto be done wholly in a `TransactionExtension`. - [x] Deprecate `SignedExtension` and move all uses in codebase to `TransactionExtension`. - [x] `ChargeTransactionPayment` - [x] `DummyExtension` - [x] `ChargeAssetTxPayment` (asset-tx-payment) - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment) - [x] `CheckWeight` - [x] `CheckTxVersion` - [x] `CheckSpecVersion` - [x] `CheckNonce` - [x] `CheckNonZeroSender` - [x] `CheckMortality` - [x] `CheckGenesis` - [x] `CheckOnlySudoAccount` - [x] `WatchDummy` - [x] `PrevalidateAttests` - [x] `GenericSignedExtension` - [x] `SignedExtension` (chain-polkadot-bulletin) - [x] `RefundSignedExtensionAdapter` - [x] Implement `fn weight` across the board. - [ ] Go through all pre-existing extensions which assume an account signer and explicitly handle the possibility of another kind of origin. - [x] `CheckNonce` should probably succeed in the case of a non-account origin. - [x] `CheckNonZeroSender` should succeed in the case of a non-account origin. - [x] `ChargeTransactionPayment` and family should fail in the case of a non-account origin. - [ ] - [x] Fix any broken tests. --------- Signed-off-by: georgepisaltu <[email protected]> Signed-off-by: Alexandru Vasile <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Oliver Tale-Yazdi <[email protected]> Signed-off-by: Alexandru Gheorghe <[email protected]> Signed-off-by: Andrei Sandu <[email protected]> Co-authored-by: Nikhil Gupta <[email protected]> Co-authored-by: georgepisaltu <[email protected]> Co-authored-by: Chevdor <[email protected]> Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Maciej <[email protected]> Co-authored-by: Javier Viola <[email protected]> Co-authored-by: Marcin S. <[email protected]> Co-authored-by: Tsvetomir Dimitrov <[email protected]> Co-authored-by: Javier Bullrich <[email protected]> Co-authored-by: Koute <[email protected]> Co-authored-by: Adrian Catangiu <[email protected]> Co-authored-by: Vladimir Istyufeev <[email protected]> Co-authored-by: Ross Bulat <[email protected]> Co-authored-by: Gonçalo Pestana <[email protected]> Co-authored-by: Liam Aharon <[email protected]> Co-authored-by: Svyatoslav Nikolsky <[email protected]> Co-authored-by: André Silva <[email protected]> Co-authored-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: s0me0ne-unkn0wn <[email protected]> Co-authored-by: ordian <[email protected]> Co-authored-by: Sebastian Kunert <[email protected]> Co-authored-by: Aaro Altonen <[email protected]> Co-authored-by: Dmitry Markin <[email protected]> Co-authored-by: Alexandru Vasile <[email protected]> Co-authored-by: Alexander Samusev <[email protected]> Co-authored-by: Julian Eager <[email protected]> Co-authored-by: Michal Kucharczyk <[email protected]> Co-authored-by: Davide Galassi <[email protected]> Co-authored-by: Dónal Murray <[email protected]> Co-authored-by: yjh <[email protected]> Co-authored-by: Tom Mi <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Will | Paradox | ParaNodes.io <[email protected]> Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Joshy Orndorff <[email protected]> Co-authored-by: Joshy Orndorff <[email protected]> Co-authored-by: PG Herveou <[email protected]> Co-authored-by: Alexander Theißen <[email protected]> Co-authored-by: Kian Paimani <[email protected]> Co-authored-by: Juan Girini <[email protected]> Co-authored-by: bader y <[email protected]> Co-authored-by: James Wilson <[email protected]> Co-authored-by: joe petrowski <[email protected]> Co-authored-by: asynchronous rob <[email protected]> Co-authored-by: Parth <[email protected]> Co-authored-by: Andrew Jones <[email protected]> Co-authored-by: Jonathan Udd <[email protected]> Co-authored-by: Serban Iorga <[email protected]> Co-authored-by: Egor_P <[email protected]> Co-authored-by: Branislav Kontur <[email protected]> Co-authored-by: Evgeny Snitko <[email protected]> Co-authored-by: Just van Stam <[email protected]> Co-authored-by: Francisco Aguirre <[email protected]> Co-authored-by: gupnik <[email protected]> Co-authored-by: dzmitry-lahoda <[email protected]> Co-authored-by: zhiqiangxu <[email protected]> Co-authored-by: Nazar Mokrynskyi <[email protected]> Co-authored-by: Anwesh <[email protected]> Co-authored-by: cheme <[email protected]> Co-authored-by: Sam Johnson <[email protected]> Co-authored-by: kianenigma <[email protected]> Co-authored-by: Jegor Sidorenko <[email protected]> Co-authored-by: Muharem <[email protected]> Co-authored-by: joepetrowski <[email protected]> Co-authored-by: Alexandru Gheorghe <[email protected]> Co-authored-by: Gabriel Facco de Arruda <[email protected]> Co-authored-by: Squirrel <[email protected]> Co-authored-by: Andrei Sandu <[email protected]> Co-authored-by: georgepisaltu <[email protected]> Co-authored-by: command-bot <>
-
- Mar 01, 2024
-
-
Alin Dima authored
https://github.com/paritytech/polkadot-sdk/issues/3130 builds on top of https://github.com/paritytech/polkadot-sdk/pull/3160 Processes the availability cores and builds a record of how many candidates it should request from prospective-parachains and their predecessors. Tries to supply as many candidates as the runtime can back. Note that the runtime changes to back multiple candidates per para are not yet done, but this paves the way for it. The following backing/inclusion policy is assumed: 1. the runtime will never back candidates of the same para which don't form a chain with the already backed candidates. Even if the others are still pending availability. We're optimistic that they won't time out and we don't want to back parachain forks (as the complexity would be huge). 2. if a candidate is timed out of the core before being included, all of its successors occupying a core will be evicted. 3. only the candidates which are made available and form a chain starting from the on-chain para head may be included/enacted and cleared from the cores. In other words, if para head is at A and the cores are occupied by B->C->D, and B and D are made available, only B will be included and its core cleared. C and D will remain on the cores awaiting for C to be made available or timed out. As point (2) above already says, if C is timed out, D will also be dropped. 4. The runtime will deduplicate candidates which form a cycle. For example if the provisioner supplies candidates A->B->A, the runtime will only back A (as the state output will be the same) Note that if a candidate is timed out, we don't guarantee that in the next relay chain block the block author will be able to fill all of the timed out cores of the para. That increases complexity by a lot. Instead, the provisioner will supply N candidates where N is the number of candidates timed out, but doesn't include their successors which will be also deleted by the runtime. This'll be backfilled in the next relay chain block. Adjacent changes: - Also fixes: https://github.com/paritytech/polkadot-sdk/issues/3141 - For non prospective-parachains, don't supply multiple candidates per para (we can't have elastic scaling without prospective parachains enabled). paras_inherent should already sanitise this input but it's more efficient this way. Note: all of these changes are backwards-compatible with the non-elastic-scaling scenario (one core per para).
-
Andrei Eres authored
### What's been done - `subsystem-bench` has been split into two parts: a cli benchmark runner and a library. - The cli runner is quite simple. It just allows us to run `.yaml` based test sequences. Now it should only be used to run benchmarks during development. - The library is used in the cli runner and in regression tests. Some code is changed to make the library independent of the runner. - Added first regression tests for availability read and write that replicate existing test sequences. ### How we run regression tests - Regression tests are simply rust integration tests without the harnesses. - They should only be compiled under the `subsystem-benchmarks` feature to prevent them from running with other tests. - This doesn't work when running tests with `nextest` in CI, so additional filters have been added to the `nextest` runs. - Each benchmark run takes a different time in the beginning, so we "warm up" the tests until their CPU usage differs by only 1%. - After the warm-up, we run the benchmarks a few more times and compare the average with the exception using a precision. ### What is still wrong? - I haven't managed to set up approval voting tests. The spread of their results is too large and can't be narrowed down in a reasonable amount of time in the warm-up phase. - The tests start an unconfigurable prometheus endpoint inside, which causes errors because they use the same 9999 port. I disable it with a flag, but I think it's better to extract the endpoint launching outside the test, as we already do with `valgrind` and `pyroscope`. But we still use `prometheus` inside the tests. ### Future work * https://github.com/paritytech/polkadot-sdk/issues/3528 * https://github.com/paritytech/polkadot-sdk/issues/3529 * https://github.com/paritytech/polkadot-sdk/issues/3530 * https://github.com/paritytech/polkadot-sdk/issues/3531 --------- Co-authored-by: Alexander Samusev <[email protected]>
-
Egor_P authored
This PR backports Node version and `spec_version` bumps to `1.8.0` from the latest release and orders prdoc files related to it.
-
Francisco Aguirre authored
If an XCM execution fails or ends with leftover assets, these will be trapped. In order to claim them, a custom XCM has to be executed, with the `ClaimAsset` instruction. However, arbitrary XCM execution is not allowed everywhere yet and XCM itself is still not easy enough to use for users out there with trapped assets. This new extrinsic in `pallet-xcm` will allow these users to easily claim their assets, without concerning themselves with writing arbitrary XCMs. Part of fixing https://github.com/paritytech/polkadot-sdk/issues/3495 --------- Co-authored-by: command-bot <> Co-authored-by: Adrian Catangiu <[email protected]>
-
- Feb 29, 2024
-
-
Andrei Sandu authored
Signed-off-by: Andrei Sandu <[email protected]>
-
Alexandru Gheorghe authored
If approval was in progress we didn't actually restart it, so we end up in a situation where we distribute our assignment, but we don't distribute any approval. --------- Signed-off-by: Alexandru Gheorghe <[email protected]>
-
Tsvetomir Dimitrov authored
This PR removes `AssignmentProviderConfig` and uses the corresponding ondemand parameters from `HostConfiguration` instead. Additionally `scheduling_lookahead` and all coretime/ondemand related parameters are extracted in a separate struct - `SchedulerParams`. The most relevant commit from the PR is [this one](https://github.com/paritytech/polkadot-sdk/pull/3181/commits/830bc0f5). Fixes https://github.com/paritytech/polkadot-sdk/issues/2268 --------- Co-authored-by: command-bot <>
-
- Feb 28, 2024
-
-
Oliver Tale-Yazdi authored
This MR is the merge of https://github.com/paritytech/substrate/pull/14414 and https://github.com/paritytech/substrate/pull/14275. It implements [RFC#13](https://github.com/polkadot-fellows/RFCs/pull/13), closes https://github.com/paritytech/polkadot-sdk/issues/198. ----- This Merge request introduces three major topicals: 1. Multi-Block-Migrations 1. New pallet `poll` hook for periodic service work 1. Replacement hooks for `on_initialize` and `on_finalize` in cases where `poll` cannot be used and some more general changes to FRAME. The changes for each topical span over multiple crates. They are listed in topical order below. # 1.) Multi-Block-Migrations Multi-Block-Migrations are facilitated by creating `pallet_migrations` and configuring `System::Config::MultiBlockMigrator` to point to it. Executive picks this up and triggers one step of the migrations pallet per block. The chain is in lockdown mode for as long as an MBM is ongoing. Executive does this by polling `MultiBlockMigrator::ongoing` and not allowing any transaction in a block, if true. A MBM is defined through trait `SteppedMigration`. A condensed version looks like this: ```rust /// A migration that can proceed in multiple steps. pub trait SteppedMigration { type Cursor: FullCodec + MaxEncodedLen; type Identifier: FullCodec + MaxEncodedLen; fn id() -> Self::Identifier; fn max_steps() -> Option<u32>; fn step( cursor: Option<Self::Cursor>, meter: &mut WeightMeter, ) -> Result<Option<Self::Cursor>, SteppedMigrationError>; } ``` `pallet_migrations` can be configured with an aggregated tuple of these migrations. It then starts to migrate them one-by-one on the next runtime upgrade. Two things are important here: - 1. Doing another runtime upgrade while MBMs are ongoing is not a good idea and can lead to messed up state. - 2. **Pallet Migrations MUST BE CONFIGURED IN `System::Config`, otherwise it is not used.** The pallet supports an `UpgradeStatusHandler` that can be used to notify external logic of upgrade start/finish (for example to pause XCM dispatch). Error recovery is very limited in the case that a migration errors or times out (exceeds its `max_steps`). Currently the runtime dev can decide in `FailedMigrationHandler::failed` how to handle this. One follow-up would be to pair this with the `SafeMode` pallet and enact safe mode when an upgrade fails, to allow governance to rescue the chain. This is currently not possible, since governance is not `Mandatory`. ## Runtime API - `Core`: `initialize_block` now returns `ExtrinsicInclusionMode` to inform the Block Author whether they can push transactions. ### Integration Add it to your runtime implementation of `Core` and `BlockBuilder`: ```patch diff --git a/runtime/src/lib.rs b/runtime/src/lib.rs @@ impl_runtime_apis! { impl sp_block_builder::Core<Block> for Runtime { - fn initialize_block(header: &<Block as BlockT>::Header) { + fn initialize_block(header: &<Block as BlockT>::Header) -> RuntimeExecutiveMode { Executive::initialize_block(header) } ... } ``` # 2.) `poll` hook A new pallet hook is introduced: `poll`. `Poll` is intended to replace mostly all usage of `on_initialize`. The reason for this is that any code that can be called from `on_initialize` cannot be migrated through an MBM. Currently there is no way to statically check this; the implication is to use `on_initialize` as rarely as possible. Failing to do so can result in broken storage invariants. The implementation of the poll hook depends on the `Runtime API` changes that are explained above. # 3.) Hard-Deadline callbacks Three new callbacks are introduced and configured on `System::Config`: `PreInherents`, `PostInherents` and `PostTransactions`. These hooks are meant as replacement for `on_initialize` and `on_finalize` in cases where the code that runs cannot be moved to `poll`. The reason for this is to make the usage of HD-code (hard deadline) more explicit - again to prevent broken invariants by MBMs. # 4.) FRAME (general changes) ## `frame_system` pallet A new memorize storage item `InherentsApplied` is added. It is used by executive to track whether inherents have already been applied. Executive and can then execute the MBMs directly between inherents and transactions. The `Config` gets five new items: - `SingleBlockMigrations` this is the new way of configuring migrations that run in a single block. Previously they were defined as last generic argument of `Executive`. This shift is brings all central configuration about migrations closer into view of the developer (migrations that are configured in `Executive` will still work for now but is deprecated). - `MultiBlockMigrator` this can be configured to an engine that drives MBMs. One example would be the `pallet_migrations`. Note that this is only the engine; the exact MBMs are injected into the engine. - `PreInherents` a callback that executes after `on_initialize` but before inherents. - `PostInherents` a callback that executes after all inherents ran (including MBMs and `poll`). - `PostTransactions` in symmetry to `PreInherents`, this one is called before `on_finalize` but after all transactions. A sane default is to set all of these to `()`. Example diff suitable for any chain: ```patch @@ impl frame_system::Config for Test { type MaxConsumers = ConstU32<16>; + type SingleBlockMigrations = (); + type MultiBlockMigrator = (); + type PreInherents = (); + type PostInherents = (); + type PostTransactions = (); } ``` An overview of how the block execution now looks like is here. The same graph is also in the rust doc. <details><summary>Block Execution Flow</summary> <p> ![Screenshot 2023-12-04 at 19 11 29](https://github.com/paritytech/polkadot-sdk/assets/10380170/e88a80c4-ef11-4faa-8df5-8b33a724c054) </p> </details> ## Inherent Order Moved to https://github.com/paritytech/polkadot-sdk/pull/2154 --------------- ## TODO - [ ] Check that `try-runtime` still works - [ ] Ensure backwards compatibility with old Runtime APIs - [x] Consume weight correctly - [x] Cleanup --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Liam Aharon <[email protected]> Co-authored-by: Juan Girini <[email protected]> Co-authored-by: command-bot <> Co-authored-by: Francisco Aguirre <[email protected]> Co-authored-by: Gavin Wood <[email protected]> Co-authored-by: Bastian Köcher <[email protected]>
-