- Mar 08, 2023
-
-
s0me0ne-unkn0wn authored
* Move PVF timeouts to executor environment parameters * Typo Co-authored-by:
Marcin S. <[email protected]> * Fix comments * Change handle_import_statements to FatalResult (#6820) * Changing dispute db errors to fatal * fmt * Change node-key for bootnodes (#6772) * Additional tracing in `provisioner`, `vote_selection` and `dispute-coordinator` (#6775) * Additional tracing in `provisioner`, `vote_selection` * Add `fetched_onchain_disputes` metric to provisioner * Some tracelines in dispute-coordinator TODO: cherry pick this in the initial branch!!! * Remove spammy logs * Remove some trace lines * Rename and fix things * Fix comments * Typo * Minor fixes * Add codec indexes; Remove macro --------- Co-authored-by:
Bradley Olson <[email protected]> Co-authored-by:
Petr Mensik <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]>
-
- Feb 24, 2023
-
-
Bastian Köcher authored
* Companion for Substrate#12520 https://github.com/paritytech/substrate/pull/12520 * Fix * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <>
-
- Feb 20, 2023
-
-
s0me0ne-unkn0wn authored
-
- Feb 16, 2023
-
-
Bastian Köcher authored
* Change `MaxMemorySize` to `MaxMemoryPages` We should set the max memory for the executor in pages (64KiB) and not in bytes. The wasm memory is always a multiple of a page and we should use the same terminology. * FMT * Fix warning
-
- Feb 15, 2023
-
-
s0me0ne-unkn0wn authored
* Re-apply changes without Diener, rebase to the lastest master * Cache pruning * Bit-pack InstantiationStrategy * Move ExecutorParams version inside the structure itself * Rework runtime API and executor parameters storage * Pass executor parameters through backing subsystem * Update Cargo.lock * Introduce `ExecutorParams` to approval voting subsys * Introduce `ExecutorParams` to dispute coordinator * `cargo fmt` * Simplify requests from backing subsys * Fix tests * Replace manual config cloning with `.clone()` * Move constants to module * Parametrize executor performing PVF pre-check * Fix Malus * Fix test runtime * Introduce session executor params as a constant defined by session info pallet * Use Parity SCALE codec instead of hand-crafted binary encoding * Get rid of constants; Add docs * Get rid of constants * Minor typo * Fix Malus after rebase * `cargo fmt` * Use transparent SCALE encoding instead of explicit * Clean up * Get rid of relay parent to session index mapping * Join environment type and version in a single enum element * Use default execution parameters if running an old runtime * `unwrap()` -> `expect()` * Correct API version * Constants are back in town * Use constants for execution environment types * Artifact separation, first try * Get rid of explicit version * PVF execution queue worker separation * Worker handshake * Global renaming * Minor fixes resolving discussions * Two-stage requesting of executor params to make use of runtime API cache * Proper error handling in pvf-checker * Executor params storage bootstrapping * Propagate migration to v3 network runtimes * Fix storage versioning * Ensure `ExecutorParams` serialization determinism; Add comments * Rename constants to make things a bit more deterministic Get rid of stale code * Tidy up a structure of active PVFs * Minor formatting * Fix comment * Add try-runtime hooks * Add storage version write on upgrade Co-authored-by:
Andronik <[email protected]> * Add pre- and post-upgrade assertions * Require to specify environment type; Remove redundant `impl`s * Add `ExecutorParamHash` creation from `H256` * Fix candidate validation subsys tests * Return splittable error from executor params request fn * Revert "Return splittable error from executor params request fn" This reverts commit a0b274177d8bb2f6e13c066741892ecd2e72a456. * Decompose approval voting metrics * Use more relevant errors * Minor formatting fix * Assert a valid environment type instead of checking * Fix `try-runtime` hooks * After-merge fixes * Add migration logs * Remove dead code * Fix tests * Fix tests * Back to the strongly typed implementation * Promote strong types to executor interface * Remove stale comment * Move executor params to `SessionInfo`: primitives and runtime * Move executor params to `SessionInfo`: node * Try to bump primitives and API version * Get rid of `MallocSizeOf` * Bump target API version to v4 * Make use of session index already in place * Back to v3 * Fix all the tests * Add migrations to all the runtimes * Make use of existing `SessionInfo` in approval voting subsys * Rename `TARGET` -> `LOG_TARGET` * Bump all the primitives to v3 * Fix Rococo ParachainHost API version * Use `RollingSessionWindow` to acquire `ExecutorParams` in disputes * Fix nits from discussions; add comments * Re-evaluate queue logic * Rework job assignment in execution queue * Add documentation * Use `RuntimeInfo` to obtain `SessionInfo` (with blackjack and caching) * Couple `Pvf` with `ExecutorParams` wherever possible * Put members of `PvfWithExecutorParams` under `Arc` for cheap cloning * Fix comment * Fix CI tests * Fix clippy warnings * Address nits from discussions * Add a placeholder for raw data * Fix non exhaustive match * Remove redundant reexports and fix imports * Keep only necessary semantic features, as discussed * Rework `RuntimeInfo` to support mock implementation for tests * Remove unneeded bound * `cargo fmt` * Revert "Remove unneeded bound" This reverts commit 932463f26b00ce290e1e61848eb9328632ef8a61. * Fix PVF host tests * Fix PVF checker tests * Fix overseer declarations * Simplify tests * `MAX_KEEP_WAITING` timeout based on `BACKGING_EXECUTION_TIMEOUT` * Add a unit test for varying executor parameters * Minor fixes from discussions * Add prechecking max. memory parameter (see paritytech/srlabs_findings#110) * Fix and improve a test * Remove `ExecutionEnvironment` and `RawData` * New primitives versioning in parachain host API * `disputes()` implementation for Kusama and Polkadot * Move `ExecutorParams` from `vstaging` to stable primitives * Move disputes from `vstaging` to stable implementation * Fix `try-runtime` * Fixes after merge * Move `ExecutorParams` to the bottom of `SessionInfo` * Revert "Move executor params to `SessionInfo`: primitives and runtime" This reverts commit dfcfb85fefd1c5be6c8a8f72dc09fd1809cfa9ce. * Always use fresh activated live hash in pvf precheck (re-apply 34b09a4c20de17e7926ed942cd0d657d18f743fa) * Fixing tests (broken commit) * Fix candidate validation tests * Fix PVF host test * Minor fixes * Address discussions * Restore migration * Fix `use` to only include what is needed instead of `*` * Add comment to never touch `DEFAULT_CONFIG` * Update migration to set default `ExecutorParams` for `dispute_period` sessions back * Use `earliest_stored_session` instead of calculations * Nit * Add logs * Treat any runtime error as `NotSupported` again * Always return default executor params if not available * Revert "Always return default executor params if not available" This reverts commit b58ac4482ef444c67a9852d5776550d08e312f30. * Add paritytech/substrate#9997 workaround * `cargo fmt` * Remove migration (again!) * Bump executor params to API v4 (backport from #6698) --------- Co-authored-by:
-
- Feb 14, 2023
-
-
Marcin S. authored* Refactor PVF preparation memory stats The original purpose of this change was to gate metrics that are unsupported by some systems behind conditional compilation directives (#[cfg]); see https://github.com/paritytech/polkadot/pull/6675#discussion_r1099996209. Then I started doing some random cleanups and simplifications and got a bit carried away.
🙈 The code should be overall tidier than before. Changes: - Don't register unsupported metrics (e.g. `max_rss` on non-Linux systems) - Introduce `PrepareStats` struct as an abstraction over the `Ok` values of `PrepareResult`. It is cleaner, and can be easily modified in the future. - Other small changes * Minor fixes to comments * Fix compile errors * Try to fix some Linux errors * Mep * Fix candidate-validation tests * Update docstring
-
- Feb 09, 2023
-
-
Anthony Alaribe authored
* Introduce jemalloc-stats feature flag * remove unneeded space * Update node/overseer/src/lib.rs Co-authored-by:
Marcin S. <[email protected]> * Update Cargo.toml Co-authored-by:
Bastian Köcher <[email protected]> * revert to defining collect_memory_stats inline --------- Co-authored-by:
-
- Feb 06, 2023
-
-
Marcin S. authored
* Add getrusage and memory tracker for precheck preparation * Log memory stats metrics after prechecking * Fix tests * Try to fix errors (linux-only so I'm relying on CI here) * Try to fix CI * Add module docs for `prepare/memory_stats.rs`; fix CI error * Report memory stats for all preparation jobs * Use `RUSAGE_SELF` instead of `RUSAGE_THREAD` Not sure why I did that -- was a brainfart on my end. * Revert last commit (RUSAGE_THREAD is correct) * Use exponential buckets * Use `RUSAGE_SELF` for `getrusage`; enable `max_rss` metric for MacOS * Increase poll interval * Revert "Use `RUSAGE_SELF` for `getrusage`; enable `max_rss` metric for MacOS" This reverts commit becf7a815409ab530fc61370abffcd1b97b9a777.
-
- Jan 17, 2023
-
-
Gavin Wood authored
* cargo fmt * Create benchmarks for XCM instructions introduced in v3 (#4564) * Create benchmarks for BurnAsset and ExpectAsset * Add benchmarks for ExpectOrigin and ExpectError * Add benchmarks for QueryPallet and ExpectPallet * Add benchmarks for ReportTransactStatus and ClearTransactStatus * cargo fmt * Use AllPalletsWithSystem in mocks * Update XCM generic benchmarks for westend * Remove default impls for some XCM weight functions * Fix compilation error * Add weight_args helper attribute * Remove manually written XcmWeightInfo * Parse trailing comma * Revert "Add weight_args helper attribute" This reverts commit 3b7c47a6182e1b9227036c38b406d494c3fcf6fd. * Fixes * Fixes * XCM v3: Introduce querier field into `QueryReponse` (#4732) * Introduce querier field into QueryReponse * Convert &Option<MultiLocation> to Option<&MultiLocation> &Option<T> is almost always never quite useful, most of the time it still gets converted to an Option<&T> via `as_ref`, so we should simply make functions that accept Option<&T> instead. * Fix tests * cargo fmt * Fix benchmarks * Appease spellchecker * Fix test * Fix tests * Fix test * Fix mock * Fixes * Fix tests * Add test for response queriers * Update xcm/pallet-xcm/src/lib.rs * Test for non-existence of querier Co-authored-by:
Keith Yeung <[email protected]> * Fixes * Fixes * Add `starts_with` function to `MultiLocation` and `Junctions` (#4835) * add matches_prefix function to MultiLocation and Junctions * rename matches_prefix to starts_with * remove unnecessary main in doc comment Co-authored-by:
Bastian Köcher <[email protected]> * Make use of starts_with in match_and_split Co-authored-by:
Bastian Köcher <[email protected]> Co-authored-by:
Parity Bot <[email protected]> * Fixes * Fixes * Fixes * Fixes * Formatting * Fixes * Formatting * Fixes * Fixes * Formatting * Formatting * Remove horrible names * Bump * Remove InvertLocation trait (#5092) * Remove InvertLocation trait * Remove unneeded functions * Formatting * Fixes * Remove XCMv1 (#5094) * Remove XCMv1 * Remove XCMv1 * Formatting * Fixes * Fixes * Formatting * derive serialize/deserialize for xcm primitives (#5036) * derive serialize/deserialize for xcm primitives * derive serialize/deserialize for xcm primitives * update v3 * update v2 Co-authored-by:
Gav Wood <[email protected]> * Update lock * Fixes * Add benchmarks for the ExchangeAsset instruction * `AliasOrigin` instruction stub (#5122) * AliasOrigin instruction stub * Fixes * Fixes * Update substrate * Fixes * Ensure same array length before using copy_from_slice * Fixes * Add benchmarks for the UniversalOrigin instruction * Remove unused import * Remove unused import * Add benchmarks for SetFeesMode instruction * Add benchmarks for asset (un)locking instructions * Leave AliasOrigin unbenchmarked * Fixes after merge * cargo fmt * Fixes * Fixes * Set TrustedReserves to None on both Kusama and Westend * Remove extraneous reserve_asset_deposited benchmark * Fix universal_origin benchmark * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark pallet --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs * Don't rely on skipped benchmark functions * Fixes * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark pallet --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/kusama/src/weights/xcm/pallet_xcm_benchmarks_generic.rs * Fix unused variables * Fixes * Spelling * Fixes * Fix codec index of VersionedXcm * Allows to customize how calls are dispatched from XCM (#5657) * CallDispatcher trait * fmt * unused import * fix test-runtime * remove JustDispatch type * fix typo in test-runtime * missing CallDispatcher * more missing CallDispatcher * Update comment `NoteAssetLocked` -> `NoteUnlockable` * Fixes * Fixes * Adjust MultiAssets weights based on new wild card variants * Fixes * Fixes * Fixes * Fixes * Fixes * Some late fixes for XCMv3 (#5237) * Maximise chances that trapped assets can be reclaimed * Do origin check as part of ExportMessage for security * Formatting * Fixes * Cleanup export XCM APIs * Formatting * Update xcm/src/v3/junctions.rs * UnpaidExecution instruction and associated barrier. * Tighten barriers (ClearOrigin/QueryResponse) * Allow only 1 ClearOrigin instruction in AllowTopLevelPaidExecutionFrom * Bi-directional teleport accounting * Revert other fix * Build fixes] * Tests build * Benchmark fixes Co-authored-by:
asynchronous rob <[email protected]> * Add more weight tests * cargo fmt * Create thread_local in XCM executor to limit recursion depth (#6304) * Create thread_local in XCM executor to limit recursion depth * Add unit test for recursion limit * Fix statefulness in tests * Remove panic * Use defer and environmental macro * Fix the implementation * Use nicer interface * Change ThisNetwork to AnyNetwork * Move recursion check up to top level * cargo fmt * Update comment Co-authored-by:
Branislav Kontur <[email protected]> * Fixes * Fixes * Remove topic register and instead use the topic field in XcmContext * Derive some common traits for DispatchBlobError * Fixes * cargo fmt * Fixes * Fixes * Fix comments * Fixes * Introduce WithOriginFilter and apply it as the CallDispatcher for runtimes * Fixes * Appease clippy and fixes * Fixes * Fix more clippy issues * Fixes * ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm * Add benchmark function for ExportMessage * Fix comment * Add upper limit to DownwardMessageQueues size * Add max size check for queue in can_queue_downward_message * Fixes * Make Transact runtime call configurable * Return Weight::MAX when there is no successful send XCM origin * Update substrate * Fixes * Fixes * Remove ExportMessage benchmark * Remove assertion on Transact instruction benchmark * Make reachable destination configurable in XCM pallet benchmarks * Fixes * Fixes * Remove cfg attribute in fuzzer * Fixes * Remove cfg attribute for XCM pallet in test runtime * Fixes * Use ReachableDest where possible * Fixes * Add benchmark for UnpaidExecution * Update substrate * Ensure benchmark functions pass filters * Add runtime-benchmarks feature to fuzzer * Ensure FixedRateOfFungible accounts for proof size weights * cargo fmt * Whitelist remark_with_event when runtime-benchmarks feature is enabled * Use remark_with_event for Transact benchmarks * Fix Cargo.lock * Allow up to 3 DescendOrigin instructions before UnpaidExecution * cargo fmt * Edit code comment * Check check_origin for unpaid execution privilege * Fixes * Small nits for xcm-v3 (#6408) * Add possibility to skip benchmark for export_message * ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic * Revert * ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic * Add HaulBlobError to `fn haul_blob` * ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic Co-authored-by: command-bot <> * Revert changes to UnpaidExecution * Change AllowUnpaidExecutionFrom to be explicit * Fix log text * cargo fmt * Add benchmarks for XCM pallet version migration (#6448) * Add benchmarks for XCM pallet version migration * cargo fmt * Fixes * Fixes * Fixes * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm * ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm * ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm * ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm * Fix benchmarks * Fix benchmarks * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm * ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm * ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm * ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm Co-authored-by: command-bot <> * Merge remote-tracking branch 'origin/master' into gav-xcm-v3 * Fixes * Fix comments (#6470) * Specify Ethereum networks by their chain id (#6286) Co-authored-by:
Squirrel <[email protected]> * Use for Kusama * Use WithComputedOrigin for Polkadot, Rococo and Westend * Update lock * Fix warning * Update xcm/pallet-xcm/src/tests.rs Co-authored-by:
Oliver Tale-Yazdi <[email protected]> * Update xcm/pallet-xcm/src/migration.rs Co-authored-by:
Alexander Popiak <[email protected]> Co-authored-by:
stanly-johnson <[email protected]> Co-authored-by:
nanocryk <[email protected]> Co-authored-by:
Vincent Geddes <[email protected]> Co-authored-by:
Shawn Tabrizi <[email protected]>
-
- Jan 12, 2023
-
-
Marcin S. authored
* pre-checking: Reject failed PVFs * paras: immediately reject any PVF that cannot reach a supermajority * Make the `quorum` reject condition a bit more clear semantically * Add comment * Update implementer's guide * Update a link Not related to the rest of the PR, but I randomly noticed and fixed this. * Update runtime/parachains/src/paras/tests.rs Co-authored-by:
s0me0ne-unkn0wn <[email protected]> * Remove unneeded loop * Log PVF retries using `info!` * Change retry logs to `warn!` and add preparation failure log * Log PVF execution failure * Clarify why we reject failed PVFs * Fix PVF reject runtime benchmarks Co-authored-by:
-
- Jan 11, 2023
-
-
Marcin S. authored
* pvf: Add checks for result sender when retrying preparation in tests * pvf: Fix missing execution request when retrying preparation * Update comment
-
- Jan 10, 2023
-
-
Marcin S. authored
* Minor fixes * Fix compile errors
-
Marcin S. authored
* Replace async-std with tokio in PVF subsystem * Rework workers to use `select!` instead of a mutex The improvement in code readability is more important than the thread overhead. * Remove unnecessary `fuse` * Add explanation for `expect()` * Update node/core/pvf/src/worker_common.rs Co-authored-by:
-
- Jan 04, 2023
-
-
Marcin S. authored
-
- Dec 20, 2022
-
-
Marcin S. authored
* PVF preparation: do not conflate errors + Adds some more granularity to the prepare errors. + Better distinguish whether errors occur on the host side or the worker. + Do not kill the worker if the error happened on the host side. + Do not retry preparation if the error was `Panic`. + Removes unnecessary indirection with `Selected` type. * Add missing docs, resolve TODOs * Address review comments and remove TODOs * Fix error in CI * Undo unnecessary change * Update couple of comments * Don't return error for stream shutdown * Update node/core/pvf/src/worker_common.rs
-
- Dec 06, 2022
-
-
Marcin S. authored
* Let the PVF host kill the worker on timeout * Fix comment * Fix inaccurate comments; add missing return statement * Fix a comment * Fix comment
-
- Nov 30, 2022
-
-
Marcin S. authored
* Put in skeleton logic for CPU-time-preparation Still needed: - Flesh out logic - Refactor some spots - Tests * Continue filling in logic for prepare worker CPU time changes * Fix compiler errors * Update lenience factor * Fix some clippy lints for PVF module * Fix compilation errors * Address some review comments * Add logging * Add another log * Address some review comments; change Mutex to AtomicBool * Refactor handling response bytes * Add CPU clock timeout logic for execute jobs * Properly handle AtomicBool flag * Use `Ordering::Relaxed` * Refactor thread coordination logic * Fix bug * Add some timing information to execute tests * Add section about the mitigation to the IG * minor: Change more `Ordering`s to `Relaxed` * candidate-validation: Fix build errors
-
alexgparity authored
* Add clippy config and remove .cargo from gitignore * first fixes * Clippyfied * Add clippy CI job * comment out rusty-cachier * minor * fix ci * remove DAG from check-dependent-project * add DAG to clippy Co-authored-by:alvicsam <[email protected]>
-
- Nov 28, 2022
-
-
Marcin S. authored
-
- Nov 23, 2022
-
-
Marcin S. authored
* Add PVF module documentation TODO (once the PRs land): - [ ] Document executor parametrization. - [ ] Document CPU time measurement of timeouts. * Update node/core/pvf/src/lib.rs Co-authored-by:
Andrei Sandu <[email protected]> * Clarify meaning of PVF acronym * Move PVF doc to implementer's guide * Clean up implementer's guide a bit * Add page for PVF types * pvf: Better separation between crate docs and implementer's guide * ci: Add "prevalidating" to the dictionary * ig: Remove types/chain.md The types contained therein did not exist and the file was not referenced anywhere. Co-authored-by:
-
- Nov 08, 2022
-
-
Marcin S. authored
* Fix a couple of typos * Retry failed PVF execution PVF execution that fails due to AmbiguousWorkerDeath should be retried once. This should reduce the occurrence of failures due to transient conditions. Closes #6195 * Address a couple of nits * Write tests; refactor (add `validate_candidate_with_retry`) * Update node/core/candidate-validation/src/lib.rs Co-authored-by:
eskimor <[email protected]> Co-authored-by:
-
Marcin S. authored
-
- Nov 01, 2022
-
-
Marcin S. authored
* Rename timeout consts and timeout parameter; bump leniency * Update implementor's guide with info about PVFs * Make glossary a bit easier to read * Add a note to LENIENT_PREPARATION_TIMEOUT * Remove PVF-specific section from glossary * Fix some typos
-
- Oct 26, 2022
-
-
Marcin S. authored
* Log exit status code for workers * Make log for execute job conclusion match prepare job conclusion Trace log for conclusion of prepare job: ```rs gum::debug!( target: LOG_TARGET, validation_code_hash = ?artifact_id.code_hash, ?worker, ?rip, "prepare worker concluded", ); ``` Co-authored-by: parity-processbot <>
-
- Oct 13, 2022
-
-
Marcin S. authored
* Add some documentation * Add `compilation_timeout` parameter for PVF preparation job * Update buckets in prometheus metrics * Update prepare/queue tests * Update pvf-prechecking overview in implementer docs * Fix some CI checks
-
- Sep 15, 2022
-
-
ordian authored
-
- Jun 24, 2022
-
-
Sergei Shulepov authored
* pvf: ensure enough stack space * fix typos Co-authored-by:
-
- May 30, 2022
-
-
Gavin Wood authored
* Fix warnings * Bump
-
- May 19, 2022
-
-
Koute authored
Switch to pooling copy-on-write instantiation strategy for WASM (companion for Substrate#11232) (#5337) * Switch to pooling copy-on-write instantiation strategy for WASM * Fix compilation of `polkadot-test-service` * Update comments * Move `max_memory_size` to `Semantics` * Rename `WasmInstantiationStrategy` to `WasmtimeInstantiationStrategy` * Update a safety comment * update lockfile for {"substrate"} Co-authored-by: parity-processbot <>
-
- Apr 09, 2022
-
-
Sergei Shulepov authored
The PVF host is designed to avoid spawning tasks to minimize knowledge of outer code. Using `async_std::task::spawn` (or Tokio's counterpart) deemed unacceptable, `SpawnNamed` undesirable. Instead there is only one task returned that is spawned by the candidate-validation subsystem. The tasks from the sub-components are polled by that root task. However, the way the tasks are bundled was incorrect. There was a giant select that was polling those tasks. Particularly, that implies that as soon as one of the arms of that select goes into await those sub-tasks stop getting polled. This is a recipe for a deadlock which indeed happened here. Specifically, the deadlock happened during sending messages to the execute queue by calling [`send_execute`](https://github.com/paritytech/polkadot/blob/a68d9be35656dcd96e378fd9dd3d613af754d48a/node/core/pvf/src/host.rs#L601). When the channel to the queue reaches the capacity, the control flow is suspended until the queue handles those messages. Since this code is essentially reached from [one of the select arms](https://github.com/paritytech/polkadot/blob/a68d9be35656dcd96e378fd9dd3d613af754d48a/node/core/pvf/src/host.rs#L371), the queue won't be given the control and thus no further progress can be made. This problem is solved by bundling the tasks one level higher instead, by `selecting` over those long-running tasks. We also stop treating returning from those long-running tasks as error conditions, since that can happen during legit shutdown.
-
- Mar 24, 2022
-
-
Koute authored
* Rename to BagError * Additional parameter for 'revert' command * Set aux revert param to None * Align to changes in how the WASM executor is configured in `substrate` * update lockfile for {"substrate"} * update lockfile for {"substrate"} * Update substrate * Update substrate Co-authored-by:Davide Galassi <[email protected]> Co-authored-by:
-
- Mar 15, 2022
-
-
Bernhard Schuster authored
* add some gum * bump expander * gum * fix all remaining issues * last fixup * Update node/gum/proc-macro/src/lib.rs Co-authored-by:
-
- Jan 31, 2022
-
-
sandreim authored
Signed-off-by:Andrei Sandu <[email protected]>
-
- Dec 24, 2021
-
-
cheme authored
* merge master (do not compile) * fix * lock * update lock * Update to refactoring. * runtime version * fmt * remove trie patch * remove patch * No layout alias for bridge proof. * update depupdate depss * No switch until migration. * master lock * test * test * Revert "test" This reverts commit 57325ef73332bf4b054aa4a667bb716fcf8a0d89. * Revert "test" This reverts commit ce74d0e2062806f72c0e9e9ca07b14165f43521e. * rename feature * state version as parameter, use the feature only on runtimes. * update * update to state version in runtime * state version from storage * update lockfile for substrate Co-authored-by: parity-processbot <>
-
- Dec 14, 2021
-
-
Sergey Pepyakin authored
We wanted to change niceness to accomodate the fact that some of the preparation tasks are low priority. For example, when a node sees that there is a new para was onboarded the node may start preparing right away. Since all other activities are more important, such as network I/O or validation of the backed candidates and preparation of the immediatelly needed PVFs. However, it turned out that this approach does not work: generally non-root processes can only decrease niceness and they cannot increase it to the previous value, as was assumed by the code. Apart from that, https://github.com/paritytech/polkadot/pull/4123 assumes all PVFs are prepared in the same way. Specifically, that if a PVF preparation failed before, then PVF pre-checking will also report that it was failed, even though it could happen that preparation failed due to being low-priority. In order to avoid such cases, we decided to simplify the whole preparation model. Preparation under low priority does not work well with that. Closes https://github.com/paritytech/polkadot/issues/4520
-
Koute authored
* Align PVF executor to changes in Substrate * Update to the newest `substrate`
-
- Dec 10, 2021
-
-
Pierre Krieger authored
* Companion PR for removing Prometheus metrics prefix * Was missing some metrics * Fix missing renames * Fix test * Fixes * Update test * Update Substrate * Second time * remove prefix from intergration test for zombienet * update zombienet image * Update Substrate Co-authored-by:
Javier Viola <[email protected]>
-
- Dec 09, 2021
-
-
Chris Sosnin authored
-
- Dec 06, 2021
-
-
Bernhard Schuster authored
-
- Dec 01, 2021
-
-
antonio-dropulic authored
407bf44a8a add missing license header (#1204) 9babb19810 Custom relay strategy (#1198) c287872a11 fix clippy things (#1200) 3a40e62789 Expose some const value and type (#1186) 32b61476d1 increase sleep before connectingMillau (#1195) aabe7041fa revert messages transactions mortality (#1194) 3651f4f909 Message transactions mortality (#1191) 364d6e155d Bump dependencies (#1180) f0389acc08 cargo +nightly fmt --all (#1192) b270b6a016 Unify error enums in substrate and ethereum clients with `thiserror` (#1094) 58c4946f74 Limit max call size of Rialto/Millau runtimes (#1187) fd56a8cd56 Add UI to the deployment (#1047) 16f01dc736 Westend -> Millau alerts are pending before notifications are sent (#1184) 5628c11ece replace collective flip with babe randomness in Rialto (#1188) 1094a63b00 ignore another (pretty bad) RUSTSEC (#1185) 379fe323ea fix/ignore cargo deny issues (#1183) 92af5e6e64 additional log in finality relay + rephrase "failed" (#1182) b996a3b681 Rialto parachain in test deployments (#1178) 28d9332b44 Resubmit transactions strategy for Polkadot/Kusama (#1175) d0172c6847 Playing with CI (#1179) fb6f42456d fix checks order when registering parachain (#1177) ee828c005a Register-parachain subcommand of substrate-relay (#1170) 8cd2b1a112 Token swap pallet benchmarks (#1174) bb811accb1 fix collision with westend bridge (#1172) 8d2fba70ed add token swaps to test deployments (#1169) b6d1bdfe2c publish rialto parachain collator image (#1171) 834ae4a10a Fix OutboundLaneData types (#1159) 5ee0ea1626 copypasted -> copied (#1168) c3bb835f18 fix spelling (#1167) f90d041dc9 Upgrade `jsonrpsee` to v0.3 (#1051) 598c9b6d0d add some basic tests for swap tokens (#1164) 05e88c61f5 publish images when tag of specific format(e.g. v2021-09-27 + v2021-09-27-1) is published (#1166) 7f3f94a6e0 Fix CI again (#1165) ff37de332f Move calculation relayer reward into `MessageDeliveryAndDispatchPayment` (#1153) 36fbba839b fix clippy warning (#1163) 16da44d018 explicit wasm build (#1158) c9c8226449 Match substrate's fmt (#1148) 2fdd7f3e5e Fix/ignore clippy warnings (#1157) 43dfcc2686 Adding LookupAddress (#1156) 951eaa5582 Add rialto-parachain runtime and node (#1142) 803d266d61 Rename MessageId -> BridgeMessageId (#1152) 5f234484fc Box large arguments of GRANDPA pallet (#1154) cf9abc1011 Fix spelling (#1150) ab83ba2e58 Relay subcommand that performs token RLT <> MLAU token swap (#1141) 832536caf0 Polkadot <> Kusama relayers (#1122) 6d0daa8975 Add `OnMessageAccepted` callback (#1134) 5d03a20b3e Integrate token swap pallet into Millau runtime (#1099) ea4cfa833e Adding MultiAddress type and ValidationCodeHash (#1139) c20325a784 Add tests for `Raw` and `BridgeSendMessage` enum `Call` variants (#1125) 6d802416e2 increase pause before pining Rialto nodes (#1137) b54fa56b62 calculate fee using full message payload (#1132) ca5d8178f5 Add parachain pallets to rialto runtime (#1053) 9eaae4142e fix transaction resubmitter limits for Millau -> Rialto transactions (#1135) 9d4e17783c add --mandatory-headers-only cli option to complex relay (#1129) 1c5e0ec1cb Add local CI info to README (#1131) a8e0929e14 chore: spellchecker fixes (#1130) 3b8e2118e3 set fee for importing mandatory headers to zero (#1127) 49bba9aa52 another bunch of words for spellchecker (#1128) 8a72eafef6 Increase pause before messages generation start (#1126) 1f0ba9a191 Move some associated types from relay_substrate_client::Chain to bp_runtime::Chain (#1087) 74bc1a5b54 Transactions resubmitter (#1083) 21ba001f26 log max balance drop when sending message (#1117) 638a7ddffa Code Cleaning (#1124) be6555c51b Fix buildah logout (#1120) 87539c4a98 Format code work (#1116) 526fe7fdd7 fix spelling (#1119) bd4ce7f241 Fix spelling (#1118) 3c1147858e added missing constants to Kusama/Polkadot primitives (#1114) 52093b22ab Fix delivery transaction estimation used by rational relayer (#1109) 77a2f2fbed Remove fund account checks from upgrade. (#1111) 824334802b Rename param and update comment (#1108) d7784bfe06 Fix spellcheck (#1110) 0b18f5906a Refactor substrate messages source and substrate messages target (#1105) b27240bbff fix compilation (#1107) 9697da4fe8 Emit mortal transactions from relay (#1073) b29396c077 Change vault vars type to env vars (#1084) 35e0bbdc0c Make clippy mandatory. (#1103) a517e8541f Remove unused deps (#1102) 873dae608a Remove unnessary deps (#1101) 13450b74ee Stored conversion rate updater (#1005) 74389829f3 [BREAKING] Migrate messages pallet to frame v2 (#1088) 424da938dd README fix (#1100) 865744c909 upgrade currency exchange pallet to frame v2 (#1097) b5038148b3 Add missing docs (#1095) 0791e911c1 Common crate for substrate-relay (#1082) 3834c9d880 Update high-level-overview.md (#1093) c93553face Increase the time window for messaging alerts. (#1092) 8b9cc3cecd migrate pallet-shift-session-manager to frame v2 (#1090) dc91813c22 migrate eth PoA pallet to frame v2 (#1091) f16bb098cc Migrate dispatch pallet to frame v2 (#1089) 19f4325348 Bridge/This Chain Ids should be exposed as constants on pallet level. (#1085) 6381122df7 Change ChainSpec::from_genesis for Rialto and Millau chains to reflect the chain names. (#1079) 0f1d33e973 Make CI happy again (#1086) 238e65d96f fix typo (#1080) fc008457b6 Token-swap-over-bridge pallet (#944) 3fb97fa5ef Fix full spellcheck (#1076) eae4ed7170 fixed wrong trace (#1075) 219a0fad04 merge two weight-related loops in messages pallet (#1071) fc85632fdb increase_message_fee depends on stored mesage size (#1066) 530f37a23b companion for https://github.com/paritytech/polkadot/pull/3507 (#1067) 53b8cba683 sc_basic_authorship=trace for millau nodes (#1074) 9874e05e98 Improve traces of message generator scripts (#1069) 7b5ee84fbb extract message_details impl into runtime common (#1070) 5a4aed5a8b refund weight for mot pruning messages (#1062) 90e3d1e111 Fix Westend -> Millau sync (#1064) 427d30ddfc When restarting client, also "restart" tokio runtime (#1065) d47c05eeef Change get pipeline sensitive variables from Vault instead of GitLab settings (#1063) d775a85415 use tokio reactor to execute jsonrpsee futures (#1061) 15c8cd61cb Use BABE to author blocks on Rialto (previously: Aura) (#1050) 5186293500 Allow reading suri && password override from file (#1059) b506298262 Update jsonrpsee reference (#1049) 1734d00517 enable weight fee adjustent in Rialto/Millau (#1044) 607265afae Pay dispatch fee at target chain cli option (#1043) ce79ef91be bump dependencies before start referencing polkadot repo (#1048) 924fa24f6d Cli option for greedy relayer + run no-losses relayer by default (#1042) e21eba7b59 Yrong README Fixup + M1 Fixes (#1045) 20d08204a2 Confirm delivery detects when more than expected messages are confirmed (#1039) 994b846b52 pre and post dispatch weights of OnDeliveryConfirmed callback (#1040) 1dd5297e84 give real value to Rialto and Millau tokens (#1038) 035bee8715 Use real conversion rate in greedy relayer strategy (#1035) 9cfaecd0f7 fixed metrics prefix (#1037) 1d8d224937 Use kebab-case for bridge arguments (#1036) f30a4c79a6 Shared reference to conversion rate metric value (#1034) c34d7a5cbb estimate transaction fee (#1015) 93404b18bb change alert period from 2m to 10m for Westend -> Millau (GRANDPA or public node itself is lagging sometimes) (#1032) git-subtree-dir: bridges git-subtree-split: 407bf44a8a5f4e60aceef2dc755cd9ff09929ac3
-
