- Jan 12, 2023
-
-
Marcin S. authored
* pre-checking: Reject failed PVFs * paras: immediately reject any PVF that cannot reach a supermajority * Make the `quorum` reject condition a bit more clear semantically * Add comment * Update implementer's guide * Update a link Not related to the rest of the PR, but I randomly noticed and fixed this. * Update runtime/parachains/src/paras/tests.rs Co-authored-by:
s0me0ne-unkn0wn <[email protected]> * Remove unneeded loop * Log PVF retries using `info!` * Change retry logs to `warn!` and add preparation failure log * Log PVF execution failure * Clarify why we reject failed PVFs * Fix PVF reject runtime benchmarks Co-authored-by:
s0me0ne-unkn0wn <[email protected]>
-
- Jan 11, 2023
-
-
Marcin S. authored
* pvf: Add checks for result sender when retrying preparation in tests * pvf: Fix missing execution request when retrying preparation * Update comment
-
- Jan 10, 2023
-
-
Marcin S. authored
* Minor fixes * Fix compile errors
-
Marcin S. authored
* Replace async-std with tokio in PVF subsystem * Rework workers to use `select!` instead of a mutex The improvement in code readability is more important than the thread overhead. * Remove unnecessary `fuse` * Add explanation for `expect()` * Update node/core/pvf/src/worker_common.rs Co-authored-by:
Bastian Köcher <[email protected]> * Update node/core/pvf/src/worker_common.rs Co-authored-by:
Bastian Köcher <[email protected]> * Address some review comments * Shutdown tokio runtime * Run cargo fmt * Add a small note about retries * Fix up merge * Rework `cpu_time_monitor_loop` to return when other thread finishes * Add error string to PrepareError::IoErr variant * Log when artifacts fail to prepare * Fix `cpu_time_monitor_loop`; fix test * Fix text * Fix a couple of potential minor data races. First data race was due to logging in the CPU monitor thread even if the job (other thread) finished. It can technically finish before or after the log. Maybe best would be to move this log to the `select!`s, where we are guaranteed to have chosen the timed-out branch, although there would be a bit of duplication. Also, it was possible for this thread to complete before we executed `finished_tx.send` in the other thread, which would trigger an error as the receiver has already been dropped. And right now, such a spurious error from `send` would be returned even if the job otherwise succeeded. * Update Cargo.lock Co-authored-by:
Bastian Köcher <[email protected]>
-
- Jan 04, 2023
-
-
Marcin S. authored
-
- Dec 20, 2022
-
-
Marcin S. authored
* PVF preparation: do not conflate errors + Adds some more granularity to the prepare errors. + Better distinguish whether errors occur on the host side or the worker. + Do not kill the worker if the error happened on the host side. + Do not retry preparation if the error was `Panic`. + Removes unnecessary indirection with `Selected` type. * Add missing docs, resolve TODOs * Address review comments and remove TODOs * Fix error in CI * Undo unnecessary change * Update couple of comments * Don't return error for stream shutdown * Update node/core/pvf/src/worker_common.rs
-
- Dec 06, 2022
-
-
Marcin S. authored
* Let the PVF host kill the worker on timeout * Fix comment * Fix inaccurate comments; add missing return statement * Fix a comment * Fix comment
-
- Dec 05, 2022
-
-
Squirrel authored
* rust 1.64 enables workspace properties * add edition, repository and authors. * of course, update the version in one place. Co-authored-by:
Andronik <[email protected]>
-
- Nov 30, 2022
-
-
Marcin S. authored
* Put in skeleton logic for CPU-time-preparation Still needed: - Flesh out logic - Refactor some spots - Tests * Continue filling in logic for prepare worker CPU time changes * Fix compiler errors * Update lenience factor * Fix some clippy lints for PVF module * Fix compilation errors * Address some review comments * Add logging * Add another log * Address some review comments; change Mutex to AtomicBool * Refactor handling response bytes * Add CPU clock timeout logic for execute jobs * Properly handle AtomicBool flag * Use `Ordering::Relaxed` * Refactor thread coordination logic * Fix bug * Add some timing information to execute tests * Add section about the mitigation to the IG * minor: Change more `Ordering`s to `Relaxed` * candidate-validation: Fix build errors
-
alexgparity authored
* Add clippy config and remove .cargo from gitignore * first fixes * Clippyfied * Add clippy CI job * comment out rusty-cachier * minor * fix ci * remove DAG from check-dependent-project * add DAG to clippy Co-authored-by:
alvicsam <[email protected]>
-
Mara Broda authored
* westend: update transaction version * polkadot: update transaction version * kusama: update transaction version * Bump spec_version to 9330 * bump versions to 0.9.33
-
- Nov 28, 2022
-
-
Marcin S. authored
-
- Nov 23, 2022
-
-
Marcin S. authored
* Add PVF module documentation TODO (once the PRs land): - [ ] Document executor parametrization. - [ ] Document CPU time measurement of timeouts. * Update node/core/pvf/src/lib.rs Co-authored-by:
Andrei Sandu <[email protected]> * Clarify meaning of PVF acronym * Move PVF doc to implementer's guide * Clean up implementer's guide a bit * Add page for PVF types * pvf: Better separation between crate docs and implementer's guide * ci: Add "prevalidating" to the dictionary * ig: Remove types/chain.md The types contained therein did not exist and the file was not referenced anywhere. Co-authored-by:
Andrei Sandu <[email protected]>
-
- Nov 08, 2022
-
-
Marcin S. authored
* Fix a couple of typos * Retry failed PVF execution PVF execution that fails due to AmbiguousWorkerDeath should be retried once. This should reduce the occurrence of failures due to transient conditions. Closes #6195 * Address a couple of nits * Write tests; refactor (add `validate_candidate_with_retry`) * Update node/core/candidate-validation/src/lib.rs Co-authored-by:
Andronik <[email protected]> Co-authored-by:
eskimor <[email protected]> Co-authored-by:
Andronik <[email protected]>
-
Marcin S. authored
-
- Nov 01, 2022
-
-
Marcin S. authored
* Rename timeout consts and timeout parameter; bump leniency * Update implementor's guide with info about PVFs * Make glossary a bit easier to read * Add a note to LENIENT_PREPARATION_TIMEOUT * Remove PVF-specific section from glossary * Fix some typos
-
- Oct 26, 2022
-
-
Marcin S. authored
* Log exit status code for workers * Make log for execute job conclusion match prepare job conclusion Trace log for conclusion of prepare job: ```rs gum::debug!( target: LOG_TARGET, validation_code_hash = ?artifact_id.code_hash, ?worker, ?rip, "prepare worker concluded", ); ``` Co-authored-by: parity-processbot <>
-
- Oct 22, 2022
-
-
Mara Broda authored
* Bump spec_version to 9310 * bump transaction_version (0.9.31) (#6171) * Bump transaction_version for polkadot * Bump transaction_version for kusama * Bump transaction_version for rococo * Bump transaction_version for westend * Bump transaction_version for polkadot * Bump transaction_version for kusama * Bump transaction_version for rococo * Bump transaction_version for westend * Bump crate versions (0.9.31)
-
- Oct 13, 2022
-
-
Marcin S. authored
* Add some documentation * Add `compilation_timeout` parameter for PVF preparation job * Update buckets in prometheus metrics * Update prepare/queue tests * Update pvf-prechecking overview in implementer docs * Fix some CI checks
-
- Sep 15, 2022
-
-
ordian authored
-
- Sep 09, 2022
-
-
Mara Broda authored
* Bump transaction_version for westend * Bump transaction_version for rococo * Bump transaction_version for kusama * Bump transaction_version for polkadot * Bump spec_version to 9290 * Bump crate versions
-
- Aug 29, 2022
-
-
Mara Broda authored
* Bump crate versions * Bump spec_version to 9280 for kusama * Bump spec_version to 9280 for polkadot * Bump spec_version to 9280 for rococo * Bump spec_version to 9280 for westend * update Cargo.lock Co-authored-by: parity-processbot <>
-
- Jul 28, 2022
-
-
Chevdor authored
* Bump crate versions * Update cargo.lock
-
- Jul 06, 2022
-
-
Chevdor authored
* Bump spec_version to 9260 * Version bump to v0.9.26
-
- Jun 27, 2022
-
-
dependabot[bot] authored
Bumps [parity-scale-codec](https://github.com/paritytech/parity-scale-codec) from 3.1.2 to 3.1.5. - [Release notes](https://github.com/paritytech/parity-scale-codec/releases) - [Changelog](https://github.com/paritytech/parity-scale-codec/blob/master/CHANGELOG.md) - [Commits](https://github.com/paritytech/parity-scale-codec/compare/parity-scale-codec-3.1.2...parity-scale-codec-3.1.5 ) --- updated-dependencies: - dependency-name: parity-scale-codec dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by:
dependabot[bot] <[email protected]> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
- Jun 24, 2022
-
-
Sergei Shulepov authored
* pvf: ensure enough stack space * fix typos Co-authored-by:
Andronik <[email protected]> * Use rayon to cache the thread Co-authored-by:
Andronik <[email protected]>
-
- Jun 23, 2022
-
-
Sergei Shulepov authored
-
- Jun 21, 2022
-
-
Mara Broda authored
* kusama: bump spec_version to 9250 * polkadot: bump spec_version to 9250 * rococo: bump spec_version to 9250 * westend: bump spec_version to 9250 * bump version to 0.9.25 * bump transaction version (polkadot & kusama) (#5690) * kusama: bump transaction_version to 12 * polkadot: bump transaction_version to 13
-
- May 30, 2022
-
-
Gavin Wood authored
* Fix warnings * Bump
-
- May 27, 2022
-
-
Mara Broda authored
* bump version to 0.9.23 * bump spec_version to 9230
-
- May 19, 2022
-
-
Koute authored
Switch to pooling copy-on-write instantiation strategy for WASM (companion for Substrate#11232) (#5337) * Switch to pooling copy-on-write instantiation strategy for WASM * Fix compilation of `polkadot-test-service` * Update comments * Move `max_memory_size` to `Semantics` * Rename `WasmInstantiationStrategy` to `WasmtimeInstantiationStrategy` * Update a safety comment * update lockfile for {"substrate"} Co-authored-by: parity-processbot <>
-
- May 16, 2022
-
-
Mara Broda authored
* bump versions to 0.9.22 * bump spec_version to 9220
-
- Apr 09, 2022
-
-
Sergei Shulepov authored
The PVF host is designed to avoid spawning tasks to minimize knowledge of outer code. Using `async_std::task::spawn` (or Tokio's counterpart) deemed unacceptable, `SpawnNamed` undesirable. Instead there is only one task returned that is spawned by the candidate-validation subsystem. The tasks from the sub-components are polled by that root task. However, the way the tasks are bundled was incorrect. There was a giant select that was polling those tasks. Particularly, that implies that as soon as one of the arms of that select goes into await those sub-tasks stop getting polled. This is a recipe for a deadlock which indeed happened here. Specifically, the deadlock happened during sending messages to the execute queue by calling [`send_execute`](https://github.com/paritytech/polkadot/blob/a68d9be35656dcd96e378fd9dd3d613af754d48a/node/core/pvf/src/host.rs#L601). When the channel to the queue reaches the capacity, the control flow is suspended until the queue handles those messages. Since this code is essentially reached from [one of the select arms](https://github.com/paritytech/polkadot/blob/a68d9be35656dcd96e378fd9dd3d613af754d48a/node/core/pvf/src/host.rs#L371), the queue won't be given the control and thus no further progress can be made. This problem is solved by bundling the tasks one level higher instead, by `selecting` over those long-running tasks. We also stop treating returning from those long-running tasks as error conditions, since that can happen during legit shutdown.
-
- Apr 08, 2022
-
-
Mara Broda authored
-
- Mar 24, 2022
-
-
Koute authored
* Rename to BagError * Additional parameter for 'revert' command * Set aux revert param to None * Align to changes in how the WASM executor is configured in `substrate` * update lockfile for {"substrate"} * update lockfile for {"substrate"} * Update substrate * Update substrate Co-authored-by:
Keith Yeung <[email protected]> Co-authored-by:
Davide Galassi <[email protected]> Co-authored-by:
Shawn Tabrizi <[email protected]> Co-authored-by: parity-processbot <>
-
- Mar 23, 2022
-
-
Georges authored
* Fixing `generate_solution_type` This is needed after changes to substrate * `VoterSnapshotPerBlock` -> `MaxElectingVoters` * rename `SizeBound` to `MaxVoters` * cargo update -p sp-io * ignore failing test * spell-check * fix pub OffchainRepeat Co-authored-by:
kianenigma <[email protected]> Co-authored-by:
Kian Paimani <[email protected]>
-
dependabot[bot] authored
Bumps [async-std](https://github.com/async-rs/async-std) from 1.10.0 to 1.11.0. - [Release notes](https://github.com/async-rs/async-std/releases) - [Changelog](https://github.com/async-rs/async-std/blob/master/CHANGELOG.md) - [Commits](https://github.com/async-rs/async-std/compare/v1.10.0...v1.11.0 ) --- updated-dependencies: - dependency-name: async-std dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by:
dependabot[bot] <[email protected]> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
- Mar 18, 2022
-
-
dependabot[bot] authored
Bumps [parity-scale-codec](https://github.com/paritytech/parity-scale-codec) from 3.1.0 to 3.1.2. - [Release notes](https://github.com/paritytech/parity-scale-codec/releases) - [Changelog](https://github.com/paritytech/parity-scale-codec/blob/master/CHANGELOG.md) - [Commits](https://github.com/paritytech/parity-scale-codec/commits ) --- updated-dependencies: - dependency-name: parity-scale-codec dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by:
dependabot[bot] <[email protected]> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
- Mar 15, 2022
-
-
Bernhard Schuster authored
* add some gum * bump expander * gum * fix all remaining issues * last fixup * Update node/gum/proc-macro/src/lib.rs Co-authored-by:
Bastian Köcher <[email protected]> * change * netowrk * fixins * chore * allow optional fmt str + args, prep for expr as kv field * tracing -> gum rename fallout * restrict further * allow multiple levels of field accesses * another round of docs and a slip of the pen * update ADR * fixup lock fiel * use target: instead of target= * minors * fix * chore * Update node/gum/README.md Co-authored-by:
Andrei Sandu <[email protected]> Co-authored-by:
Bastian Köcher <[email protected]> Co-authored-by:
Andrei Sandu <[email protected]>
-
- Mar 14, 2022
-
-
dependabot[bot] authored
Bumps [parity-scale-codec](https://github.com/paritytech/parity-scale-codec) from 3.0.0 to 3.1.0. - [Release notes](https://github.com/paritytech/parity-scale-codec/releases) - [Changelog](https://github.com/paritytech/parity-scale-codec/blob/master/CHANGELOG.md) - [Commits](https://github.com/paritytech/parity-scale-codec/compare/v3.0...parity-scale-codec-v3.1.0 ) --- updated-dependencies: - dependency-name: parity-scale-codec dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by:
dependabot[bot] <[email protected]> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-