- Jan 12, 2023
-
-
Marcin S. authored
* pre-checking: Reject failed PVFs * paras: immediately reject any PVF that cannot reach a supermajority * Make the `quorum` reject condition a bit more clear semantically * Add comment * Update implementer's guide * Update a link Not related to the rest of the PR, but I randomly noticed and fixed this. * Update runtime/parachains/src/paras/tests.rs Co-authored-by:
s0me0ne-unkn0wn <[email protected]> * Remove unneeded loop * Log PVF retries using `info!` * Change retry logs to `warn!` and add preparation failure log * Log PVF execution failure * Clarify why we reject failed PVFs * Fix PVF reject runtime benchmarks Co-authored-by:
s0me0ne-unkn0wn <[email protected]>
-
- Jan 11, 2023
-
-
Marcin S. authored
* pvf: Add checks for result sender when retrying preparation in tests * pvf: Fix missing execution request when retrying preparation * Update comment
-
s0me0ne-unkn0wn authored
* Re-export current primitives in crate root * Add missing exports * restart CI
-
- Jan 10, 2023
-
-
Marcin S. authored
* Minor fixes * Fix compile errors
-
eskimor authored
* First iteration of message sender. * dyn Fn variant (no cloning) * Full implementation + Clone, without allocs on `Send` * Further clarifications/cleanup. * MessageSender -> NestingSender * Doc update/clarification. * dispute-coordinator: Send disputes on startup. + Some fixes, cleanup. * Fix whitespace. * Dispute distribution fixes, cleanup. * Cargo.lock * Fix spaces. * More format fixes. What is cargo fmt doing actually? * More fmt fixes. * Fix nesting sender. * Fixes. * Whitespace * Enable logging. * Guide update. * Fmt fixes, typos. * Remove unused function. * Simplifications, doc fixes. * Update roadmap/implementers-guide/src/node/disputes/dispute-coordinator.md Co-authored-by:
Marcin S. <[email protected]> * Fmt + doc example fix. Co-authored-by:
eskimor <[email protected]> Co-authored-by:
Marcin S. <[email protected]>
-
Marcin S. authored
* Replace async-std with tokio in PVF subsystem * Rework workers to use `select!` instead of a mutex The improvement in code readability is more important than the thread overhead. * Remove unnecessary `fuse` * Add explanation for `expect()` * Update node/core/pvf/src/worker_common.rs Co-authored-by:
Bastian Köcher <[email protected]> * Update node/core/pvf/src/worker_common.rs Co-authored-by:
Bastian Köcher <[email protected]> * Address some review comments * Shutdown tokio runtime * Run cargo fmt * Add a small note about retries * Fix up merge * Rework `cpu_time_monitor_loop` to return when other thread finishes * Add error string to PrepareError::IoErr variant * Log when artifacts fail to prepare * Fix `cpu_time_monitor_loop`; fix test * Fix text * Fix a couple of potential minor data races. First data race was due to logging in the CPU monitor thread even if the job (other thread) finished. It can technically finish before or after the log. Maybe best would be to move this log to the `select!`s, where we are guaranteed to have chosen the timed-out branch, although there would be a bit of duplication. Also, it was possible for this thread to complete before we executed `finished_tx.send` in the other thread, which would trigger an error as the receiver has already been dropped. And right now, such a spurious error from `send` would be returned even if the job otherwise succeeded. * Update Cargo.lock Co-authored-by:
Bastian Köcher <[email protected]>
-
- Jan 09, 2023
-
-
Tsvetomir Dimitrov authored
https://github.com/paritytech/polkadot/pull/6494 updates disputes participation priority on Active Leaves update. This operation might trigger participation in some cases and as a result some of the message ordering is not as nice as it used to be. As a side effect of this `resume_dispute_without_local_statement` was failing occasionally. The solution is not to expect that `BlockNumber`, `CandidateEvents`, `FetchOnChainVotes` and `Ancestors` messages are executed after `FinalizedBlockNumber` and in any specific order. This should be okay as the code is in helper function and doesn't affect the actual test behaviour. Fixes https://github.com/paritytech/polkadot/issues/6514
-
- Jan 06, 2023
-
-
Marcin S. authored
See https://github.com/paritytech/polkadot/issues/6015. (Doesn't close yet, there might be more we can do.)
-
- Jan 05, 2023
-
-
ordian authored
* av-store: write meta for unknown finalized blocks * fix test
-
- Jan 04, 2023
-
-
Marcin S. authored
-
Bradley Olson authored
* Fixing filter to keep all backing votes * Comment and implementers guide edit * Formatting * Using fallthrough * Fmt
-
- Dec 30, 2022
-
-
Bradley Olson authored
* Passed candidate events from scraper to participation * First draft PR 5875 * Added support for timestamp in changes * Some necessary refactoring * Removed SessionIndex from unconfirmed_disputes key * Removed duplicate logic in import statements * Replaced queue_participation call with re-prio * Simplifying refactor. Backed were already handled * Removed unneeded spam slots logic * Implementers guide edits * Undid the spam slots refactor * Added comments and implementers guide edit * Added test for participation upon backing * Round of fixes + ran fmt * Round of changes + fmt * Error handling draft * Changed errors to bubble up from reprioritization * Starting to construct new test * Clarifying participation function rename * Reprio test draft * Very rough bump to priority queue test draft * Improving logging * Most concise reproduction of error on third import * Add `handle_approval_vote_request` * Removing reprioritization on included event test * Removing unneeded test config * cargo fmt * Test works * Fixing final nits * Tweaks to test Tsveto figured out Co-authored-by:
eskimor <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]>
-
- Dec 26, 2022
-
-
Bradley Olson authored
-
- Dec 21, 2022
-
-
Michal Kucharczyk authored
* BlockId removal: refactor: HeaderBackend::status It changes the arguments of `HeaderBackend::status` method from: `BlockId<Block>` to: `Block::Hash` This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) * Update node/core/chain-api/src/tests.rs Co-authored-by:
Adrian Catangiu <[email protected]> * unused import removed * update lockfile for {"substrate"} Co-authored-by:
Adrian Catangiu <[email protected]> Co-authored-by: parity-processbot <>
-
- Dec 20, 2022
-
-
Marcin S. authored
* PVF preparation: do not conflate errors + Adds some more granularity to the prepare errors. + Better distinguish whether errors occur on the host side or the worker. + Do not kill the worker if the error happened on the host side. + Do not retry preparation if the error was `Panic`. + Removes unnecessary indirection with `Selected` type. * Add missing docs, resolve TODOs * Address review comments and remove TODOs * Fix error in CI * Undo unnecessary change * Update couple of comments * Don't return error for stream shutdown * Update node/core/pvf/src/worker_common.rs
-
Michal Kucharczyk authored
* BlockId removal: refactor: HeaderBackend::header It changes the arguments of: - `HeaderBackend::header`, - `Client::header` methods from: `BlockId<Block>` to: `Block::Hash` This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292) * missed fixes * BlockId removal: refactor: HeaderBackend::expect_header It changes the arguments of `HeaderBackend::expect_header` method from: `BlockId<Block>` to: `Block::Hash` * update lockfile for {"substrate"} * misspell fixed Co-authored-by: parity-processbot <>
-
- Dec 15, 2022
-
-
eskimor authored
* Add trace log * More tracing. * Fix redundant rate limit. * Add trace log. * Improve logging. Co-authored-by:
eskimor <[email protected]>
-
- Dec 07, 2022
-
-
Bradley Olson authored
* Refactoring to condense disputes tests * Removing unhelpful comment lines * Addressing Tsveto's suggestions * Fixing formatting nit
-
eskimor authored
* Guide updates * Consider more dead forks. * Ensure backing votes don't get overridden. * Fix spelling. * Fix comments. * Update node/primitives/src/lib.rs Co-authored-by:
Tsvetomir Dimitrov <[email protected]> Co-authored-by:
eskimor <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]>
-
- Dec 06, 2022
-
-
Marcin S. authored
* Let the PVF host kill the worker on timeout * Fix comment * Fix inaccurate comments; add missing return statement * Fix a comment * Fix comment
-
Marcin S. authored
* Begin removing `parity-util-mem`; remove `collect_memory_stats` * Update some dependencies that were using `parity-util-mem` * Remove `trie-memory-tracker` feature * Update Cargo.lock * Update `kvdb-shared-tests` * Add back jemalloc * Add missing license header * update lockfile for {"substrate"} Co-authored-by: parity-processbot <> Co-authored-by:
Andronik <[email protected]>
-
- Dec 05, 2022
-
-
Squirrel authored
* rust 1.64 enables workspace properties * add edition, repository and authors. * of course, update the version in one place. Co-authored-by:
Andronik <[email protected]>
-
- Dec 02, 2022
-
-
alexgparity authored
* Store values needed to create inherent data when needed instead of creating them early on * Point deps to substrate branch * Arc the client * Cargo update * Fix main cargo files * Undo cargo file changes * Add overseer dep to inherents * Update deps * Simplify code * Update benchmark * Update node/client/src/benchmarking.rs Co-authored-by:
Bastian Köcher <[email protected]> * Update node/core/parachains-inherent/src/lib.rs Co-authored-by:
Bastian Köcher <[email protected]> * Update node/core/parachains-inherent/src/lib.rs Co-authored-by:
Bastian Köcher <[email protected]> * Revert "Update node/core/parachains-inherent/src/lib.rs" This reverts commit 8b9555dc2451acfabab173d259e00da2728b7aa2. * Revert "Update node/core/parachains-inherent/src/lib.rs" This reverts commit 816c92d0e001e71f677d0acbcf22bdc3f511bc56. * cargo update -p sp-io * fmt Co-authored-by:
Bastian Köcher <[email protected]>
-
- Nov 30, 2022
-
-
Marcin S. authored
* Put in skeleton logic for CPU-time-preparation Still needed: - Flesh out logic - Refactor some spots - Tests * Continue filling in logic for prepare worker CPU time changes * Fix compiler errors * Update lenience factor * Fix some clippy lints for PVF module * Fix compilation errors * Address some review comments * Add logging * Add another log * Address some review comments; change Mutex to AtomicBool * Refactor handling response bytes * Add CPU clock timeout logic for execute jobs * Properly handle AtomicBool flag * Use `Ordering::Relaxed` * Refactor thread coordination logic * Fix bug * Add some timing information to execute tests * Add section about the mitigation to the IG * minor: Change more `Ordering`s to `Relaxed` * candidate-validation: Fix build errors
-
alexgparity authored
* Add clippy config and remove .cargo from gitignore * first fixes * Clippyfied * Add clippy CI job * comment out rusty-cachier * minor * fix ci * remove DAG from check-dependent-project * add DAG to clippy Co-authored-by:
alvicsam <[email protected]>
-
Mara Broda authored
* westend: update transaction version * polkadot: update transaction version * kusama: update transaction version * Bump spec_version to 9330 * bump versions to 0.9.33
-
- Nov 29, 2022
-
-
Marcin S. authored
-
- Nov 28, 2022
-
-
Andrei Sandu authored
* RollingSession: add fn contains Signed-off-by:
Andrei Sandu <[email protected]> * handle_import_statements fix ancient dispute check Signed-off-by:
Andrei Sandu <[email protected]> * work with earliest session instead of latest Signed-off-by:
Andrei Sandu <[email protected]> * update comment Signed-off-by:
Andrei Sandu <[email protected]> Signed-off-by:
Andrei Sandu <[email protected]>
-
Marcin S. authored
-
Mattia L.V. Bradascio authored
* Add buckets below 5ms * Add more specific histogram buckets * Add more specific buckets * cargo fmt
-
- Nov 23, 2022
-
-
Marcin S. authored
* Add PVF module documentation TODO (once the PRs land): - [ ] Document executor parametrization. - [ ] Document CPU time measurement of timeouts. * Update node/core/pvf/src/lib.rs Co-authored-by:
Andrei Sandu <[email protected]> * Clarify meaning of PVF acronym * Move PVF doc to implementer's guide * Clean up implementer's guide a bit * Add page for PVF types * pvf: Better separation between crate docs and implementer's guide * ci: Add "prevalidating" to the dictionary * ig: Remove types/chain.md The types contained therein did not exist and the file was not referenced anywhere. Co-authored-by:
Andrei Sandu <[email protected]>
-
- Nov 17, 2022
-
-
Tsvetomir Dimitrov authored
* Change best effort queue behaviour in `dispute-coordinator` Use the same type of queue (`BTreeMap<CandidateComparator, ParticipationRequest>`) for best effort and priority in `dispute-coordinator`. Rework `CandidateComparator` to handle unavailable parent block numbers. Best effort queue will order disputes the same way as priority does - by parent's block height. Disputes on candidates for which the parent's block number can't be obtained will be treated with the lowest priority. * Fix tests: Handle `ChainApiMessage::BlockNumber` in `handle_sync_queries` * Some tests are deadlocking on sending messages via overseer so change `SingleItemSink`to `mpsc::Sender` with a buffer of 1 * Fix a race in test after adding a buffered queue for overseer messages * Fix the rest of the tests * Guide update - best-effort queue * Guide update: clarification about spam votes * Fix tests in `availability-distribution` * Update comments * Add `make_buffered_subsystem_context` in `subsystem-test-helpers` * Code review feedback * Code review feedback * Code review feedback * Don't add best effort candidate if it is already in priority queue * Remove an old comment * Fix insert in best_effort
-
- Nov 16, 2022
-
-
Tsvetomir Dimitrov authored
* Fix typos * Filter unconfirmed disputes in provisioner - random_selection * Rework dispute coordinator to return `DisputeStatus` with `ActiveDisputes` message. * Rework the random_selection implementation of `select_disptues` in `provisioner` to return only confirmed disputes. * Filter unconfirmed disputes in provisioner - prioritized_selection * Add test for unconfirmed disputes handling * Fix `dispute-distribution` tests
-
- Nov 15, 2022
-
-
Marcin S. authored
Was seeing these warnings when running `cargo check --all`: ``` warning: for loop over an `Option`. This is more readably written as an `if let` statement --> node/core/approval-voting/src/lib.rs:1147:21 | 1147 | for activated in update.activated { | ^^^^^^^^^^^^^^^^ | = note: `#[warn(for_loops_over_fallibles)]` on by default help: to check pattern in a loop use `while let` | 1147 | while let Some(activated) = update.activated { | ~~~~~~~~~~~~~~~ ~~~ help: consider using `if let` to clear intent | 1147 | if let Some(activated) = update.activated { | ~~~~~~~~~~~~ ~~~ ``` My guess is that `activated` used to be a SmallVec or similar, as is `deactivated`. It was changed to an `Option`, the `for` still compiled (it's technically correct, just weird), and the compiler didn't catch it until now.
-
- Nov 14, 2022
-
-
eskimor authored
* Only report concluded if there is an actual dispute. Hence no "non"-disputes will be added to disputes anymore. * Fix redundant check. * Test for no onesided disputes. Co-authored-by:
eskimor <[email protected]>
-
- Nov 12, 2022
-
-
ordian authored
* approval-voting: remove a redundant check * candidate-validation: remove unreachable check
-
- Nov 11, 2022
-
-
Tsvetomir Dimitrov authored
* Scraper processes CandidateBacked events * Change definition of best-effort * Fix `dispute-coordinator` tests * Unit test for dispute filtering * Clarification comment * Add tests * Fix logic If a dispute is not backed, not included and not confirmed we don't participate but we do import votes. * Add metrics for refrained participations * Revert "Add tests" This reverts commit 7b8391a087922ced942cde9cd2b50ff3f633efc0. * Revert "Unit test for dispute filtering" This reverts commit 92ba5fe678214ab360306313a33c781338e600a0. * fix dispute-coordinator tests * Fix scraping * new tests * Small fixes in guide * Apply suggestions from code review Co-authored-by:
Andrei Sandu <[email protected]> * Fix some comments and remove a pointless test * Code review feedback * Clarification comment in tests * Some tests * Reference counted `CandidateHash` in scraper * Proper handling for Backed and Included candidates in scraper Backed candidates which are not included should be kept for a predetermined window of finalized blocks. E.g. if a candidate is backed but not included in block 2, and the window size is 2, the same candidate should be cleaned after block 4 is finalized. Add reference counting for candidates in scraper. A candidate can be added on multiple block heights so we have to make sure we don't clean it prematurely from the scraper. Add tests. * Update comments in tests * Guide update * Fix cleanup logic for `backed_candidates_by_block_number` * Simplify cleanup * Make spellcheck happy * Update tests * Extract candidate backing logic in separate struct * Code review feedback * Treat backed and included candidates in the same fashion * Update some comments * Small improvements in test * spell check * Fix some more comments * clean -> prune * Code review feedback * Reword comment * spelling Co-authored-by:
Andrei Sandu <[email protected]>
-
- Nov 08, 2022
-
-
Marcin S. authored
* Fix a couple of typos * Retry failed PVF execution PVF execution that fails due to AmbiguousWorkerDeath should be retried once. This should reduce the occurrence of failures due to transient conditions. Closes #6195 * Address a couple of nits * Write tests; refactor (add `validate_candidate_with_retry`) * Update node/core/candidate-validation/src/lib.rs Co-authored-by:
Andronik <[email protected]> Co-authored-by:
eskimor <[email protected]> Co-authored-by:
Andronik <[email protected]>
-
Andrei Sandu authored
* Impl dynamic window size. Keep sessions for unfinalized chain Signed-off-by:
Andrei Sandu <[email protected]> * feedback Signed-off-by:
Andrei Sandu <[email protected]> * Stretch also in contructor plus tests Signed-off-by:
Andrei Sandu <[email protected]> * review feedback Signed-off-by:
Andrei Sandu <[email protected]> * fix approval-voting tests Signed-off-by:
Andrei Sandu <[email protected]> * grunting: dispute coordinator tests Signed-off-by:
Andrei Sandu <[email protected]> * add session window column Signed-off-by:
Andrei Sandu <[email protected]> * integrate approval vote and fix tests Signed-off-by:
Andrei Sandu <[email protected]> * fix rolling session tests Signed-off-by:
Andrei Sandu <[email protected]> * Small refactor Signed-off-by:
Andrei Sandu <[email protected]> * WIP, tests failing Signed-off-by:
Andrei Sandu <[email protected]> * Fix approval voting tests Signed-off-by:
Andrei Sandu <[email protected]> * fix dispute-coordinator tests Signed-off-by:
Andrei Sandu <[email protected]> * remove uneeded param Signed-off-by:
Andrei Sandu <[email protected]> * fmt Signed-off-by:
Andrei Sandu <[email protected]> * fix loose ends Signed-off-by:
Andrei Sandu <[email protected]> * allow failure and tests for it Signed-off-by:
Andrei Sandu <[email protected]> * fix comment Signed-off-by:
Andrei Sandu <[email protected]> * comment fix Signed-off-by:
Andrei Sandu <[email protected]> * style fix Signed-off-by:
Andrei Sandu <[email protected]> * new col doesn't need to be ordered Signed-off-by:
Andrei Sandu <[email protected]> * fmt and spellcheck Signed-off-by:
Andrei Sandu <[email protected]> * db persist tests Signed-off-by:
Andrei Sandu <[email protected]> * Add v2 config and cols Signed-off-by:
Andrei Sandu <[email protected]> * DB upgrade WIP Signed-off-by:
Andrei Sandu <[email protected]> * Fix comments Signed-off-by:
Andrei Sandu <[email protected]> * add todo Signed-off-by:
Andrei Sandu <[email protected]> * update to parity-db to "0.4.2" Signed-off-by:
Andrei Sandu <[email protected]> * migration complete Signed-off-by:
Andrei Sandu <[email protected]> * One session window size Signed-off-by:
Andrei Sandu <[email protected]> * fix merge damage Signed-off-by:
Andrei Sandu <[email protected]> * fix build errors Signed-off-by:
Andrei Sandu <[email protected]> * fmt Signed-off-by:
Andrei Sandu <[email protected]> * comment fix Signed-off-by:
Andrei Sandu <[email protected]> * fix build Signed-off-by:
Andrei Sandu <[email protected]> * make error more explicit Signed-off-by:
Andrei Sandu <[email protected]> * add comment Signed-off-by:
Andrei Sandu <[email protected]> * refactor conflict merge Signed-off-by:
Andrei Sandu <[email protected]> * rename col_data Signed-off-by:
Andrei Sandu <[email protected]> * add doc comment Signed-off-by:
Andrei Sandu <[email protected]> * fix build Signed-off-by:
Andrei Sandu <[email protected]> * migration: move all cols to v2 Signed-off-by:
Andrei Sandu <[email protected]> Signed-off-by:
Andrei Sandu <[email protected]>
-
Marcin S. authored
-