Skip to content
  1. Dec 05, 2022
  3. Dec 02, 2022
    • alexgparity's avatar
      Reduce provisioner work (#6328) · a7eee7dd
      alexgparity authored 
      

* Store values needed to create inherent data when needed instead of creating them early on

* Point deps to substrate branch

* Arc the client

* Cargo update

* Fix main cargo files

* Undo cargo file changes

* Add overseer dep to inherents

* Update deps

* Simplify code

* Update benchmark

* Update node/client/src/benchmarking.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: default avatarBastian Köcher <[email protected]>

* Revert "Update node/core/parachains-inherent/src/lib.rs"

This reverts commit 8b9555dc2451acfabab173d259e00da2728b7aa2.

* Revert "Update node/core/parachains-inherent/src/lib.rs"

This reverts commit 816c92d0e001e71f677d0acbcf22bdc3f511bc56.

* cargo update -p sp-io

* fmt

Co-authored-by: default avatarBastian Köcher <[email protected]>
      a7eee7dd
  5. Nov 30, 2022
    • Marcin S.'s avatar
      Use CPU clock timeout for PVF jobs (#6282) · 28a4e909
      Marcin S. authored 
      * Put in skeleton logic for CPU-time-preparation

Still needed:
- Flesh out logic
- Refactor some spots
- Tests

* Continue filling in logic for prepare worker CPU time changes

* Fix compiler errors

* Update lenience factor

* Fix some clippy lints for PVF module

* Fix compilation errors

* Address some review comments

* Add logging

* Add another log

* Address some review comments; change Mutex to AtomicBool

* Refactor handling response bytes

* Add CPU clock timeout logic for execute jobs

* Properly handle AtomicBool flag

* Use `Ordering::Relaxed`

* Refactor thread coordination logic

* Fix bug

* Add some timing information to execute tests

* Add section about the mitigation to the IG

* minor: Change more `Ordering`s to `Relaxed`

* candidate-validation: Fix build errors
      28a4e909
    • alexgparity's avatar
      Clippyfy (#6341) · 9ea14e66
      alexgparity authored 
      

* Add clippy config and remove .cargo from gitignore

* first fixes

* Clippyfied

* Add clippy CI job

* comment out rusty-cachier

* minor

* fix ci

* remove DAG from check-dependent-project

* add DAG to clippy

Co-authored-by: default avataralvicsam <[email protected]>
      9ea14e66
    • Mara Broda's avatar
      sync versions with current release (0.9.33) (#6363) · b76086c6
      Mara Broda authored 
      * westend: update transaction version

* polkadot: update transaction version

* kusama: update transaction version

* Bump spec_version to 9330

* bump versions to 0.9.33
      b76086c6
  7. Nov 29, 2022
  9. Nov 28, 2022
  11. Nov 23, 2022
    • Marcin S.'s avatar
      Add PVF module documentation (#6293) · 1dec2433
      Marcin S. authored 
      

* Add PVF module documentation

TODO (once the PRs land):

- [ ] Document executor parametrization.

- [ ] Document CPU time measurement of timeouts.

* Update node/core/pvf/src/lib.rs

Co-authored-by: default avatarAndrei Sandu <[email protected]>

* Clarify meaning of PVF acronym

* Move PVF doc to implementer's guide

* Clean up implementer's guide a bit

* Add page for PVF types

* pvf: Better separation between crate docs and implementer's guide

* ci: Add "prevalidating" to the dictionary

* ig: Remove types/chain.md

The types contained therein did not exist and the file was not referenced
anywhere.

Co-authored-by: default avatarAndrei Sandu <[email protected]>
      1dec2433
  13. Nov 17, 2022
    • Tsvetomir Dimitrov's avatar
      Change best effort queue behaviour in `dispute-coordinator` (#6275) · ccad411e
      Tsvetomir Dimitrov authored 
      * Change best effort queue behaviour in `dispute-coordinator`

Use the same type of queue (`BTreeMap<CandidateComparator,
ParticipationRequest>`) for best effort and priority in
`dispute-coordinator`.

Rework `CandidateComparator` to handle unavailable parent
block numbers.

Best effort queue will order disputes the same way as priority does - by
parent's block height. Disputes on candidates for which the parent's
block number can't be obtained will be treated with the lowest priority.

* Fix tests: Handle `ChainApiMessage::BlockNumber` in `handle_sync_queries`

* Some tests are deadlocking on sending messages via overseer so change `SingleItemSink`to `mpsc::Sender` with a buffer of 1

* Fix a race in test after adding a buffered queue for overseer messages

* Fix the rest of the tests

* Guide update - best-effort queue

* Guide update: clarification about spam votes

* Fix tests in `availability-distribution`

* Update comments

* Add `make_buffered_subsystem_context` in `subsystem-test-helpers`

* Code review feedback

* Code review feedback

* Code review feedback

* Don't add best effort candidate if it is already in priority queue

* Remove an old comment

* Fix insert in best_effort
      ccad411e
  15. Nov 16, 2022
    • Tsvetomir Dimitrov's avatar
      Provisioner should ignore unconfirmed disputes (#6294) · a0f4287d
      Tsvetomir Dimitrov authored 
      * Fix typos

* Filter unconfirmed disputes in provisioner -  random_selection

* Rework dispute coordinator to return `DisputeStatus` with
`ActiveDisputes` message.
* Rework the random_selection implementation of `select_disptues` in
  `provisioner` to return only confirmed disputes.

* Filter unconfirmed disputes in provisioner - prioritized_selection

* Add test for unconfirmed disputes handling

* Fix `dispute-distribution` tests
      a0f4287d
  17. Nov 15, 2022
    • Marcin S.'s avatar
      Fixes "for loop over an `Option`" warnings (#6291) · d53513ff
      Marcin S. authored 
      Was seeing these warnings when running `cargo check --all`:

```
warning: for loop over an `Option`. This is more readably written as an `if let` statement
    --> node/core/approval-voting/src/lib.rs:1147:21
     |
1147 |             for activated in update.activated {
     |                              ^^^^^^^^^^^^^^^^
     |
     = note: `#[warn(for_loops_over_fallibles)]` on by default
help: to check pattern in a loop use `while let`
     |
1147 |             while let Some(activated) = update.activated {
     |             ~~~~~~~~~~~~~~~         ~~~
help: consider using `if let` to clear intent
     |
1147 |             if let Some(activated) = update.activated {
     |             ~~~~~~~~~~~~         ~~~
```

My guess is that `activated` used to be a SmallVec or similar, as is
`deactivated`. It was changed to an `Option`, the `for` still compiled (it's
technically correct, just weird), and the compiler didn't catch it until now.
      d53513ff
  19. Nov 14, 2022
  21. Nov 12, 2022
  23. Nov 11, 2022
    • Tsvetomir Dimitrov's avatar
      Update disputes prioritisation in `dispute-coordinator` (#6130) · ff09b18d
      Tsvetomir Dimitrov authored 
      

* Scraper processes CandidateBacked events

* Change definition of best-effort

* Fix `dispute-coordinator` tests

* Unit test for dispute filtering

* Clarification comment

* Add tests

* Fix logic

If a dispute is not backed, not included and not confirmed we
don't participate but we do import votes.

* Add metrics for refrained participations

* Revert "Add tests"

This reverts commit 7b8391a087922ced942cde9cd2b50ff3f633efc0.

* Revert "Unit test for dispute filtering"

This reverts commit 92ba5fe678214ab360306313a33c781338e600a0.

* fix dispute-coordinator tests

* Fix scraping

* new tests

* Small fixes in guide

* Apply suggestions from code review

Co-authored-by: default avatarAndrei Sandu <[email protected]>

* Fix some comments and remove a pointless test

* Code review feedback

* Clarification comment in tests

* Some tests

* Reference counted `CandidateHash` in scraper

* Proper handling for Backed and Included candidates in scraper

Backed candidates which are not included should be kept for a
predetermined window of finalized blocks. E.g. if a candidate is backed
but not included in block 2, and the window size is 2, the same
candidate should be cleaned after block 4 is finalized.

Add reference counting for candidates in scraper. A candidate can be
added on multiple block heights so we have to make sure we don't clean
it prematurely from the scraper.

Add tests.

* Update comments in tests

* Guide update

* Fix cleanup logic for `backed_candidates_by_block_number`

* Simplify cleanup

* Make spellcheck happy

* Update tests

* Extract candidate backing logic in separate struct

* Code review feedback

* Treat  backed and included candidates in the same fashion

* Update some comments

* Small improvements in test

* spell check

* Fix some more comments

* clean -> prune

* Code review feedback

* Reword comment

* spelling

Co-authored-by: default avatarAndrei Sandu <[email protected]>
      ff09b18d
  25. Nov 08, 2022
  27. Nov 07, 2022
  29. Nov 01, 2022
    • Marcin S.'s avatar
      PVF timeouts follow-up (#6151) · 1f821976
      Marcin S. authored 
      * Rename timeout consts and timeout parameter; bump leniency

* Update implementor's guide with info about PVFs

* Make glossary a bit easier to read

* Add a note to LENIENT_PREPARATION_TIMEOUT

* Remove PVF-specific section from glossary

* Fix some typos
      1f821976
  31. Oct 31, 2022
  33. Oct 26, 2022
    • Marcin S.'s avatar
      Make some fixes to logging in PVF subsystem (#6180) · d4e3501e
      Marcin S. authored 
      * Log exit status code for workers

* Make log for execute job conclusion match prepare job conclusion

Trace log for conclusion of prepare job:

```rs
gum::debug!(
	target: LOG_TARGET,
	validation_code_hash = ?artifact_id.code_hash,
	?worker,
	?rip,
	"prepare worker concluded",
);
```

Co-authored-by: parity-processbot <>
      d4e3501e
  35. Oct 22, 2022
    • Boluwatife Bakre's avatar
      Use a more typesafe approach for managing indexed data (#6150) · 8eb1f461
      Boluwatife Bakre authored 
      

* Fix for issue #2403

* Nightly fmt

* Quick documentation fixes

* Default Implementation

* iter() function integrated

* Implemented iter functionalities

* Fmt

* small change

* updates node-network

* updates in dispute-coordinator

* Updates

* benchmarking fix

* minor fix

* test fixes in runtime api

* Update primitives/src/v2/mod.rs

Co-authored-by: default avatarAndronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: default avatarAndronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: default avatarAndronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: default avatarAndronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: default avatarAndronik <[email protected]>

* Removal of [index], shorting of FromIterator, Renaming of GroupValidators to ValidatorGroups

* Removal of ops import

* documentation fixes for spell check

* implementation of generic type

* Refactoring

* Test and documentation fixes

* minor test fix

* minor test fix

* minor test fix

* Update node/network/statement-distribution/src/lib.rs

Co-authored-by: default avatarAndronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: default avatarAndronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: default avatarAndronik <[email protected]>

* removed IterMut

* Update node/core/dispute-coordinator/src/import.rs

Co-authored-by: default avatarAndronik <[email protected]>

* Update node/core/dispute-coordinator/src/initialized.rs

Co-authored-by: default avatarAndronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: default avatarAndronik <[email protected]>

* fmt

* IterMut

* documentation update

Co-authored-by: default avatarAndronik <[email protected]>

* minor adjustments and new TypeIndex trait

* spelling fix

* TypeIndex fix

Co-authored-by: default avatarAndronik <[email protected]>
      8eb1f461
    • Mara Broda's avatar
      sync versions with current release (0.9.31) (#6176) · f8cc39a7
      Mara Broda authored 
      * Bump spec_version to 9310

* bump transaction_version (0.9.31) (#6171)

* Bump transaction_version for polkadot

* Bump transaction_version for kusama

* Bump transaction_version for rococo

* Bump transaction_version for westend

* Bump transaction_version for polkadot

* Bump transaction_version for kusama

* Bump transaction_version for rococo

* Bump transaction_version for westend

* Bump crate versions (0.9.31)
      f8cc39a7
  37. Oct 13, 2022
  39. Oct 05, 2022
    • ordian's avatar
      update kvdb & co (#6111) · af6a5cd9
      ordian authored 
      

* toml changes

* REVERTME: patch

* adapt parachains db interface

* fix Cargo.toml patch after master rebase

* fix av-store

* fix chain-selection

* fix parachains-db?

* Revert "fix Cargo.toml patch after master rebase"

This reverts commit 3afcbf033c86027b3f2b909d83ec703591bdd287.

* Revert "REVERTME: patch"

This reverts commit 464b717cf4142d3d09c3d77b83700b632d8c5f54.

* Use `Ok` imported from prelude

Co-authored-by: default avatarBastian Köcher <[email protected]>

* update lockfile for {"substrate"}

* Revert "update lockfile for {"substrate"}"

This reverts commit fdc623de226f7645741b86c4b1a7d030fed2172d.

* cargo update -p sp-io

Co-authored-by: default avatarBastian Köcher <[email protected]>
Co-authored-by: parity-processbot <>
      af6a5cd9
  41. Oct 04, 2022
  43. Sep 30, 2022
  45. Sep 29, 2022
    • Robert Klotzner's avatar
      Demote warning (#6080) · 260f70e3
      Robert Klotzner authored 
      This can happen under heavy load - no value in warning here.

Scenario this happens:

- New slot
- We get asked about what fork to build on
- We start building (create inherent gets called)
- We learn about a better fork (but Babe does not care as it already has a fork)
- 6 seconds passed - New slot
- Babe asks about what for to build on - we answer with the best block we learned about 6 seconds ago (slightly after we got asked the last time)
- We start building (on that old block)
- Milliseconds later we learn about a new block (the one from the slot we are actually in now)
- We kick the old leaf -> create inherent fails

Full discussion: https://github.com/paritytech/substrate/issues/12375
      260f70e3
  47. Sep 27, 2022
  49. Sep 19, 2022
    • Tsvetomir Dimitrov's avatar
      Improved dispute votes import in provisioner (#5567) · 6ae9720c
      Tsvetomir Dimitrov authored 
      * Add `DisputeState` to `DisputeCoordinatorMessage::RecentDisputes`

The new signature of the message is:
```
RecentDisputes(oneshot::Sender<Vec<(SessionIndex, CandidateHash, DisputeStatus)>>),
```

As part of the change also add `DispiteStatus` to
`polkadot_node_primitives`.

* Move dummy_signature() in primitives/test-helpers

* Enable staging runtime api on Rococo

* Implementation

* Move disputes to separate module
* Vote prioritisation
* Duplicates handling
* Double vote handling
* Unit tests
* Logs and metrics
* Code review feedback
* Fix ACTIVE/INACTIVE separation and update partition names
* Add `fn dispute_is_inactive` to node primitives and refactor `fn get_active_with_status()` logic
* Keep the 'old' logic if the staging api is not enabled
* Fix some comments in tests
* Add warning message if there are any inactive_unknown_onchain disputes
* Add file headers and remove `use super::*;` usage outside tests
* Adding doc comments
* Fix test methods names

* Fix staging api usage

* Fix `get_disputes` runtime function implementation

* Fix compilation error

* Fix arithmetic operations in tests

* Use smaller test data

* Rename `RuntimeApiRequest::StagingDisputes` to `RuntimeApiRequest::Disputes`

* Remove `staging-client` feature flag

* fmt

* Remove `vstaging` feature flag

* Some comments regarding the staging api

* Rename dispute selection modules in provisioner
with_staging_api -> prioritized_selection
without_staging_api -> random_selection

* Comments for staging api

* Comments

* Additional logging

* Code review feedback

process_selected_disputes -> into_multi_dispute_statement_set
typo
In trait VoteType: vote_value -> is_valid

* Code review feedback

* Fix metrics

* get_disputes -> disputes

* Get time only once during partitioning

* Fix partitioning

* Comments

* Reduce the number of hardcoded api versions

* Code review feedback

* Unused import

* Comments

* More precise log messages

* Code review feedback

* Code review feedback

* Code review feedback - remove `trait VoteType`

* Code review feedback

* Trace log for DisputeCoordinatorMessage::QueryCandidateVotes counter in vote_selection
      6ae9720c
  51. Sep 15, 2022
  53. Sep 14, 2022
  55. Sep 09, 2022
  57. Sep 06, 2022
  59. Sep 05, 2022
  61. Sep 02, 2022