Skip to content
  1. May 04, 2023
  2. Apr 28, 2023
  3. Apr 26, 2023
  4. Apr 25, 2023
  5. Apr 24, 2023
    • Andrei Sandu's avatar
      Fix rolling session window (#7126) · 634b2f6a
      Andrei Sandu authored
      * Use stored sessions 🤦
      
      
      
      Signed-off-by: default avatarAndrei Sandu <[email protected]>
      
      * Add test
      
      Signed-off-by: default avatarAndrei Sandu <[email protected]>
      
      * remove redundant
      
      Signed-off-by: default avatarAndrei Sandu <[email protected]>
      
      ---------
      
      Signed-off-by: default avatarAndrei Sandu <[email protected]>
      634b2f6a
    • Tsvetomir Dimitrov's avatar
      Rework `dispute-coordinator` to use `RuntimeInfo` for obtaining session... · 3f4ce632
      Tsvetomir Dimitrov authored
      
      Rework `dispute-coordinator` to use `RuntimeInfo` for obtaining session information instead of `RollingSessionWindow` (#6968)
      
      * Pass `SessionInfo` directly to `CandidateEnvironment::new` otherwise it should be an async function
      
      * Replace calls to `RollingSessionWindow` with `RuntimeInfo`
      
      Adjust `dispute-coordinator` initialization to use `RuntimeInfo`
      
      * Modify `dispute-coordinator` initialization
      
      * Pass `Hash` to `process_on_chain_votes` so that `RuntimeInfo` calls can be made
      
      Remove some fixmes
      
      * Pass `Hash` to `handle_import_statements` to perform `RuntimeInfo` calls
      
      * remove todo comments
      
      * Remove `error` from `Initialized`
      
      Rework new session handling code
      
      * Remove db code which is no longer used
      
      * Update stale comment and remove unneeded type specification
      
      * Cache SessionInfo on startup
      
      * Use `DISPUTE_WINDOW` from primitives
      
      * Fix caching in `process_active_leaves_update`
      
      * handle_import_statements: leaf_hash -> block_hash
      
      * Restore `ensure_available_session_info`
      
      * Don't interrupt `process_on_chain_votes` if SessionInfo can't be fetched
      
      * Small style improvements in logging
      
      * process_on_chain_votes: leaf_hash -> block_hash
      
      * Restore `note_earliest_session` - it is required to prune disputes and votes
      
      * Cache new sessions only when there is an actual session change
      
      * Fix tests
      
      * `CandidateEnvironment::new` gets `session_idx` and fetches SessionInfo by itself to avoid the invariant where the input SessionIndex and SessionInfo parameters don't match
      
      * Fix handling of missing session info
      
      * Move sessions caching in `handle_startup` and fix tests
      
      * Load `relay_parent` from db in `handle_import_statements` instead of passing it as a parameter via two functions
      
      * Don't do two db reads
      
      * Fix the twisted logic in `handle_import_statements`
      
      * fixup
      
      * Small style fix
      
      * Decrease log levels for caching errors to debug and fix a typo
      
      * Update outdated comment
      
      * Remove `ensure_available_session_info`
      
      * Load relay parent from db in `process_on_chain_votes`
      
      * Revert "Load relay parent from db in `process_on_chain_votes`"
      
      This reverts commit 978ad4f223d517faa7a7fbad96e3f8de4fa17501.
      
      * Keep track of highest seen session and last session cached without gaps.
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarordian <[email protected]>
      
      * Handle session caching failure on startup correctly
      
      * Update node/core/dispute-coordinator/src/initialized.rs
      
      Co-authored-by: default avatarordian <[email protected]>
      
      * Simplify session caching retries
      
      * Update stale comment
      
      * Fix lower bound calculation for session caching
      
      ---------
      
      Co-authored-by: default avatarordian <[email protected]>
      3f4ce632
  6. Apr 21, 2023
    • ordian's avatar
      update rocksdb to 0.20.1 (#7113) · 439ab875
      ordian authored
      * update rocksdb to 0.20.1
      
      * update lockfile for {"substrate"}
      
      ---------
      
      Co-authored-by: parity-processbot <>
      439ab875
    • Marcin S.'s avatar
      PVF: Move PVF workers into separate crate (#7101) · e277f95b
      Marcin S. authored
      * Move PVF workers into separate crate
      
      * Fix indentation
      
      * Fix compilation errors
      
      * Fix more compilation errors
      
      * Rename `worker.rs` files, make host interface to worker more clear
      
      * Fix more compilation errors
      
      * Fix more compilation errors
      
      * Add link to issue
      
      * Address review comments
      
      * Update comment
      e277f95b
  7. Apr 20, 2023
    • Marcin S.'s avatar
      PVF: Don't dispute on missing artifact (#7011) · 0940cdd1
      Marcin S. authored
      * PVF: Don't dispute on missing artifact
      
      A dispute should never be raised if the local cache doesn't provide a certain
      artifact. You can not dispute based on this reason, as it is a local hardware
      issue and not related to the candidate to check.
      
      Design:
      
      Currently we assume that if we prepared an artifact, it remains there on-disk
      until we prune it, i.e. we never check again if it's still there.
      
      We can change it so that instead of artifact-not-found triggering a dispute, we
      retry once (like we do for AmbiguousWorkerDeath, except we don't dispute if it
      still doesn't work). And when enqueuing an execute job, we check for the
      artifact on-disk, and start preparation if not found.
      
      Changes:
      
      - [x] Integration test (should fail without the following changes)
      - [x] Check if artifact exists when executing, prepare if not
      - [x] Return an internal error when file is missing
      - [x] Retry once on internal errors
      - [x] Document design (update impl guide)
      
      * Add some context to wasm error message (it is quite long)
      
      * Fix impl guide
      
      * Add check for missing/inaccessible file
      
      * Add comment referencing Substrate issue
      
      * Add test for retrying internal errors
      
      ---------
      
      Co-authored-by: parity-processbot <>
      0940cdd1
  8. Apr 19, 2023
  9. Apr 18, 2023
  10. Apr 17, 2023
  11. Apr 13, 2023
  12. Apr 12, 2023
    • Sam Johnson's avatar
      Companion for #13846 (#7022) · b3070be0
      Sam Johnson authored
      * globally upgrade syn to 1.0.109
      
      * globally upgrade quote to 1.0.26
      
      * globally upgrade proc-macro2 to 1.0.56
      
      * globally bump syn to v2.0.13
      
      * update expander to v1.0.0
      
      * temporary commit to prove new version of expander works
      
      (new version hasn't been released yet so using git)
      
      * use expander 2.0.0
      
      * upgrade to syn 2.0.14
      
      * update lock file
      b3070be0
  13. Apr 11, 2023
    • Mira Ressel's avatar
      apply clippy 1.68 suggestions · 0e9b0a64
      Mira Ressel authored
      * Simplify &(ref foo, _) tuple pattern matches
      * Drop unneccessary .clone() calls
      * Replace .position(cond).is_some() by .any(cond)
      * Drop unneccessary lifetime specs
      0e9b0a64
  14. Apr 08, 2023
  15. Apr 07, 2023
    • Tsvetomir Dimitrov's avatar
      Onchain scraper in `dispute-coordinator` will scrape... · 63a805ba
      Tsvetomir Dimitrov authored
      Onchain scraper in `dispute-coordinator` will scrape `SCRAPED_FINALIZED_BLOCKS_COUNT` blocks before finality (#7013)
      
      * Onchain scraper in `dispute-coordinator` will scrape `SCRAPED_FINALIZED_BLOCKS_COUNT` blocks before finality
      
      The purpose is to make the availability of a `CandidateReceipt` for finalized candidates more likely.
      
      For details see:  https://github.com/paritytech/polkadot/issues/7009
      
      * Fix off by one error
      
      * Replace `SCRAPED_FINALIZED_BLOCKS_COUNT` with `DISPUTE_CANDIDATE_LIFETIME_AFTER_FINALIZATION`
      63a805ba
    • Marcin S.'s avatar
      PVF: Minor refactor in workers code (#7012) · 5889119f
      Marcin S. authored
      * Move version check to `worker_event_loop`
      
      * More minor refactors
      
      - More consistent use of `format_invalid` and `format_internal`.
      - Fix a doc error.
      - Fix some poorly-named local variables.
      5889119f
  16. Apr 05, 2023
  17. Apr 04, 2023
  18. Apr 03, 2023
  19. Mar 31, 2023
    • Mattia L.V. Bradascio's avatar
      Consolidate subsystem spans so they are all children of the leaf-activated root span (#6458) · 713f6625
      Mattia L.V. Bradascio authored
      * Pass the PerLeafSpan as mutable reference to handle_new_head function
      
      * cargo +nightly fmt --all
      
      * Add mock span for test
      
      * cargo +nightly fmt --all
      
      * add new-blocks-hashes to span
      
      * ref span in match statement, set span to disabled if not passed
      
      * remove second match clause, make handle_new_head_span mutable
      
      * cargo +nightly fmt --all
      
      * improve tag on error and warning
      
      * add imported blocks and info span
      
      * cargo +nightly fmt --all
      
      * Improve error for imported_blocks_and_info trace
      
      * format tags on get_header_span
      
      * add lost-to-finality tag
      
      * add missing bracket
      
      * - Add bitfield child span
      - Add block db insertion span
      
      * - fix update-bitfield span tag
      
      * - Fix type conversion to u64
      - Add missing argument
      
      * - Cargo fmt
      
      * - Test add_follows_from
      
      * - Revert as  relationship between spans not working correctly
      
      * - use drop to test if parent-child relationship can be re-established
      
      * - remove bitfield span, check if parent-child relationship can be reestablished
      
      * - Remove dangling bitfield span which is not used, to see if parent-child relationship can be re-established
      
      * Another dangling bitfield span
      
      * cargo fmt
      
      * - add imported blocks and info span
      - add candidate span per candidate
      
      * add tags before moving block_header to push scope
      
      * - Add db-insertion span
      
      * cargo fmt
      
      * fix types
      
      * * Pass mutable reference to span in handle_new_head
      * Change get-header-span tags in handle_new_head
      * Create cache-session-info span in handle_new_head
      * Create optional argument in determine_new_blocks
      * Pass mutable reference to handle_new_head_span in determine_new_blocks in handle_new_head function
      * Add candidate-hash, candidate-number, lost-to-finality tags to candidate_span in handle_new_head function
      * Manually drop db_insertion_span and remove superfluous tags  to it, only keeping approved-bitfields tag
      * Add ApprovalVoting stage in jaeger
      
      * * Pass mutable reference to jaeger::Span in stead of PerLeafSpan
      * Add block-import span
      
      * *Pass optional_span (optional argument) to determine_new_blocks util function
      
      * * Add num-candidates int tag to block_import_span
      
      * * Add head tag to cache_session_span
      
      * * Create PerLeafSpan in handle_from_overseer (this is required to establish parent-child relationship between approval-voting span, and leaf-activated root span)
      
      * * Add candidate-import-span as child of block-import-span
      * Add candidate-hash and num-approval tags to candidate-import-span
      
      * * Fix num-candidate tag to bitvec-len tag in candidate-import-span
      
      * *Fix imported_blocKs_and_info span to create new-block-span as not dealing with candidates
      
      * Consider the future::select! block
      
      * Use HashMap<Hash, jaeger::PerLeafSpan>
      
      * Remove Stage 9
      
      * Add missing spans
      
      * cargo +nightly fmt --all
      
      * Remove optional span argument for determine_new_blocks
      
      * * Remove no-longer needed default PerLeafSpan implementation
      * Remove no-longer necessary mock span given re-factoring of handle_new_head() no longer neeing mutable span
      * Split validation-result and request-data (availability and validation code) spans into two by dropping request_validation_data_spans
      * Remove drop statements for cache_session_info_span
      *
      
      * Remove unnecessary span
      
      * Remove another excessively spammy span
      
      * Add missing spans from State in import tests
      
      * Use functional approach to get spans
      
      * - Add functional approach for the approval-voting span
      - Add doc on block_numbers given labelling ambiguity
      - Add span pruning logic
      - Use .add_para_id on validation_result_span
      
      * Replace for hash_set in hash_set_iter with map closure
      
      * cargo +nightly fmt --all
      
      * Change from unconsumed `map` to `.for_each`
      
      * cargo +nightly fmt --all
      
      * Refactor add_para_id to validation_result_span
      
      * cargo +nightly fmt --all
      
      * Remove duplicate tag
      
      * Add missing tag to handle-approved-ancestor span
      
      * Refactor span pruning to only invoke retain once
      
      * Typo in span name
      
      * - Replace unwrap_or with unwrap_or_else due to lazy evaluation of trace-identifier in polkadot_node_jaeger
      - Remove some redundant spans
      
      * Add approval-distribution spans
      
      * - Add unwrap_or_else on note-approved-in-chain-selection
      - Use child_with_trace_id to add traceID string tag on span (note this does not change the traceID, but just adds a tag)
      
      * cargo +nightly fmt --all
      
      * - Add traceID tags were necessary in approval-voting and availability-distribution
      - Always use block-hash tag in stead of relay-parent tag in approval-distribution
      
      * Remove schedule-wakeup span as it will duplicate spans on existing wakeups (which should be a no-op)
      
      * Remove a couple of warnings related to mutability
      
      * Fix failing tests in availability distribution
      
      * Add traceID tag to launch-approval and validation-result
      
      * Reshuffle the validation and validation result spans to where more appropriate and add block-hash tag
      
      * - Add tranche and should-trigger tag to process-wakeup span
      - Add candidate-hash and traceID to check-and-import-approval span
      
      * cargo fmt
      
      * - Adjustments after PR comments
      
      * Move span pruning after other pruning logic
      
      * Remove DerefMut - no longer needed
      
      * Relabel request-chunk spans
      
      * - Fix typo in span label
      - Add docs for drops
      
      * Add new approval-voting span pruning logic
      
      * Undo removal of !
      
      * cargo fmt
      713f6625
  20. Mar 30, 2023
  21. Mar 29, 2023
    • s0me0ne-unkn0wn's avatar
      Check spawned worker version vs node version before PVF preparation (#6861) · 55b4aceb
      s0me0ne-unkn0wn authored
      
      
      * Check spawned worker version vs node version before PVF preparation
      
      * Address discussions
      
      * Propagate errors and shutdown preparation and execution pipelines properly
      
      * Add logs; Fix execution worker checks
      
      * Revert "Propagate errors and shutdown preparation and execution pipelines properly"
      
      This reverts commit b96cc3160ff58db5ff001d8ca0bfea9bd4bdd0f2.
      
      * Don't try to shut down; report the condition and exit worker
      
      * Get rid of `VersionMismatch` preparation error
      
      * Merge master
      
      * Add docs; Fix tests
      
      * Update Cargo.lock
      
      * Kill again, but only the main node process
      
      * Move unsafe code to a common safe function
      
      * Fix libc dependency error on MacOS
      
      * pvf spawning: Add some logging, add a small integration test
      
      * Minor fixes
      
      * Restart CI
      
      ---------
      
      Co-authored-by: default avatarMarcin S <[email protected]>
      55b4aceb
    • Oliver Tale-Yazdi's avatar
      Fix feature (#6966) · faa2d5b0
      Oliver Tale-Yazdi authored
      
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      faa2d5b0
  22. Mar 28, 2023
    • Tsvetomir Dimitrov's avatar
      Histogram support in runtime metrics (#6935) · 22f404f0
      Tsvetomir Dimitrov authored
      
      
      * Histogram support in runtime metrics
      
      Add support for histograms to the runtime metrics. Additionally add
      `polkadot_parachain_verify_dispute_signature` histogram which tracks the
      time needed from the runtime to verify a single validator signature of a
      dispute statement.
      
      * Add noops
      
      * u64 instead of f64
      
      * Update buckets
      
      * Wrap `get_current_time()` in runtime metrics
      
      * Change the dimension of the Histogram from usec to sec
      
      * Fix a compilation error
      
      * Update buckets
      
      * Fix `on_signature_check_complete` calculation
      
      * Update buckets
      
      * Update buckets
      
      * formatting
      
      * Another weights update
      
      * Adjust buckets again
      
      * Final buckets adjustment
      
      * Revert "Fix a compilation error"
      
      This reverts commit 06290b40a39eeb78de2602d8916a39edf7a8b714.
      
      * Update primitives/src/v4/metrics.rs
      
      Co-authored-by: default avatarAndrei Sandu <[email protected]>
      
      * Use `saturating_sub` for time difference calculation
      
      * Pass nanoseconds to client instead of seconds (using f64 in runtime is dangerous)
      
      ---------
      
      Co-authored-by: default avatarAndrei Sandu <[email protected]>
      22f404f0
  23. Mar 24, 2023