Skip to content
  1. Jul 04, 2023
    • Bastian Köcher's avatar
      Do not depend on native runtimes for `RuntimeApi` (#7451) · e53d15aa
      Bastian Köcher authored
      * Implement runtime apis for fake runtime
      
      These runtime api implementations are only used to make the compiler
      think that we have implemented all required runtime apis. They will not
      be called as we switch the executor to `WasmExecutor`. In the near
      future we will not require these fake implementations anymore after
      Substrate has shifted away from this compile time requirement.
      
      This brings us the advantage that the `polkadot-service` doesn't need to
      depend on the runtimes for getting the `RuntimeApi` type.
      
      It also removes around 1min of build time on my machine ;)
      
      * Fix warning
      
      * FMT
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      * Use more descriptive id
      
      * Fix warnings
      
      * Adapt path
      
      * Fix 🙈
      
      ---------
      
      Co-authored-by: command-bot <>
      e53d15aa
  2. Jun 29, 2023
  3. Jun 23, 2023
  4. Jun 22, 2023
  5. Jun 21, 2023
  6. Jun 16, 2023
    • juangirini's avatar
      Companion for Soft deprecate GenesisConfig (#7288) · ced77b3d
      juangirini authored
      * update genesis config
      
      * fix wrong RuntimeGenesisConfig name
      
      * update lockfile for {"substrate"}
      
      ---------
      
      Co-authored-by: parity-processbot <>
      ced77b3d
    • Mira Ressel's avatar
      docs: update references to NetworkConfiguration::extra_sets (#7386) · e7560710
      Mira Ressel authored
      * docs: update references to NetworkConfiguration::extra_sets
      
      Since paritytech/substrate#14080, this struct field no longer exists,
      now the `add_notification_protocol()` function of
      `sc_network::config::FullNetworkConfiguration` is used.
      
      Also neuter the doc links for now; rustdoc can't resolve them
      (presumably because sc_network::config isn't in scope, though weirdly
      enough even spelling the link out as
      ``[`FullNetworkConfiguration`](struct@sc_network::config::FullNetworkConfiguration)``
      doesn't work?). Normally this wouldn't be an issue and rustdoc would
      just not generate links, but rust 1.70 has a bug that completely crashes
      rustdoc in this case.
      e7560710
    • Aaro Altonen's avatar
      Wait peers to connect before doing anything else (#7375) · 134655ee
      Aaro Altonen authored
      There is a race condition in `NetworkBridgeRx` between registering
      incoming peers and handling overseer signals. While the peer connection
      was sent to `NetworkBridgeRx` first, sometimes the peer would be added to
      `validation/collation_peers` with enough delay that the bridge handled
      an overseer signal that was meant do send notifications to the registered
      peer and as the peer was not present in the bridge, the notification was
      never sent and the test would hang.
      
      When peers are registered to `NetworkBridgeRx` using `connect_peer()`,
      wait until they show up in `shared.validation/collation_peers` before
      doing anything else.
      
      Co-authored-by: parity-processbot <>
      134655ee
  7. Jun 15, 2023
    • Andrei Eres's avatar
      Delay reputation updates (#7214) · 0a1bc654
      Andrei Eres authored
      
      
      * Add futures-timer
      
      * Make cost_or_benefit public
      
      * Update ReportPeer message format
      
      * Add delay to reputation updates (dirtywork)
      
      * Update ReputationAggregator
      
      * Update tests
      
      * Fix flucky tests
      
      * Move reputation to state
      
      * Use the main loop for handling reputation sendings
      
      * Update
      
      * Move reputation to utils
      
      * Update reputation sending
      
      * Fix arguments order
      
      * Update state
      
      * Remove new from state
      
      * Add constant
      
      * Add failing test for delay
      
      * Change mocking approach
      
      * Fix type errors
      
      * Fix comments
      
      * Add message handling to select
      
      * Fix bitfields-distribution tests
      
      * Add docs to reputation aggregator
      
      * Replace .into_base_rep
      
      * Use one REPUTATION_CHANGE_INTERVAL by default
      
      * Add reputation change to statement-distribution
      
      * Update polkadot-availability-bitfield-distribution
      
      * Update futures selecting in subsystems
      
      * Update reputation adding
      
      * Send malicious changes right away without adding to state
      
      * Add reputation to StatementDistributionSubsystem
      
      * Handle reputation in statement distribution
      
      * Add delay test for polkadot-statement-distribution
      
      * Fix collator-protocol tests before applying reputation delay
      
      * Remove into_base_rep
      
      * Add reputation to State
      
      * Fix failed tests
      
      * Add reputation delay
      
      * Update tests
      
      * Add batched network message for peer reporting
      
      * Update approval-distribution tests
      
      * Update bitfield-distribution tests
      
      * Update statement-distribution tests
      
      * Update collator-protocol tests
      
      * Remove levels in matching
      
      * Address clippy errors
      
      * Fix overseer test
      
      * Add a metric for original count of rep changes
      
      * Update Reputation
      
      * Revert "Add a metric for original count of rep changes"
      
      This reverts commit 6c9b0c1ec34491d16e562bdcba8db6b9dcf484db.
      
      * Update node/subsystem-util/src/reputation.rs
      
      Co-authored-by: default avatarVsevolod Stakhov <[email protected]>
      
      * Remove redundant vec
      
      ---------
      
      Co-authored-by: default avatarVsevolod Stakhov <[email protected]>
      0a1bc654
  8. Jun 14, 2023
    • Andrei Sandu's avatar
      adjust buckets (#7368) · edfbbc8c
      Andrei Sandu authored
      
      
      Signed-off-by: default avatarAndrei Sandu <[email protected]>
      edfbbc8c
    • eskimor's avatar
      Unify code paths of create_inherent and enter (#7137) · 596a9ccd
      eskimor authored
      
      
      * Remove redundant enter call.
      
      * Remove optionality in dispute signature checking
      
      * Make enter_inner and create_inherent the same.
      
      * Remove redundant metric.
      
      * Unification: enter and create_inherent.
      
      * Remove `enter_inner` function.
      
      * Remove dead code.
      
      * Remove redundant import.
      
      * Remove dead code in disputes.
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent
      
      * ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent
      
      * ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent
      
      * Merge fix.
      
      * Fix tests.
      
      * Remove obsolete comment.
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent
      
      * ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent
      
      * ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent
      
      * Review remarks, fixes.
      
      * Guide updates.
      
      * Fmt.
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent
      
      ---------
      
      Co-authored-by: default avatareskimor <[email protected]>
      Co-authored-by: command-bot <>
      596a9ccd
  9. Jun 12, 2023
  10. Jun 11, 2023
  11. Jun 08, 2023
  12. Jun 06, 2023
  13. Jun 05, 2023
    • ordian's avatar
      dispute-coordinator: past session dispute slashing (#6811) · 01a19b45
      ordian authored
      
      
      * runtime/vstaging: unapplied_slashes runtime API
      
      * runtime/vstaging: key_ownership_proof runtime API
      
      * runtime/ParachainHost: submit_report_dispute_lost
      
      * fix key_ownership_proof API
      
      * runtime: submit_report_dispute_lost runtime API
      
      * nits
      
      * Update node/subsystem-types/src/messages.rs
      
      Co-authored-by: default avatarMarcin S. <[email protected]>
      
      * revert unrelated fmt changes
      
      * dispute-coordinator: past session dispute slashing
      
      * encapsule runtime api call for submitting report
      
      * prettify: extract into a function
      
      * do not exit on runtime api error
      
      * fix tests
      
      * try initial zombienet test
      
      * try something
      
      * fix a typo
      
      * try cumulus-based collator
      
      * fix clippy
      
      * build polkadot-debug images with fast-runtime enabled
      
      * wip
      
      * runtime/inclusion: fix availability_threshold
      
      * fix wip
      
      * fix wip II
      
      * revert native provider
      
      * propagate tx submission error
      
      * DEBUG: sync=trace
      
      * print key ownership proof len
      
      * panic repro
      
      * log validator index in panic message
      
      * post merge fixes
      
      * replace debug assertion with a log
      
      * fix compilation
      
      * Let's log the dispatch info in validate block.
      
      * fix double encoding
      
      * Revert "Let's log the dispatch info in validate block."
      
      This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4.
      
      * Revert "Let's log the dispatch info in validate block."
      
      This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4.
      
      * fix compilation
      
      * update to latest zombienet and fix test
      
      * lower finality lag to 11
      
      * bump zombienet again
      
      * add a workaround, but still does not work
      
      * Update .gitlab-ci.yml
      
      bump zombienet.
      
      * add a comment and search logs on all nodes
      
      ---------
      
      Co-authored-by: default avatarMarcin S. <[email protected]>
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      Co-authored-by: default avatarJavier Viola <[email protected]>
      01a19b45
  14. Jun 02, 2023
    • Marcin S.'s avatar
      PVF: Instantiate wasm in pre-checking (#7246) · 5bbb87c4
      Marcin S. authored
      * PVF: Instantiate wasm in pre-checking
      
      * Move `runtime_construction_check` to prepare thread, use bytes
      
      * [minor] Update comment
      
      * Fix compile error
      
      * Update Cargo.lock
      
      * Update docs
      
      * Add some missing docs!
      5bbb87c4
  15. May 31, 2023
  16. May 30, 2023
    • Tsvetomir Dimitrov's avatar
      `RollingSessionWindow` cleanup (#7204) · eb1ed63b
      Tsvetomir Dimitrov authored
      
      
      * Replace `RollingSessionWindow` with `RuntimeInfo` - initial commit
      
      * Fix tests in import
      
      * Fix the rest of the tests
      
      * Remove dead code
      
      * Fix todos
      
      * Simplify session caching
      
      * Comments for `SessionInfoProvider`
      
      * Separate `SessionInfoProvider` from `State`
      
      * `cache_session_info_for_head` becomes freestanding function
      
      * Remove unneeded `mut` usage
      
      * fn session_info -> fn get_session_info() to avoid name clashes. The function also tries to initialize `SessionInfoProvider`
      
      * Fix SessionInfo retrieval
      
      * Code cleanup
      
      * Don't wrap `SessionInfoProvider` in an `Option`
      
      * Remove `earliest_session()`
      
      * Remove pre-caching -> wip
      
      * Fix some tests and code cleanup
      
      * Fix all tests
      
      * Fixes in tests
      
      * Fix comments, variable names and small style changes
      
      * Fix a warning
      
      * impl From<SessionWindowSize> for NonZeroUsize
      
      * Fix logging for `get_session_info` - remove redundant logs and decrease log level to DEBUG
      
      * Code review feedback
      
      * Storage migration removing `COL_SESSION_WINDOW_DATA` from parachains db
      
      * Remove `col_session_data` usages
      
      * Storage migration clearing columns w/o removing them
      
      * Remove session data column usages from `approval-voting` and `dispute-coordinator` tests
      
      * Add some test cases from `RollingSessionWindow` to `dispute-coordinator` tests
      
      * Fix formatting in initialized.rs
      
      * Fix a corner case in `SessionInfo` caching for `dispute-coordinator`
      
      * Remove `RollingSessionWindow` ;(
      
      * Revert "Fix formatting in initialized.rs"
      
      This reverts commit 0f94664ec9f3a7e3737a30291195990e1e7065fc.
      
      * v2 to v3 migration drops `COL_DISPUTE_COORDINATOR_DATA` instead of clearing it
      
      * Fix `NUM_COLUMNS` in `approval-voting`
      
      * Use `columns::v3::NUM_COLUMNS` when opening db
      
      * Update node/service/src/parachains_db/upgrade.rs
      
      Co-authored-by: default avatarAndrei Sandu <[email protected]>
      
      * Don't write in `COL_DISPUTE_COORDINATOR_DATA` for `test_rocksdb_migrate_2_to_3`
      
      * Fix `NUM+COLUMNS` in approval_voting
      
      * Fix formatting
      
      * Fix columns usage
      
      * Clarification comments about the different db versions
      
      ---------
      
      Co-authored-by: default avatarAndrei Sandu <[email protected]>
      eb1ed63b
  17. May 29, 2023
  18. May 26, 2023
  19. May 25, 2023
    • Marcin S.'s avatar
      PVF: Refactor workers into separate crates, remove host dependency (#7253) · 8782dde4
      Marcin S. authored
      * PVF: Refactor workers into separate crates, remove host dependency
      
      * Fix compile error
      
      * Remove some leftover code
      
      * Fix compile errors
      
      * Update Cargo.lock
      
      * Remove worker main.rs files
      
      I accidentally copied these from the other PR. This PR isn't intended to
      introduce standalone workers yet.
      
      * Address review comments
      
      * cargo fmt
      
      * Update a couple of comments
      
      * Update log targets
      8782dde4
  20. May 24, 2023
  21. May 23, 2023
  22. May 19, 2023
  23. May 17, 2023
    • Tsvetomir Dimitrov's avatar
      Revert chain if at least f+1 validators voted against a candidate (#7151) · 0759495c
      Tsvetomir Dimitrov authored
      * Make `issue_explicit_statement_with_index` regular function
      
      * Make `issue_backing_statement_with_index` regular function
      
      * Issue `RevertBlocks` as soon as a dispute has `byzantine threshold + 1` invalid votes.
      
      * Remove a comment
      
      * Fix `has_fresh_byzantine_threshold_against()`
      
      * Extend `informs_chain_selection_when_dispute_concluded_against` test
      0759495c
  24. May 16, 2023
    • Marcin S.'s avatar
      PVF: Vote invalid on panics in execution thread (after a retry) (#7155) · 82e4dbcc
      Marcin S. authored
      
      
      * PVF: Remove `rayon` and some uses of `tokio`
      
      1. We were using `rayon` to spawn a superfluous thread to do execution, so it was removed.
      
      2. We were using `rayon` to set a threadpool-specific thread stack size, and AFAIK we couldn't do that with `tokio` (it's possible [per-runtime](https://docs.rs/tokio/latest/tokio/runtime/struct.Builder.html#method.thread_stack_size) but not per-thread). Since we want to remove `tokio` from the workers [anyway](https://github.com/paritytech/polkadot/issues/7117), I changed it to spawn threads with the `std::thread` API instead of `tokio`.[^1]
      
      [^1]: NOTE: This PR does not totally remove the `tokio` dependency just yet.
      
      3. Since `std::thread` API is not async, we could no longer `select!` on the threads as futures, so the `select!` was changed to a naive loop.
      
      4. The order of thread selection was flipped to make (3) sound (see note in code).
      
      I left some TODO's related to panics which I'm going to address soon as part of https://github.com/paritytech/polkadot/issues/7045.
      
      * PVF: Vote invalid on panics in execution thread (after a retry)
      
      Also make sure we kill the worker process on panic errors and internal errors to
      potentially clear any error states independent of the candidate.
      
      * Address a couple of TODOs
      
      Addresses a couple of follow-up TODOs from
      https://github.com/paritytech/polkadot/pull/7153.
      
      * Add some documentation to implementer's guide
      
      * Fix compile error
      
      * Fix compile errors
      
      * Fix compile error
      
      * Update roadmap/implementers-guide/src/node/utility/candidate-validation.md
      
      Co-authored-by: default avatarAndrei Sandu <[email protected]>
      
      * Address comments + couple other changes (see message)
      
      - Measure the CPU time in the prepare thread, so the observed time is not
        affected by any delays in joining on the thread.
      
      - Measure the full CPU time in the execute thread.
      
      * Implement proper thread synchronization
      
      Use condvars i.e. `Arc::new((Mutex::new(true), Condvar::new()))` as per the std
      docs.
      
      Considered also using a condvar to signal the CPU thread to end, in place of an
      mpsc channel. This was not done because `Condvar::wait_timeout_while` is
      documented as being imprecise, and `mpsc::Receiver::recv_timeout` is not
      documented as such. Also, we would need a separate condvar, to avoid this case:
      the worker thread finishes its job, notifies the condvar, the CPU thread returns
      first, and we join on it and not the worker thread. So it was simpler to leave
      this part as is.
      
      * Catch panics in threads so we always notify condvar
      
      * Use `WaitOutcome` enum instead of bool condition variable
      
      * Fix retry timeouts to depend on exec timeout kind
      
      * Address review comments
      
      * Make the API for condvars in workers nicer
      
      * Add a doc
      
      * Use condvar for memory stats thread
      
      * Small refactor
      
      * Enumerate internal validation errors in an enum
      
      * Fix comment
      
      * Add a log
      
      * Fix test
      
      * Update variant naming
      
      * Address a missed TODO
      
      ---------
      
      Co-authored-by: default avatarAndrei Sandu <[email protected]>
      82e4dbcc
    • Marcin S.'s avatar
      b75b137b
  25. May 15, 2023
  26. May 12, 2023
    • Tsvetomir Dimitrov's avatar
      Replace `RollingSessionWindow` in approval-voting with `RuntimeInfo` (#7123) · 12dd9277
      Tsvetomir Dimitrov authored
      * Replace `RollingSessionWindow` with `RuntimeInfo` - initial commit
      
      * Fix tests in import
      
      * Fix the rest of the tests
      
      * Remove dead code
      
      * Fix todos
      
      * Simplify session caching
      
      * Comments for `SessionInfoProvider`
      
      * Separate `SessionInfoProvider` from `State`
      
      * `cache_session_info_for_head` becomes freestanding function
      
      * Remove unneeded `mut` usage
      
      * fn session_info -> fn get_session_info() to avoid name clashes. The function also tries to initialize `SessionInfoProvider`
      
      * Fix SessionInfo retrieval
      
      * Code cleanup
      
      * Don't wrap `SessionInfoProvider` in an `Option`
      
      * Remove `earliest_session()`
      
      * Remove pre-caching -> wip
      
      * Fix some tests and code cleanup
      
      * Fix all tests
      
      * Fixes in tests
      
      * Fix comments, variable names and small style changes
      
      * Fix a warning
      
      * impl From<SessionWindowSize> for NonZeroUsize
      
      * Fix logging for `get_session_info` - remove redundant logs and decrease log level to DEBUG
      
      * Code review feedback
      12dd9277