Skip to content
  1. Apr 19, 2022
    • asynchronous rob's avatar
      Reduce network bandwidth, improve parablock times: optimize approval-distribution (#5164) · 79ecc538
      asynchronous rob authored 
      

* gossip-support: be explicit about dimensions

* some guide updates

* update network-bridge to distinguish x and y dimensions

* get everything to compile

* beginnings

* some TODOs

* polkadot runtime: use relevant_authorities

* make gossip topologies per-session

* better formatting

* gossip support: use current session validators

* expand in comment

* adjust tests and fix index bug

* add past/present/future connection test and clean up code

* fmt

* network bridge: updated types

* update protocols to new gossip topology message

* guide updates

* add session to BlockApprovalMeta

* add session to block info

* refactor knowledge and remove most unify logic

* start replacing gossip_peers with new SessionTopologies

* add routing information to message state

* add some utilities to SessionTopology

* implement new gossip topology logic

* re-implement unify_with_peer

* distribute assignments according to topology

* finish grid topology implementation

* refactor network bridge slightly

* issue connection requests on all past/present/future

* fmt

* address grumbles

* tighten invariants in unify_with_peer

* implement random propagation

* refactor: extract required routing adjustment logic

* some block-age logic

* aggressively propagate messages when finality is slow

* overhaul aggression system to have 3 levels

* add aggression metrics

* remove aggression L3

* reduce random circulation

* remove PeerData

* get approval tests compiling

* use btree_map in known_by to make deterministic

* Revert "use btree_map in known_by to make deterministic"

This reverts commit 330d65343a7bb6fe4dd0f24bd8dbc15c0cbdbd9d.

* test XY grid propagation

* remove stray println

* test unshared dimension propagation

* add random gossip check

* test unify_with_peer better

* test sending after getting gossip topology

* test L1 aggression on originator

* test L1 aggression for non-originators

* test non-originator aggression L2

* fnt

* ~spellcheck

* fix statement-distribution tests

* fix flaky test

* fix metrics typo

* re-send periodically

* test resending

* typo

Co-authored-by: default avatarBernhard Schuster <[email protected]>

* add more metrics about apd messages

* add back unify_with_peer logs

* make Resend an enum

* be more explicit when resending

* fmt

* fix error

* add a TODO for refactoring

* remove debug metrics

* add some guide stuff

* fmt

* update runtime API in test-runtim

Co-authored-by: default avatarBernhard Schuster <[email protected]>
      79ecc538
    • asynchronous rob's avatar
      flatten dispute-coordinator 'real' module. (#5254) · edfa24bb
      asynchronous rob authored 
      

* flatton dispute-coordinator 'real' module,

* Drop obsolete modules.

Co-authored-by: default avatarRobert Klotzner <[email protected]>
      edfa24bb
    • Robert Klotzner's avatar
      dispute-coordinator: Cleanup + Bug fixes (#5323) · f820db49
      Robert Klotzner authored 
      * Make import confirmation oneshot optional.

* Cleanup for further improvements.

* Queue adoptions.

* Fix fieldname

* Use correct relay parent

* Fix scraper tests.

* Small optimization.

* Fix all tests.

* Fix other tests.

* fmt

* spelling

* Fix warning.
      f820db49
  3. Apr 18, 2022
  5. Apr 16, 2022
  7. Apr 15, 2022
    • Oliver Tale-Yazdi's avatar
      Co #11198: Expose `benchmark machine` command (#5326) · 8c3dbe0c
      Oliver Tale-Yazdi authored 
      

* Expose benchmark machine command

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>

* error.into

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

* Revert "update lockfile for {"substrate"}"

This reverts commit 41c2ddda511076567a98c85efa631374071c9e5a.

* cargo update -p sp-io

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>

Co-authored-by: parity-processbot <>
      8c3dbe0c
    • Georges's avatar
      Companion for Adding benchmarking for new `frame_election_provider_support` #11149 (#5241) · a74c527d
      Georges authored 
      

* `GenesisElectionOf` after changes to substrate

* Benchmarking for `frame_election_provider_support`

* Introducing `OnChainSeqPhragmen` after changes
to substrate.
Pallet renaming in substrate

* Fixing `test-runtime`

* Use `UnboundedExecution` after change to substrate

* update lockfile for {"substrate"}

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark pallet --chain=polkadot-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark pallet --chain=kusama-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark pallet --chain=westend-dev --steps=50 --repeat=20 --pallet=frame_election_provider_support --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Co-authored-by: parity-processbot <>
Co-authored-by: default avatarParity Bot <[email protected]>
      a74c527d
    • dependabot[bot]'s avatar
      Bump scale-info from 2.0.1 to 2.1.1 (#5325) · e3326d26
      dependabot[bot] authored 
      

Bumps [scale-info](https://github.com/paritytech/scale-info) from 2.0.1 to 2.1.1.
- [Release notes](https://github.com/paritytech/scale-info/releases)
- [Changelog](https://github.com/paritytech/scale-info/blob/master/CHANGELOG.md)
- [Commits](https://github.com/paritytech/scale-info/commits/v2.1.1)

---
updated-dependencies:
- dependency-name: scale-info
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: default avatardependabot[bot] <[email protected]>

Co-authored-by: default avatardependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
      e3326d26
  9. Apr 14, 2022
    • dependabot[bot]'s avatar
      Bump quote from 1.0.17 to 1.0.18 (#5301) · b9e0e149
      dependabot[bot] authored 
      

Bumps [quote](https://github.com/dtolnay/quote) from 1.0.17 to 1.0.18.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](https://github.com/dtolnay/quote/compare/1.0.17...1.0.18)

---
updated-dependencies:
- dependency-name: quote
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: default avatardependabot[bot] <[email protected]>

Co-authored-by: default avatardependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
      b9e0e149
    • Nazar Mokrynskyi's avatar
      Update zstd dependency (#5148) · 2f01960e
      Nazar Mokrynskyi authored 
      

* Update zstd dependency

* Update Cargo.lock

Co-authored-by: default avatarBastian Köcher <[email protected]>
      2f01960e
    • Shawn Tabrizi's avatar
      Add XCM Benchmarks to Kusama (#4442) · 3f56e1ff
      Shawn Tabrizi authored 
      

* add xcm benchmarks to kusama

* Create temp

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/kusama/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::fungible --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/kusama/src/weights/xcm/pallet_xcm_benchmarks_fungible.rs

* add weights to kusama runtime

* Fixes

* Fixes

* Fixes

* Fixes

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::fungible --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/kusama/src/weights/xcm/pallet_xcm_benchmarks_fungible.rs

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/kusama/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

* use new benchmark

* Fix license year

* update substrate deps

* Add Configuration::ActiveConfig storage key to the whitelist

* Update Cargo.lock

Co-authored-by: default avatarParity Bot <[email protected]>
Co-authored-by: default avatarKeith Yeung <[email protected]>
      3f56e1ff
  11. Apr 13, 2022
  13. Apr 12, 2022
    • dependabot[bot]'s avatar
      Bump tracing from 0.1.32 to 0.1.33 (#5299) · c8423f51
      dependabot[bot] authored 
      

Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.32 to 0.1.33.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.32...tracing-0.1.33)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: default avatardependabot[bot] <[email protected]>

Co-authored-by: default avatardependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
      c8423f51
    • Tsvetomir Dimitrov's avatar
      Add staging runtime api (#5048) · fd020c07
      Tsvetomir Dimitrov authored 
      * Move `trait ParachainHost` to a separate version independent module

`trait ParachainHost` is no longer part of a specific primitives
version. Instead there is a single trait for stable and staging api
versions. The trait contains stable AND staging methods. The latter are
explicitly marked as unstable.

* Fix `use` primitives

`polkadot_primitives::v2` becomes `polkadot_primitives::runtime_api`

* Staging API declaration and stubs

Introduces the concept for 'staging functions' in runtime API. These
functions are still in testing and they are meant to be used only
within test networks (Westend).
They coexist with the stable calls for technical reasons - maintaining
different runtime APIs for different networks is hard to implement.

Check the doc comments in source files for more details how the staging
API should be used.

* Add new staging method - get_session_disputes()

Add `staging_get_session_disputes` to `ParachainHost` as the first
method of the staging API.

* Hide vstaging runtime api implementations  behind feature flag

* Fix test runtime

* fn staging_get_session_disputes() is renamed to fn staging_get_disputes()
      fd020c07
    • Denis_P's avatar
      CI: rename ambiguous jobs (#5313) · 04f5a15b
      Denis_P authored
      04f5a15b
  15. Apr 11, 2022
  17. Apr 09, 2022
    • dependabot[bot]'s avatar
      Bump proc-macro2 from 1.0.36 to 1.0.37 (#5265) · 3fef84a6
      dependabot[bot] authored 
      

Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.36 to 1.0.37.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](https://github.com/dtolnay/proc-macro2/compare/1.0.36...1.0.37)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: default avatardependabot[bot] <[email protected]>

Co-authored-by: default avatardependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
      3fef84a6
    • Sergei Shulepov's avatar
      Fixes the dead lock when any of the channels get at capacity. (#5297) · b89dc00e
      Sergei Shulepov authored 
      The PVF host is designed to avoid spawning tasks to minimize knowledge
of outer code. Using `async_std::task::spawn` (or Tokio's counterpart)
deemed unacceptable, `SpawnNamed` undesirable. Instead there is only one
task returned that is spawned by the candidate-validation subsystem.
The tasks from the sub-components are polled by that root task.

However, the way the tasks are bundled was incorrect. There was a giant
select that was polling those tasks. Particularly, that implies that as soon as
one of the arms of that select goes into await those sub-tasks stop
getting polled. This is a recipe for a deadlock which indeed happened
here.

Specifically, the deadlock happened during sending messages to the
execute queue by calling
[`send_execute`](https://github.com/paritytech/polkadot/blob/a68d9be3/node/core/pvf/src/host.rs#L601).
When the channel to the queue reaches the capacity, the control flow is
suspended until the queue handles those messages. Since this code is
essentially reached from [one of the select
arms](https://github.com/paritytech/polkadot/blob/a68d9be3/node/core/pvf/src/host.rs#L371),
the queue won't be given the control and thus no further progress can be
made.

This problem is solved by bundling the tasks one level higher instead,
by `selecting` over those long-running tasks.

We also stop treating returning from those long-running tasks as error
conditions, since that can happen during legit shutdown.
      b89dc00e
  19. Apr 08, 2022
  21. Apr 07, 2022