Skip to content
Snippets Groups Projects
Select Git revision
0 results
Compare History
user avatar
subsystem-bench: add regression tests for availability read and write (#3311)
Andrei Eres authored 
### What's been done
- `subsystem-bench` has been split into two parts: a cli benchmark
runner and a library.
- The cli runner is quite simple. It just allows us to run `.yaml` based
test sequences. Now it should only be used to run benchmarks during
development.
- The library is used in the cli runner and in regression tests. Some
code is changed to make the library independent of the runner.
- Added first regression tests for availability read and write that
replicate existing test sequences.

### How we run regression tests
- Regression tests are simply rust integration tests without the
harnesses.
- They should only be compiled under the `subsystem-benchmarks` feature
to prevent them from running with other tests.
- This doesn't work when running tests with `nextest` in CI, so
additional filters have been added to the `nextest` runs.
- Each benchmark run takes a different time in the beginning, so we
"warm up" the tests until their CPU usage differs by only 1%.
- After the warm-up, we run the benchmarks a few more times and compare
the average with the exception using a precision.

### What is still wrong?
- I haven't managed to set up approval voting tests. The spread of their
results is too large and can't be narrowed down in a reasonable amount
of time in the warm-up phase.
- The tests start an unconfigurable prometheus endpoint inside, which
causes errors because they use the same 9999 port. I disable it with a
flag, but I think it's better to extract the endpoint launching outside
the test, as we already do with `valgrind` and `pyroscope`. But we still
use `prometheus` inside the tests.

### Future work
* https://github.com/paritytech/polkadot-sdk/issues/3528
* https://github.com/paritytech/polkadot-sdk/issues/3529
* https://github.com/paritytech/polkadot-sdk/issues/3530
* https://github.com/paritytech/polkadot-sdk/issues/3531

---------

Co-authored-by: default avatarAlexander Samusev <41779041+alvicsam@users.noreply.github.com>
f0e589d7
Name Last commit Last update
.cargo
.config
.github
.gitlab
bridges
cumulus
docker
docs
polkadot
prdoc
scripts
substrate
.gitignore
.gitlab-ci.yml
.prdoc.toml
.rustfmt.toml
BRIDGES.md
Cargo.lock
Cargo.toml
README.md
README.md

NOTE: We have recently made significant changes to our repository structure. In order to streamline our development process and foster better contributions, we have merged three separate repositories Cumulus, Substrate and Polkadot into this repository. Read more about the changes here.

Polkadot SDK

StackExchange

The Polkadot SDK repository provides all the resources needed to start building on the Polkadot network, a multi-chain blockchain platform that enables different blockchains to interoperate and share information in a secure and scalable way. The Polkadot SDK comprises three main pieces of software:

Polkadot

PolkadotForum Polkadot-license

Implementation of a node for the https://polkadot.network in Rust, using the Substrate framework. This directory currently contains runtimes for the Westend and Rococo test networks. Polkadot, Kusama and their system chain runtimes are located in the runtimes repository maintained by the Polkadot Technical Fellowship.

Substrate

SubstrateRustDocs Substrate-license

Substrate is the primary blockchain SDK used by developers to create the parachains that make up the Polkadot network. Additionally, it allows for the development of self-sovereign blockchains that operate completely independently of Polkadot.

Cumulus

CumulusRustDocs Cumulus-license

Cumulus is a set of tools for writing Substrate-based Polkadot parachains.

Releases

[!NOTE]
Our release process is still Work-In-Progress and may not yet reflect the aspired outline here.

The Polkadot-SDK has two release channels: stable and nightly. Production software is advised to only use stable. nightly is meant for tinkerers to try out the latest features. The detailed release process is described in RELEASE.md.

Stable

stable releases have a support duration of three months. In this period, the release will not have any breaking changes. It will receive bug fixes, security fixes, performance fixes and new non-breaking features on a two week cadence.

Nightly

nightly releases are released every night from the master branch, potentially with breaking changes. They have pre-release version numbers in the format major.0.0-nightlyYYMMDD.

Upstream Dependencies

Below are the primary upstream dependencies utilized in this project:

Security

The security policy and procedures can be found in docs/contributor/SECURITY.md.

Contributing & Code of Conduct

Ensure you follow our contribution guidelines. In every interaction and contribution, this project adheres to the Contributor Covenant Code of Conduct.

Additional Resources

  • For monitoring upcoming changes and current proposals related to the technical implementation of the Polkadot network, visit the Requests for Comment (RFC) repository. While it's maintained by the Polkadot Fellowship, the RFC process welcomes contributions from everyone.