Skip to content
  1. Mar 27, 2024
  3. Mar 25, 2024
  5. Mar 19, 2024
  7. Mar 13, 2024
  9. Mar 01, 2024
    • Andrei Eres's avatar
      subsystem-bench: add regression tests for availability read and write (#3311) · f0e589d7
      Andrei Eres authored 
      ### What's been done
- `subsystem-bench` has been split into two parts: a cli benchmark
runner and a library.
- The cli runner is quite simple. It just allows us to run `.yaml` based
test sequences. Now it should only be used to run benchmarks during
development.
- The library is used in the cli runner and in regression tests. Some
code is changed to make the library independent of the runner.
- Added first regression tests for availability read and write that
replicate existing test sequences.

### How we run regression tests
- Regression tests are simply rust integration tests without the
harnesses.
- They should only be compiled under the `subsystem-benchmarks` feature
to prevent them from running with other tests.
- This doesn't work when running tests with `nextest` in CI, so
additional filters have been added to the `nextest` runs.
- Each benchmark run takes a different time in the beginning, so we
"warm up" the tests until their CPU usage differs by only 1%.
- After the warm-up, we run the benchmarks a few more times and compare
the average with the exception using a precision.

### What is still wrong?
- I haven't managed to set up approval voting tests. The spread of their
results is too large and can't be narrowed down in a reasonable amount
of time in the warm-up phase.
- The tests start an unconfigurable prometheus endpoint inside, which
causes errors because they use the same 9999 port. I disable it with a
flag, but I think it's better to extract the endpoint launching outside
the test, as we already do with `valgrind` and `pyroscope`. But we still
use `prometheus` inside the tests.

### Future work
* https://github.com/paritytech/polkadot-sdk/issues/3528
* https://github.com/paritytech/polkadot-sdk/issues/3529
* https://github.com/paritytech/polkadot-sdk/issues/3530
* https://github.com/paritytech/polkadot-sdk/issues/3531



---------

Co-authored-by: default avatarAlexander Samusev <[email protected]>
      f0e589d7
  11. Feb 26, 2024
  13. Feb 09, 2024
  15. Jan 11, 2024
  17. Jan 08, 2024
  19. Dec 31, 2023
  21. Dec 18, 2023
  23. Dec 12, 2023
  25. Dec 05, 2023
  27. Nov 29, 2023
  29. Nov 28, 2023
  31. Nov 25, 2023
  33. Nov 22, 2023
  35. Oct 31, 2023
  37. Oct 17, 2023
  39. Oct 11, 2023
  41. Sep 29, 2023
    • Bastian Köcher's avatar
      Remove kusama and polkadot runtime crates (#1731) · bf90cb0b
      Bastian Köcher authored 
      This pull request is removing the Kusama and Polkadot runtime crates. As
still some crates dependent on the runtime crates, this pull request is
doing some more changes.

- It removes the `hostperfcheck` CLI command. This CLI command could
compare the current node against the standard hardware by doing some
checks. Later we added the hardware benchmark feature to Substrate. This
hardware benchmark is running on every node startup and prints a warning
if the current node is too slow. This makes this CLI command a duplicate
that was also depending on the kusama runtime.

- The pull request is removing the emulated integration tests that were
requiring the Kusama or Polkadot runtime crates.
      bf90cb0b
  43. Sep 22, 2023
  45. Sep 11, 2023
  47. Sep 09, 2023
  49. Sep 07, 2023
  51. Sep 04, 2023
  53. Aug 30, 2023
  55. Aug 29, 2023
  57. Aug 28, 2023
    • Alexander Samusev's avatar
      [ci] Add missing components to CI (#1201) · 6e394f84
      Alexander Samusev authored 
      * [ci] Add missing components to CI

* add timestamp

* add pipeline-stopper-artifacts to jobs

* break clippy

* mv definition

* fix clippy

* comment timestamp, rm verbose

* commen cancel

* rm --verbose from build jobs

* disable rusty-cachier before_script

* enable timestamp back

* rollback .gitignore

* rollback .gitignore

* rollback .gitignore
      6e394f84
  59. Aug 26, 2023
  61. Aug 25, 2023