Add overhead benchmark to frame-omni-bencher (#5891)
# Benchmark Overhead Command for Parachains
This implements the `benchmark overhead` command for parachains. Full
context is available at:
https://github.com/paritytech/polkadot-sdk/issues/5303. Previous attempt
was this https://github.com/paritytech/polkadot-sdk/pull/5283, but here
we have integration into frame-omni-bencher and improved tooling.
## Changes Overview
Users are now able to use `frame-omni-bencher` to generate
`extrinsic_weight.rs` and `block_weight.rs` files for their runtime. The
core logic for generating these remains untouched; this PR provides
mostly machinery to make it work for parachains at all.
Similar to the pallet benchmarks, we gain the option to benchmark based
on just a runtime:
```
frame-omni-bencher v1 benchmark overhead --runtime {{runtime}}
```
or with a spec:
```
frame-omni-bencher v1 benchmark overhead --chain {{spec}} --genesis-builder spec
```
In this case, the genesis state is generated from the runtime presets.
However, it is also possible to use `--chain` and genesis builder `spec`
to generate the genesis state from the chain spec.
Additionally, we use metadata to perform some checks based on the
pallets the runtime exposes:
- If we see the `ParaInherent` pallet, we assume that we are dealing
with a relay chain. This means that we don't need proof recording during
import (since there is no storage weight).
- If we detect the `ParachainSystem` pallet, we assume that we are
dealing with a parachain and take corresponding actions like patching a
para id into the genesis state.
On the inherent side, I am currently supplying the standard inherents
every parachain needs.
In the current state, `frame-omni-bencher` supports all system chains.
In follow-up PRs, we could add additional inherents to increase
compatibility.
Since we are building a block during the benchmark, we also need to
build an extrinsic. By default, I am leveraging subxt to build the xt
dynamically. If a chain is not compatible with the `SubstrateConfig`
that comes with `subxt`, it can provide a custom extrinsic builder to
benchmarking-cli. This requires either a custom bencher implementation
or an integration into the parachains node.
Also cumulus-test-runtime has been migrated to provide genesis configs.
## Chain Compatibility
The current version here is compatible with the system chains and common
substrate chains. The way to go for others would be to customize the
frame-omni-bencher by providing a custom extrinsicbuilder. I did an
example implementation that works for mythical:
https://github.com/skunert/mythical-bencher
## Follow-Ups
- After #6040 is finished, we should integrate this here to make the
tooling truly useful. In the current form, the state is fairly small and
not representative.
## How to Review
I recommend starting from
[here](https://github.com/paritytech/polkadot-sdk/pull/5891/files#diff-50830ff756b3ac3403b7739d66c9e3a5185dbea550669ca71b28d19c7a2a54ecR264),
this method is the main entry point for omni-bencher and `polkadot`
binary.
TBD:
- [x] PRDoc
---------
Co-authored-by:
Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
parent
6f96f721
Showing
- Cargo.lock 25 additions, 0 deletionsCargo.lock
- cumulus/client/parachain-inherent/src/mock.rs 1 addition, 0 deletionscumulus/client/parachain-inherent/src/mock.rs
- cumulus/polkadot-omni-node/lib/src/command.rs 10 additions, 0 deletionscumulus/polkadot-omni-node/lib/src/command.rs
- cumulus/polkadot-omni-node/src/main.rs 1 addition, 4 deletionscumulus/polkadot-omni-node/src/main.rs
- cumulus/polkadot-parachain/src/main.rs 4 additions, 4 deletionscumulus/polkadot-parachain/src/main.rs
- cumulus/test/client/src/lib.rs 3 additions, 3 deletionscumulus/test/client/src/lib.rs
- cumulus/test/runtime/Cargo.toml 4 additions, 0 deletionscumulus/test/runtime/Cargo.toml
- cumulus/test/runtime/src/genesis_config_presets.rs 72 additions, 0 deletionscumulus/test/runtime/src/genesis_config_presets.rs
- cumulus/test/runtime/src/lib.rs 6 additions, 7 deletionscumulus/test/runtime/src/lib.rs
- cumulus/test/service/Cargo.toml 1 addition, 0 deletionscumulus/test/service/Cargo.toml
- cumulus/test/service/benches/transaction_throughput.rs 3 additions, 3 deletionscumulus/test/service/benches/transaction_throughput.rs
- cumulus/test/service/benches/validate_block.rs 4 additions, 1 deletioncumulus/test/service/benches/validate_block.rs
- cumulus/test/service/src/bench_utils.rs 5 additions, 2 deletionscumulus/test/service/src/bench_utils.rs
- cumulus/test/service/src/chain_spec.rs 42 additions, 40 deletionscumulus/test/service/src/chain_spec.rs
- cumulus/test/service/src/lib.rs 2 additions, 2 deletionscumulus/test/service/src/lib.rs
- cumulus/zombienet/examples/small_network.toml 13 additions, 13 deletionscumulus/zombienet/examples/small_network.toml
- polkadot/cli/src/command.rs 38 additions, 40 deletionspolkadot/cli/src/command.rs
- polkadot/node/service/src/benchmarking.rs 0 additions, 47 deletionspolkadot/node/service/src/benchmarking.rs
- polkadot/tests/benchmark_overhead.rs 0 additions, 8 deletionspolkadot/tests/benchmark_overhead.rs
- prdoc/pr_5891.prdoc 33 additions, 0 deletionsprdoc/pr_5891.prdoc
Please register or sign in to comment