Add overhead benchmark to frame-omni-bencher (#5891)
# Benchmark Overhead Command for Parachains
This implements the `benchmark overhead` command for parachains. Full
context is available at:
https://github.com/paritytech/polkadot-sdk/issues/5303. Previous attempt
was this https://github.com/paritytech/polkadot-sdk/pull/5283, but here
we have integration into frame-omni-bencher and improved tooling.
## Changes Overview
Users are now able to use `frame-omni-bencher` to generate
`extrinsic_weight.rs` and `block_weight.rs` files for their runtime. The
core logic for generating these remains untouched; this PR provides
mostly machinery to make it work for parachains at all.
Similar to the pallet benchmarks, we gain the option to benchmark based
on just a runtime:
```
frame-omni-bencher v1 benchmark overhead --runtime {{runtime}}
```
or with a spec:
```
frame-omni-bencher v1 benchmark overhead --chain {{spec}} --genesis-builder spec
```
In this case, the genesis state is generated from the runtime presets.
However, it is also possible to use `--chain` and genesis builder `spec`
to generate the genesis state from the chain spec.
Additionally, we use metadata to perform some checks based on the
pallets the runtime exposes:
- If we see the `ParaInherent` pallet, we assume that we are dealing
with a relay chain. This means that we don't need proof recording during
import (since there is no storage weight).
- If we detect the `ParachainSystem` pallet, we assume that we are
dealing with a parachain and take corresponding actions like patching a
para id into the genesis state.
On the inherent side, I am currently supplying the standard inherents
every parachain needs.
In the current state, `frame-omni-bencher` supports all system chains.
In follow-up PRs, we could add additional inherents to increase
compatibility.
Since we are building a block during the benchmark, we also need to
build an extrinsic. By default, I am leveraging subxt to build the xt
dynamically. If a chain is not compatible with the `SubstrateConfig`
that comes with `subxt`, it can provide a custom extrinsic builder to
benchmarking-cli. This requires either a custom bencher implementation
or an integration into the parachains node.
Also cumulus-test-runtime has been migrated to provide genesis configs.
## Chain Compatibility
The current version here is compatible with the system chains and common
substrate chains. The way to go for others would be to customize the
frame-omni-bencher by providing a custom extrinsicbuilder. I did an
example implementation that works for mythical:
https://github.com/skunert/mythical-bencher
## Follow-Ups
- After #6040 is finished, we should integrate this here to make the
tooling truly useful. In the current form, the state is fairly small and
not representative.
## How to Review
I recommend starting from
[here](https://github.com/paritytech/polkadot-sdk/pull/5891/files#diff-50830ff756b3ac3403b7739d66c9e3a5185dbea550669ca71b28d19c7a2a54ecR264),
this method is the main entry point for omni-bencher and `polkadot`
binary.
TBD:
- [x] PRDoc
---------
Co-authored-by: Michal Kucharczyk <[email protected]>
Please register or sign in to comment