Skip to content
Snippets Groups Projects
  • Sebastian Kunert's avatar
    Add overhead benchmark to frame-omni-bencher (#5891) · 40547f9f
    Sebastian Kunert authored
    
    # Benchmark Overhead Command for Parachains
    
    This implements the `benchmark overhead` command for parachains. Full
    context is available at:
    https://github.com/paritytech/polkadot-sdk/issues/5303. Previous attempt
    was this https://github.com/paritytech/polkadot-sdk/pull/5283, but here
    we have integration into frame-omni-bencher and improved tooling.
    
    ## Changes Overview
    
    Users are now able to use `frame-omni-bencher` to generate
    `extrinsic_weight.rs` and `block_weight.rs` files for their runtime. The
    core logic for generating these remains untouched; this PR provides
    mostly machinery to make it work for parachains at all.
    
    Similar to the pallet benchmarks, we gain the option to benchmark based
    on just a runtime:
    
    ```
    frame-omni-bencher v1 benchmark overhead --runtime {{runtime}}
    ```
    
    or with a spec:
    
    ```
    frame-omni-bencher v1 benchmark overhead --chain {{spec}} --genesis-builder spec
    ```
    
    In this case, the genesis state is generated from the runtime presets.
    However, it is also possible to use `--chain` and genesis builder `spec`
    to generate the genesis state from the chain spec.
    
    Additionally, we use metadata to perform some checks based on the
    pallets the runtime exposes:
    
    - If we see the `ParaInherent` pallet, we assume that we are dealing
    with a relay chain. This means that we don't need proof recording during
    import (since there is no storage weight).
    - If we detect the `ParachainSystem` pallet, we assume that we are
    dealing with a parachain and take corresponding actions like patching a
    para id into the genesis state.
    
    On the inherent side, I am currently supplying the standard inherents
    every parachain needs.
    
    In the current state, `frame-omni-bencher` supports all system chains.
    In follow-up PRs, we could add additional inherents to increase
    compatibility.
    
    Since we are building a block during the benchmark, we also need to
    build an extrinsic. By default, I am leveraging subxt to build the xt
    dynamically. If a chain is not compatible with the `SubstrateConfig`
    that comes with `subxt`, it can provide a custom extrinsic builder to
    benchmarking-cli. This requires either a custom bencher implementation
    or an integration into the parachains node.
    
    Also cumulus-test-runtime has been migrated to provide genesis configs.
    
    ## Chain Compatibility
    The current version here is compatible with the system chains and common
    substrate chains. The way to go for others would be to customize the
    frame-omni-bencher by providing a custom extrinsicbuilder. I did an
    example implementation that works for mythical:
    https://github.com/skunert/mythical-bencher
    
    ## Follow-Ups
    - After #6040 is finished, we should integrate this here to make the
    tooling truly useful. In the current form, the state is fairly small and
    not representative.
    
    ## How to Review
    I recommend starting from
    [here](https://github.com/paritytech/polkadot-sdk/pull/5891/files#diff-50830ff756b3ac3403b7739d66c9e3a5185dbea550669ca71b28d19c7a2a54ecR264),
    this method is the main entry point for omni-bencher and `polkadot`
    binary.
    
    TBD:
    - [x] PRDoc
    
    ---------
    
    Co-authored-by: default avatarMichal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
    Unverified
    40547f9f
Code owners
Assign users and groups as approvers for specific file changes. Learn more.