1. Apr 09, 2024
  2. Apr 08, 2024
    • Léa Narzis's avatar
    • Aaro Altonen's avatar
      Integrate litep2p into Polkadot SDK (#2944) · 80616f6d
      Aaro Altonen authored
      [litep2p](https://github.com/altonen/litep2p) is a libp2p-compatible P2P
      networking library. It supports all of the features of `rust-libp2p`
      that are currently being utilized by Polkadot SDK.
      
      Compared to `rust-libp2p`, `litep2p` has a quite different architecture
      which is why the new `litep2p` network backend is only able to use a
      little of the existing code in `sc-network`. The design has been mainly
      influenced by how we'd wish to structure our networking-related code in
      Polkadot SDK: independent higher-levels protocols directly communicating
      with the network over links that support bidirectional backpressure. A
      good example would be `NotificationHandle`/`RequestResponseHandle`
      abstractions which allow, e.g., `SyncingEngine` to directly communicate
      with peers to announce/request blocks.
      
      I've tried running `polkadot --network-backend litep2p` with a few
      different peer configurations and there is a noticeable reduction in
      networking CPU usage. For high load (`--out-peers 200`), networking CPU
      usage goes down from ~110% to ~30% (80 pp) and for normal load
      (`--out-peers 40`), the usage goes down from ~55% to ~18% (37 pp).
      
      These should not be taken as final numbers because:
      
      a) there are still some low-hanging optimization fruits, such as
      enabling [receive window
      auto-tuning](https://github.com/libp2p/rust-yamux/pull/176
      
      ), integrating
      `Peerset` more closely with `litep2p` or improving memory usage of the
      WebSocket transport
      b) fixing bugs/instabilities that incorrectly cause `litep2p` to do less
      work will increase the networking CPU usage
      c) verification in a more diverse set of tests/conditions is needed
      
      Nevertheless, these numbers should give an early estimate for CPU usage
      of the new networking backend.
      
      This PR consists of three separate changes:
      * introduce a generic `PeerId` (wrapper around `Multihash`) so that we
      don't have use `NetworkService::PeerId` in every part of the code that
      uses a `PeerId`
      * introduce `NetworkBackend` trait, implement it for the libp2p network
      stack and make Polkadot SDK generic over `NetworkBackend`
        * implement `NetworkBackend` for litep2p
      
      The new library should be considered experimental which is why
      `rust-libp2p` will remain as the default option for the time being. This
      PR currently depends on the master branch of `litep2p` but I'll cut a
      new release for the library once all review comments have been
      addresses.
      
      ---------
      
      Signed-off-by: default avatarAlexandru Vasile <[email protected]>
      Co-authored-by: default avatarDmitry Markin <[email protected]>
      Co-authored-by: default avatarAlexandru Vasile <[email protected]>
      Co-authored-by: default avatarAlexandru Vasile <[email protected]>
      80616f6d
    • Oliver Tale-Yazdi's avatar
      [FRAME] Runtime Omni Bencher (#3512) · 9543d314
      Oliver Tale-Yazdi authored
      This MR contains two major changes and some maintenance cleanup.  
      
      ## 1. Free Standing Pallet Benchmark Runner
      
      Closes https://github.com/paritytech/polkadot-sdk/issues/3045, depends
      on your runtime exposing the `GenesisBuilderApi` (like
      https://github.com/paritytech/polkadot-sdk/pull/1492).
      
      Introduces a new binary crate: `frame-omni-bencher`.  
      It allows to directly benchmark a WASM blob - without needing a node or
      chain spec.
      
      This makes it much easier to generate pallet weights and should allow us
      to remove bloaty code from the node.
      It should work for all FRAME runtimes that dont use 3rd party host calls
      or non `BlakeTwo256` block hashing (basically all polkadot parachains
      should work).
      
      It is 100% backwards compatible with the old CLI args, when the `v1`
      compatibility command is used. This is done to allow for forwards
      compatible addition of new commands.
      
      ### Example (full example in the Rust docs)
      
      Installing the CLI:
      ```sh
      cargo install --locked --path substrate/utils/frame/omni-bencher
      frame-omni-bencher --help
      ```
      
      Building the Westend runtime:
      ```sh
      cargo build -p westend-runtime --release --features runtime-benchmarks
      ```
      
      Benchmarking the runtime:
      ```sh
      frame-omni-bencher v1 benchmark pallet --runtime target/release/wbuild/westend-runtime/westend_runtime.compact.compressed.wasm --all
      ```
      
      ## 2. Building the Benchmark Genesis State in the Runtime
      
      Closes https://github.com/paritytech/polkadot-sdk/issues/2664
      
      This adds `--runtime` and `--genesis-builder=none|runtime|spec`
      arguments to the `benchmark pallet` command to make it possible to
      generate the genesis storage by the runtime. This can be used with both
      the node and the freestanding benchmark runners. It utilizes the new
      `GenesisBuilder` RA and depends on having
      https://github.com/paritytech/polkadot-sdk/pull/3412
      
       deployed.
      
      ## 3. Simpler args for `PalletCmd::run`
      
      You can do three things here to integrate the changes into your node:
      - nothing: old code keeps working as before but emits a deprecated
      warning
      - delete: remove the pallet benchmarking code from your node and use the
      omni-bencher instead
      - patch: apply the patch below and keep using as currently. This emits a
      deprecated warning at runtime, since it uses the old way to generate a
      genesis state, but is the smallest change.
      
      ```patch
      runner.sync_run(|config| cmd
      -    .run::<HashingFor<Block>, ReclaimHostFunctions>(config)
      +    .run_with_spec::<HashingFor<Block>, ReclaimHostFunctions>(Some(config.chain_spec))
      )
      ```
      
      ## 4. Maintenance Change
      - `pallet-nis` get a `BenchmarkSetup` config item to prepare its
      counterparty asset.
      - Add percent progress print when running benchmarks.
      - Dont immediately exit on benchmark error but try to run as many as
      possible and print errors last.
      
      ---------
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: default avatarLiam Aharon <[email protected]>
      9543d314
    • Sebastian Kunert's avatar
      Add best block indicator to informant message + print parent block on import message (#4021) · fdb1dba2
      Sebastian Kunert authored
      Sometimes you need to debug some issues just by the logs and reconstruct
      what happened.
      In these scenarios it would be nice to know if a block was imported as
      best block, and what it parent was.
      So here I propose to change the output of the informant to this:
      
      ```
      2024-04-05 20:38:22.004  INFO ⋮substrate: [Parachain]  Imported #18 (0xe7b3…4555 -> 0xbd6f…ced7)    
      2024-04-05 20:38:24.005  INFO ⋮substrate: [Parachain]  Imported #19 (0xbd6f…ced7 -> 0x4dd0…d81f)    
      2024-04-05 20:38:24.011  INFO ⋮substrate: [jobless-children-5352] 🌟 Imported #42 (0xed2e…27fc -> 0x718f…f30e)    
      2024-04-05 20:38:26.005  INFO ⋮substrate: [Parachain]  Imported #20 (0x4dd0…d81f -> 0x6e85…e2b8)    
      2024-04-05 20:38:28.004  INFO ⋮substrate: [Parachain] 🌟 Imported #21 (0x6e85…e2b8 -> 0xad53…2a97)    
      2024-04-05 20:38:30.004  INFO ⋮substrate: [Parachain] 🌟
      
       Imported #22 (0xad53…2a97 -> 0xa874…890f)    
      ```
      
      ---------
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      fdb1dba2
    • Bastian Köcher's avatar
      sc-beefy-consensus: Remove unneeded stream. (#4015) · c1063a53
      Bastian Köcher authored
      The stream was just used to communicate from the validator the peer
      reports back to the gossip engine. Internally the gossip engine just
      forwards these reports to the networking engine. So, we can just do this
      directly.
      
      The reporting stream was also pumped [in the worker behind the
      engine](https://github.com/paritytech/polkadot-sdk/blob/9d6261892814fa27c97881c0321c008d7340b54b/substrate/client/consensus/beefy/src/worker.rs#L939).
      This means if there was a lot of data incoming over the engine, the
      reporting stream was almost never processed and thus, it could have
      started to grow and we have seen issues around this.
      
      Partly Closes: https://github.com/paritytech/polkadot-sdk/issues/3945
      c1063a53
    • HongKuang's avatar
      Fix some typos (#4018) · bd4471b4
      HongKuang authored
      
      
      Signed-off-by: default avatarhongkuang <[email protected]>
      bd4471b4
  3. Apr 06, 2024
  4. Apr 05, 2024
  5. Apr 04, 2024
  6. Apr 03, 2024
  7. Apr 02, 2024
  8. Apr 01, 2024
  9. Mar 31, 2024
  10. Mar 29, 2024
  11. Mar 28, 2024