Skip to content
Snippets Groups Projects
  1. Aug 05, 2024
  2. Jun 24, 2024
    • Oliver Tale-Yazdi's avatar
      Lift all dependencies (the big one) (#4716) · 8efa0544
      Oliver Tale-Yazdi authored
      
      After preparing in https://github.com/paritytech/polkadot-sdk/pull/4633,
      we can lift also all internal dependencies up to the workspace.
      
      This does not actually change anything, but uses `workspace = true` for
      all dependencies. You can check it with:
      ```bash
      git checkout -q $(git merge-base oty-lift-all-deps origin/master)
      cargo tree -e features > master.out
      
      git checkout -q oty-lift-all-deps
      cargo tree -e features > new.out
      diff master.out new.out
      ```
      
      It did not yet lift 100% of dependencies, some inside of `target.*` or
      some that had conflicting aliases introduced recently. But i will do
      these together in a follow-up with CI checks.
      
      Can be reproduced with [zepter](https://github.com/ggwpez/zepter/):
      `zepter transpose d lift-to-workspace "regex:.*" --version-resolver
      highest --skip-package "polkadot-sdk" --ignore-errors --fix`.
      
      ---------
      
      Signed-off-by: default avatarOliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
  3. Apr 08, 2024
    • Aaro Altonen's avatar
      Integrate litep2p into Polkadot SDK (#2944) · 80616f6d
      Aaro Altonen authored
      
      [litep2p](https://github.com/altonen/litep2p) is a libp2p-compatible P2P
      networking library. It supports all of the features of `rust-libp2p`
      that are currently being utilized by Polkadot SDK.
      
      Compared to `rust-libp2p`, `litep2p` has a quite different architecture
      which is why the new `litep2p` network backend is only able to use a
      little of the existing code in `sc-network`. The design has been mainly
      influenced by how we'd wish to structure our networking-related code in
      Polkadot SDK: independent higher-levels protocols directly communicating
      with the network over links that support bidirectional backpressure. A
      good example would be `NotificationHandle`/`RequestResponseHandle`
      abstractions which allow, e.g., `SyncingEngine` to directly communicate
      with peers to announce/request blocks.
      
      I've tried running `polkadot --network-backend litep2p` with a few
      different peer configurations and there is a noticeable reduction in
      networking CPU usage. For high load (`--out-peers 200`), networking CPU
      usage goes down from ~110% to ~30% (80 pp) and for normal load
      (`--out-peers 40`), the usage goes down from ~55% to ~18% (37 pp).
      
      These should not be taken as final numbers because:
      
      a) there are still some low-hanging optimization fruits, such as
      enabling [receive window
      auto-tuning](https://github.com/libp2p/rust-yamux/pull/176), integrating
      `Peerset` more closely with `litep2p` or improving memory usage of the
      WebSocket transport
      b) fixing bugs/instabilities that incorrectly cause `litep2p` to do less
      work will increase the networking CPU usage
      c) verification in a more diverse set of tests/conditions is needed
      
      Nevertheless, these numbers should give an early estimate for CPU usage
      of the new networking backend.
      
      This PR consists of three separate changes:
      * introduce a generic `PeerId` (wrapper around `Multihash`) so that we
      don't have use `NetworkService::PeerId` in every part of the code that
      uses a `PeerId`
      * introduce `NetworkBackend` trait, implement it for the libp2p network
      stack and make Polkadot SDK generic over `NetworkBackend`
        * implement `NetworkBackend` for litep2p
      
      The new library should be considered experimental which is why
      `rust-libp2p` will remain as the default option for the time being. This
      PR currently depends on the master branch of `litep2p` but I'll cut a
      new release for the library once all review comments have been
      addresses.
      
      ---------
      
      Signed-off-by: default avatarAlexandru Vasile <alexandru.vasile@parity.io>
      Co-authored-by: default avatarDmitry Markin <dmitry@markin.tech>
      Co-authored-by: default avatarAlexandru Vasile <60601340+lexnv@users.noreply.github.com>
      Co-authored-by: default avatarAlexandru Vasile <alexandru.vasile@parity.io>
  4. Mar 05, 2024
    • Kian Paimani's avatar
      Repot all templates into a single directory (#3460) · 4c810609
      Kian Paimani authored
      The first step towards
      https://github.com/paritytech/polkadot-sdk/issues/3155
      
      Brings all templates under the following structure
      
      ```
      templates
      |   parachain
      |   |   polkadot-launch
      |   |   runtime              --> parachain-template-runtime
      |   |   pallets              --> pallet-parachain-template
      |   |   node                 --> parachain-template-node
      |   minimal
      |   |   runtime              --> minimal-template-runtime
      |   |   pallets              --> pallet-minimal-template
      |   |   node                 --> minimal-template-node
      |   solochain
      |   |   runtime              --> solochain-template-runtime
      |   |   pallets              --> pallet-template (the naming is not consistent here)
      |   |   node                 --> solochain-template-node
      ```
      
      The only note-worthy changes in this PR are: 
      
      - More `Cargo.toml` fields are forwarded to use the one from the
      workspace.
      - parachain template now has weights and benchmarks
      - adds a shell pallet to the minimal template
      - remove a few unused deps 
      
      
      A list of possible follow-ups: 
      
      - [ ] Unify READMEs, create a parent README for all
      - [ ] remove references to `docs.substrate.io` in templates
      - [ ] make all templates use `#[derive_impl]`
      - [ ] update and unify all licenses
      - [ ] Remove polkadot launch, use
      https://github.com/paritytech/polkadot-sdk/blob/35349df9/cumulus/zombienet/examples/small_network.toml
      instead.
  5. Oct 27, 2023
    • juangirini's avatar
      feat: FRAME umbrella crate. (#1337) · 43415ef5
      juangirini authored
      
      ### Original PR https://github.com/paritytech/substrate/pull/14137
      
      This PR brings in the first version of the "_`frame` umbrella crate_".
      This crate is intended to serve two purposes:
      
      1. documentation
      2. easier development with frame. Ideally, we want most users to be able
      to build a frame-based pallet and runtime using just `frame` (plus
      `scale-codec` and `scale-info`).
      
      The crate is not finalized and is not yet intended for external use.
      Therefore, the version is set to `0.0.1-dev`, this PR is `silent`, and
      the entire crate is hidden behind the `experimental` flag. The main
      intention in merging it early on is to be able to iterate on it in the
      rest of
      [`developer-hub`](https://github.com/paritytech/polkadot-sdk-docs/)
      efforts.
      
      The public API of the `frame` crate is at the moment as follows: 
      
      ```
      pub mod frame
      pub use frame::log
      pub use frame::pallet
      pub mod frame::arithmetic
      pub use frame::arithmetic::<<sp_arithmetic::*>>
      pub use frame::arithmetic::<<sp_arithmetic::traits::*>>
      pub mod frame::deps
      pub use frame::deps::codec
      pub use frame::deps::frame_executive
      pub use frame::deps::frame_support
      pub use frame::deps::frame_system
      pub use frame::deps::scale_info
      pub use frame::deps::sp_api
      pub use frame::deps::sp_arithmetic
      pub use frame::deps::sp_block_builder
      pub use frame::deps::sp_consensus_aura
      pub use frame::deps::sp_consensus_grandpa
      pub use frame::deps::sp_core
      pub use frame::deps::sp_inherents
      pub use frame::deps::sp_io
      pub use frame::deps::sp_offchain
      pub use frame::deps::sp_runtime
      pub use frame::deps::sp_std
      pub use frame::deps::sp_version
      pub mod frame::derive
      pub use frame::derive::CloneNoBound
      pub use frame::derive::Debug
      pub use frame::derive::Debug
      pub use frame::derive::DebugNoBound
      pub use frame::derive::Decode
      pub use frame::derive::Decode
      pub use frame::derive::DefaultNoBound
      pub use frame::derive::Encode
      pub use frame::derive::Encode
      pub use frame::derive::EqNoBound
      pub use frame::derive::PartialEqNoBound
      pub use frame::derive::RuntimeDebug
      pub use frame::derive::RuntimeDebugNoBound
      pub use frame::derive::TypeInfo
      pub use frame::derive::TypeInfo
      pub mod frame::prelude
      pub use frame::prelude::<<frame_support::pallet_prelude::*>>
      pub use frame::prelude::<<frame_system::pallet_prelude::*>>
      pub use frame::prelude::<<sp_std::prelude::*>>
      pub use frame::prelude::CloneNoBound
      pub use frame::prelude::Debug
      pub use frame::prelude::Debug
      pub use frame::prelude::DebugNoBound
      pub use frame::prelude::Decode
      pub use frame::prelude::Decode
      pub use frame::prelude::DefaultNoBound
      pub use frame::prelude::Encode
      pub use frame::prelude::Encode
      pub use frame::prelude::EqNoBound
      pub use frame::prelude::PartialEqNoBound
      pub use frame::prelude::RuntimeDebug
      pub use frame::prelude::RuntimeDebugNoBound
      pub use frame::prelude::TypeInfo
      pub use frame::prelude::TypeInfo
      pub use frame::prelude::frame_system
      pub mod frame::primitives
      pub use frame::primitives::BlakeTwo256
      pub use frame::primitives::H160
      pub use frame::primitives::H256
      pub use frame::primitives::H512
      pub use frame::primitives::Hash
      pub use frame::primitives::Keccak256
      pub use frame::primitives::U256
      pub use frame::primitives::U512
      pub mod frame::runtime
      pub mod frame::runtime::apis
      pub use frame::runtime::apis::<<frame_system_rpc_runtime_api::*>>
      pub use frame::runtime::apis::<<sp_api::*>>
      pub use frame::runtime::apis::<<sp_block_builder::*>>
      pub use frame::runtime::apis::<<sp_consensus_aura::*>>
      pub use frame::runtime::apis::<<sp_consensus_grandpa::*>>
      pub use frame::runtime::apis::<<sp_offchain::*>>
      pub use frame::runtime::apis::<<sp_session::runtime_api::*>>
      pub use frame::runtime::apis::<<sp_transaction_pool::runtime_api::*>>
      pub use frame::runtime::apis::ApplyExtrinsicResult
      pub use frame::runtime::apis::CheckInherentsResult
      pub use frame::runtime::apis::InherentData
      pub use frame::runtime::apis::OpaqueMetadata
      pub use frame::runtime::apis::impl_runtime_apis
      pub use frame::runtime::apis::sp_api
      pub mod frame::runtime::prelude
      pub use frame::runtime::prelude::<<frame_executive::*>>
      pub use frame::runtime::prelude::ConstBool
      pub use frame::runtime::prelude::ConstI128
      pub use frame::runtime::prelude::ConstI16
      pub use frame::runtime::prelude::ConstI32
      pub use frame::runtime::prelude::ConstI64
      pub use frame::runtime::prelude::ConstI8
      pub use frame::runtime::prelude::ConstU128
      pub use frame::runtime::prelude::ConstU16
      pub use frame::runtime::prelude::ConstU32
      pub use frame::runtime::prelude::ConstU64
      pub use frame::runtime::prelude::ConstU8
      pub use frame::runtime::prelude::NativeVersion
      pub use frame::runtime::prelude::RuntimeVersion
      pub use frame::runtime::prelude::construct_runtime
      pub use frame::runtime::prelude::create_runtime_str
      pub use frame::runtime::prelude::derive_impl
      pub use frame::runtime::prelude::frame_support
      pub use frame::runtime::prelude::ord_parameter_types
      pub use frame::runtime::prelude::parameter_types
      pub use frame::runtime::prelude::runtime_version
      pub mod frame::runtime::testing_prelude
      pub use frame::runtime::testing_prelude::BuildStorage
      pub use frame::runtime::testing_prelude::Storage
      pub mod frame::runtime::types_common
      pub type frame::runtime::types_common::AccountId = <<frame::runtime::types_common::Signature as sp_runtime::traits::Verify>::Signer as sp_runtime::traits::IdentifyAccount>::AccountId
      pub type frame::runtime::types_common::BlockNumber = u32
      pub type frame::runtime::types_common::BlockOf<T, Extra> = sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<frame::runtime::types_common::BlockNumber, sp_runtime::traits::BlakeTwo256>, sp_runtime::generic::unchecked_extrinsic::UncheckedExtrinsic<sp_runtime::multiaddress::MultiAddress<frame::runtime::types_common::AccountId, ()>, <T as frame_system::pallet::Config>::RuntimeCall, frame::runtime::types_common::Signature, Extra>>
      pub type frame::runtime::types_common::OpaqueBlock = sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<frame::runtime::types_common::BlockNumber, sp_runtime::traits::BlakeTwo256>, sp_runtime::OpaqueExtrinsic>
      pub type frame::runtime::types_common::Signature = sp_runtime::MultiSignature
      pub type frame::runtime::types_common::SystemSignedExtensionsOf<T> = (frame_system::extensions::check_non_zero_sender::CheckNonZeroSender<T>, frame_system::extensions::check_spec_version::CheckSpecVersion<T>, frame_system::extensions::check_tx_version::CheckTxVersion<T>, frame_system::extensions::check_genesis::CheckGenesis<T>, frame_system::extensions::check_mortality::CheckMortality<T>, frame_system::extensions::check_nonce::CheckNonce<T>, frame_system::extensions::check_weight::CheckWeight<T>)
      pub mod frame::testing_prelude
      pub use frame::testing_prelude::<<frame_executive::*>>
      pub use frame::testing_prelude::<<frame_system::mocking::*>>
      pub use frame::testing_prelude::BuildStorage
      pub use frame::testing_prelude::ConstBool
      pub use frame::testing_prelude::ConstI128
      pub use frame::testing_prelude::ConstI16
      pub use frame::testing_prelude::ConstI32
      pub use frame::testing_prelude::ConstI64
      pub use frame::testing_prelude::ConstI8
      pub use frame::testing_prelude::ConstU128
      pub use frame::testing_prelude::ConstU16
      pub use frame::testing_prelude::ConstU32
      pub use frame::testing_prelude::ConstU64
      pub use frame::testing_prelude::ConstU8
      pub use frame::testing_prelude::NativeVersion
      pub use frame::testing_prelude::RuntimeVersion
      pub use frame::testing_prelude::Storage
      pub use frame::testing_prelude::TestState
      pub use frame::testing_prelude::assert_err
      pub use frame::testing_prelude::assert_err_ignore_postinfo
      pub use frame::testing_prelude::assert_error_encoded_size
      pub use frame::testing_prelude::assert_noop
      pub use frame::testing_prelude::assert_ok
      pub use frame::testing_prelude::assert_storage_noop
      pub use frame::testing_prelude::construct_runtime
      pub use frame::testing_prelude::create_runtime_str
      pub use frame::testing_prelude::derive_impl
      pub use frame::testing_prelude::frame_support
      pub use frame::testing_prelude::frame_system
      pub use frame::testing_prelude::if_std
      pub use frame::testing_prelude::ord_parameter_types
      pub use frame::testing_prelude::parameter_types
      pub use frame::testing_prelude::runtime_version
      pub use frame::testing_prelude::storage_alias
      pub mod frame::traits
      pub use frame::traits::<<frame_support::traits::*>>
      pub use frame::traits::<<sp_runtime::traits::*>>
      ```
      
      ---
      
      The road to full stabilization is
      
      - [ ] https://github.com/paritytech/polkadot-sdk/issues/127
      - [ ] have a more intentional version bump, as opposed to the current bi
      weekly force-major-bump
      - [ ] revise the internal API of `frame`, especially what goes into the
      `prelude`s.
      - [ ] migrate all internal pallets and runtime to use `frame`
      
      ---------
      
      Co-authored-by: default avatarkianenigma <kian@parity.io>
      Co-authored-by: default avatarKian Paimani <5588131+kianenigma@users.noreply.github.com>
      Co-authored-by: default avatarOliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
      Co-authored-by: default avatarFrancisco Aguirre <franciscoaguirreperez@gmail.com>