1. Aug 02, 2023
    • drskalman's avatar
      Support for multiple signature scheme for BEEFY primitves (#14373) · 3fef703e
      drskalman authored
      
      
      * Merged BEEFY primitives with generic signature and keyset commitment support from old pull to current code
      
      * - Add bls-experimental feature to application-crypto and beefy primitives
      - Fix remaining crypto -> ecdsa_crypto
      - code build but not tests
      
      * Make beefy primitive tests compile
      
      * move bls related beefy primitives code and test behind bls-experimental flag
      
      * Make BEEFY clients complies with BEEFY API depending on AuthorityId
      
      * - Rename `BeefyAuthoritySet.root` → `BeefyAuthoritySet.keyset_commitment`.
      - Remove apk proof keyset_commitment from `BeefyAuthoritySet`.
      - Fix failing signed commitment and signature to witness test.
      - Make client compatible with BeefyAPI generic on AuthorityId.
      - `crypto` → `ecdsa_crypto` in BEEFY client and frame.
      
      * Commit Cargo lock remove ark-serialize from BEEFY primitives
      
      * Use Codec instead of Encode + Decode in primitives/consensus/beefy/src/lib.rs
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * - Make `BeefyApi` generic over Signature type.
      - Make new `BeeyApi` functinos also generic over AuthorityId and Signature
      
      * Unmake BeefyAPI generic over Signature. Recover Signature type from AuthId.
      
      * - dont use hex or hex-literal use array-bytes instead in beefy primitives and bls crypto.
      - CamelCase ECDSA and BLS everywhere.
      
      * Move the definition of BEEFY key type from `primitives/beefy` to `crypto.rs` according to new convention.
      
      * - Add bls377_generate_new to `sp-io` and `application_crypto::bls`.
      - Add `bls-experimental` to `sp-io`
      
      Does not compile because PassByCodec can not derive PassBy using customly implemented PassByIner.
      
      * Implement PassBy for `bls::Public` manually
      
      * fix Beefy `KEY_TYPE` in `frame/beefy` tests to come from `sp-core::key_types` enum
      
      * specify both generic for `hex2array_unchecked` in `sp-core/bls.rs`
      
      * Rename `crypto`→`ecdsa_crypto` in `primitives/consensus/beefy/src/test_utils.rs` docs
      
      * remove commented-out code in `primitives/consensus/beefy/src/commitment.rs`
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * Fix inconsistency in panic message in  `primitives/io/src/lib.rs`
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * Remove redundant feature activation in `primitives/io/Cargo.toml`
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * - make `w3f-bls` a dev-dependancy only for beefy primitives.
      
      - clean up comments.
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * export BEEFY KEY_TYPE from primitives/consensus/beefy
      make `frame/consensus/beefy` in dependent of sp_crypto_app
      use consistent naming in the beefy primitive tests.
      
      * - implement `BeefyAuthorityId` for `bls_crypto::AuthorityId`.
      - implement `bls_verify_works` test for BEEFY `bls_crypto`.
      
      * Remove BEEFY `ecdsa_n_bls_crypto` for now for later re-introduction
      
      * Make commitment and witness BEEFY tests not use Keystore.
      
      * put `bls_beefy_verify_works` test under `bls-experimental` flag.
      
      * bump up Runtime `BeefyAPI` to version 3 due to introducing generic AuthorityId.
      
      * reuse code and encapsulate w3f-bls backend in sp-core as most as possible
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * Make comments in primities BEEFY `commitment.rs` and `witness.rs``tests convention conforming
      
      * Use master dep versions
      
      * Trivial change. Mostly to trigger CI
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarAndré Silva <[email protected]>
      
      * Fix Cargo.toml
      
      * Trigger CI with cumulus companion
      
      * Trigger CI after polkadot companion change
      
      ---------
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      Co-authored-by: default avatarAndré Silva <[email protected]>
      3fef703e
    • Dmitry Markin's avatar
      Get rid of `Peerset` compatibility layer (#14337) · 8dc3bd72
      Dmitry Markin authored
      
      
      * Move bootnodes from individual `SetConfig`s to `PeersetConfig`
      
      * Move `SetId` & `SetConfig` from `peerset` to `protocol_controller`
      
      * Remove unused `DropReason`
      
      * Move `Message` & `IncomingIndex` from `peerset` to `protocol_controller`
      
      * Restore running fuzz test
      
      * Get rid of `Peerset` in `fuzz` test
      
      * Spawn runners instead of manual polling in `fuzz` test
      
      * Migrate `Protocol` from `Peerset` to `PeerStore` & `ProtocolController`
      
      * Migrate `NetworkService` from `Peerset` to `PeerStore` & `ProtocolController`
      
      * Migrate `Notifications` from `Peerset` to `ProtocolController`s
      
      * Migrate `Notifications` tests from `Peerset` to `ProtocolController`
      
      * Fix compilation of `NetworkService` & `Protocol`
      
      * Fix borrowing issues in `Notifications`
      
      * Migrate `RequestResponse`from `Peerset` to `PeerStore`
      
      * rustfmt
      
      * Migrate request-response tests from `Peerset` to `PeerStore`
      
      * Migrate `reconnect_after_disconnect` test to `PeerStore` & `ProtocolController`
      
      * Fix `Notifications` tests
      
      * Remove `Peerset` completely
      
      * Fix bug with counting sync peers in `Protocol`
      
      * Eliminate indirect calls to `PeerStore` via `Protocol`
      
      * Eliminate indirect calls to `ProtocolController` via `Protocol`
      
      * Handle `Err` outcome from `remove_peers_from_reserved_set`
      
      * Add note about disconnecting sync peers in `Protocol`
      
      * minor: remove unneeded `clone()`
      
      * minor: extra comma removed
      
      * minor: use `Stream` API of `from_protocol_controllers` channel
      
      * minor: remove TODO
      
      * minor: replace `.map().flatten()` with `.flat_map()`
      
      * minor: update `ProtocolController` docs
      
      * rustfmt
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarAaro Altonen <[email protected]>
      
      * Extract `MockPeerStore` to `mock.rs`
      
      * Move `PeerStore` initialization to `build_network`
      
      * minor: remove unused import
      
      * minor: clarify error message
      
      * Convert `syncs_header_only_forks` test into single-threaded
      
      ---------
      
      Co-authored-by: default avatarAaro Altonen <[email protected]>
      8dc3bd72
  2. Aug 01, 2023
  3. Jul 31, 2023
  4. Jul 27, 2023
  5. Jul 25, 2023
    • Marijn Schouten's avatar
      change HashFor to HashingFor (#14483) · c9b54e10
      Marijn Schouten authored
      * change HashFor to HashingFor
      
      * fmt
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      ---------
      
      Co-authored-by: command-bot <>
      c9b54e10
    • Alexandru Vasile's avatar
      chainHead_storage: Iterate over keys (#14628) · 00787a10
      Alexandru Vasile authored
      
      
      * chainHead: Iterate over key,values and key,hashes
      
      Signed-off-by: default avatarAlexandru Vasile <[email protected]>
      
      * chainHead/tests: Multi query with iteration over keys
      
      Signed-off-by: default avatarAlexandru Vasile <[email protected]>
      
      * chainHead/events: Fix typo in StorageQuery
      
      Signed-off-by: default avatarAlexandru Vasile <[email protected]>
      
      * chainHead: Take 10 from key iterator
      
      Signed-off-by: default avatarAlexandru Vasile <[email protected]>
      
      ---------
      
      Signed-off-by: default avatarAlexandru Vasile <[email protected]>
      Co-authored-by: parity-processbot <>
      00787a10
    • Anton's avatar
      chore: update libp2p to 0.52.1 (#14429) · 59d8b864
      Anton authored
      * update libp2p to 0.52.0
      
      * proto name now must implement `AsRef<str>`
      
      * update libp2p version everywhere
      
      * ToSwarm, FromBehaviour, ToBehaviour
      
      also LocalProtocolsChange and RemoteProtocolsChange
      
      * new NetworkBehaviour invariants
      
      * replace `Vec<u8>` with `StreamProtocol`
      
      * rename ConnectionHandlerEvent::Custom to NotifyBehaviour
      
      * remove DialError & ListenError invariants
      
      also fix pending_events
      
      * use connection_limits::Behaviour
      
      See https://github.com/libp2p/rust-libp2p/pull/3885
      
      * impl `void::Void` for `BehaviourOut`
      
      also use `Behaviour::with_codec`
      
      * KademliaHandler no longer public
      
      * fix StreamProtocol construction
      
      * update libp2p-identify to 0.2.0
      
      * remove non-existing methods from PollParameters
      
      rename ConnectionHandlerUpgrErr to StreamUpgradeError
      
      * `P2p` now contains `PeerId`, not `Multihash`
      
      * use multihash-codetable crate
      
      * update Cargo.lock
      
      * reformat text
      
      * comment out tests for now
      
      * remove `.into()` from P2p
      
      * confirm observed addr manually
      
      See https://github.com/libp2p/rust-libp2p/blob/master/protocols/identify/CHANGELOG.md#0430
      
      * remove SwarmEvent::Banned
      
      since we're not using `ban_peer_id`, this can be safely removed.
      we may want to introduce `libp2p::allow_block_list` module in the future.
      
      * fix imports
      
      * replace `libp2p` with smaller deps in network-gossip
      
      * bring back tests
      
      * finish rewriting tests
      
      * uncomment handler tests
      
      * Revert "uncomment handler tests"
      
      This reverts commit 720a06815887f4e10767c62b58864a7ec3a48e50.
      
      * add a fixme
      
      * update Cargo.lock
      
      * remove extra From
      
      * make void uninhabited
      
      * fix discovery test
      
      * use autonat protocols
      
      confirming external addresses manually is unsafe in open networks
      
      * fix SyncNotificationsClogged invariant
      
      * only set server mode manually in tests
      
      doubt that we need to set it on node since we're adding public addresses
      
      * address @dmitry-markin comments
      
      * remove autonat
      
      * removed unused var
      
      * fix EOL
      
      * update smallvec and sha2
      
      in attempt to compile polkadot
      
      * bump k256
      
      in attempt to build cumulus
      
      ---------
      
      Co-authored-by: parity-processbot <>
      59d8b864
  6. Jul 24, 2023
    • Michal Kucharczyk's avatar
      `test-runtime`: `GenesisBuilder` runtime API impl + tests (#14310) · 53cbda1d
      Michal Kucharczyk authored
      
      
      * test-runtime: GenesisBuilder runtime API impl + tests
      
      This PR provides implementation of `GenesisBuilder` API for `substrate-test-runtime`, can be considered as reference imiplementation for other runtimes.
      The `GenesisBuilder` implementation is gated by `gensis-config` feature.
      
      Tested scenarios:
      - default `GenesisConfig` to JSON blob,
      - deserialization of `GenesisConfig` from custom JSON, and storing its keys into the Storage (genesis storage creation).
      - creation of genesis storage using partial JSON definition,
      - checking if invalid/renamed JSON files causes the runtime to panic,
      
      * missing file added
      
      * client: GenesisBuilder helper added
      
      * feature renamed: genesis-config -> genesis-builder
      
      * Update Cargo.toml
      
      * Update Cargo.toml
      
      * Update Cargo.toml
      
      * Update Cargo.toml
      
      * redundant function removed
      
      * genesis builder helper: introduced RuntimeGenesisBuild
      
      * test-runtime: get rid of unused T
      
      * redundant bound removed
      
      * helper: use GenesisBuild again
      
      * tests adjusted for on_genesis
      
      * test-runtime: support for BuildGenesisConfig
      
      * helper: BuildGenesisConfig support
      
      * Update client/genesis-builder/src/lib.rs
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * Update test-utils/runtime/src/test_json/readme.md
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * Update test-utils/runtime/src/test_json/readme.md
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * Update test-utils/runtime/src/genesismap.rs
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      
      * jsons are now human-friendly
      
      * fix
      
      * improvements
      
      * jsons fixed
      
      * helper: no_defaults added
      
      * test-runtime: no_defaults added
      
      * test-runtime: patching fn removed
      
      * helper: patching fn removed
      
      * helper: moved to frame_support
      
      * test-runtime: fixes
      
      * Cargo.lock updated
      
      * fmt + naming
      
      * test-runtime: WasmExecutor used
      
      * helper / test-runtime: struct removed
      
      * test-runtime: merge fixes
      
      * Cargo.lock + test-utils/runtime/Cargo.toml updated
      
      * doc fixed
      
      * client/rpc: test fixed (new rt api)
      
      * client/rpc-spec-v2: test fix
      
      * doc fix
      
      * test-runtime: disable-genesis-builder feature
      
      * fix
      
      * fix
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      * test-runtime: rerun added to build script
      
      ---------
      
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      Co-authored-by: parity-processbot <>
      53cbda1d
    • Aaro Altonen's avatar
      Accept only `--in-peers` many inbound full nodes in `SyncingEngine` (#14603) · f008e069
      Aaro Altonen authored
      * Accept only `--in-peers` many inbound full nodes in `SyncingEngine`
      
      Due to full and light nodes being stored in the same set, it's possible
      that `SyncingEngine` accepts more than `--in-peers` many inbound full
      nodes which leaves some of its outbound slots unoccupied.
      
      `ProtocolController` still tries to occupy these slots by opening
      outbound substreams. As these substreams are accepted by the remote peer,
      the connection is relayed to `SyncingEngine` which rejects the node
      because it's already full. This in turn results in the substream being
      inactive and the peer getting evicted.
      
      Fixing this properly would require relocating the light peer slot
      allocation away from `ProtocolController` or alternatively moving entire
      the substream validation there, both of which are epic refactorings and
      not necessarily in line with other goals. As a temporary measure, verify
      in `SyncingEngine` that it doesn't accept more than the specified amount
      of inbound full peers.
      
      * Fix tests
      
      * Apply review comments
      f008e069
  7. Jul 23, 2023
  8. Jul 21, 2023
  9. Jul 20, 2023
  10. Jul 19, 2023
    • Kian Paimani's avatar
      add doc-only `substrate` entry point crate (#14581) · 817c97d6
      Kian Paimani authored
      
      
      * add doc-only substrate entry point crate
      
      * document a few more things
      
      * add more
      
      * fix width
      
      * Update primitives/io/src/lib.rs
      
      Co-authored-by: default avatarGonçalo Pestana <[email protected]>
      
      * add link
      
      * update cargo toml file
      
      * fix sp-io docs
      
      * improve
      
      * small update
      
      * add license
      
      * satisfy license job
      
      * add a line about FRAME
      
      * CI happy now
      
      * make CI more happy
      
      * Let the check run for the whole workspace
      
      * Forward the substrate node again as default run
      
      * update binary names
      
      * upate verison test
      
      * Fix fix fix
      
      * Fix
      
      * rename to substrate-node in more places
      
      * Revert "rename to substrate-node in more places"
      
      This reverts commit 66960f84a1b6f1f7c638b4040e28e9fbabb8adf5.
      
      * fix
      
      * Fix build pipeline
      
      * Fix properly plus add some docs
      
      ---------
      
      Co-authored-by: default avatarGonçalo Pestana <[email protected]>
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      817c97d6
  11. Jul 18, 2023
  12. Jul 17, 2023
  13. Jul 14, 2023
  14. Jul 11, 2023
    • Bastian Köcher's avatar
      Removal of execution strategies (#14387) · 5eb816d7
      Bastian Köcher authored
      
      
      * Start
      
      * More work!
      
      * Moar
      
      * More changes
      
      * More fixes
      
      * More worrk
      
      * More fixes
      
      * More fixes to make it compile
      
      * Adds `NoOffchainStorage`
      
      * Pass the extensions
      
      * Small basti making small progress
      
      * Fix merge errors and remove `ExecutionContext`
      
      * Move registration of `ReadRuntimeVersionExt` to `ExecutionExtension`
      
      Instead of registering `ReadRuntimeVersionExt` in `sp-state-machine` it is moved to
      `ExecutionExtension` which provides the default extensions.
      
      * Fix compilation
      
      * Register the global extensions inside runtime api instance
      
      * Fixes
      
      * Fix `generate_initial_session_keys` by passing the keystore extension
      
      * Fix the grandpa tests
      
      * Fix more tests
      
      * Fix more tests
      
      * Don't set any heap pages if there isn't an override
      
      * Fix small fallout
      
      * FMT
      
      * Fix tests
      
      * More tests
      
      * Offchain worker custom extensions
      
      * More fixes
      
      * Make offchain tx pool creation reusable
      
      Introduces an `OffchainTransactionPoolFactory` for creating offchain transactions pools that can be
      registered in the runtime externalities context. This factory will be required for a later pr to
      make the creation of offchain transaction pools easier.
      
      * Fixes
      
      * Fixes
      
      * Set offchain transaction pool in BABE before using it in the runtime
      
      * Add the `offchain_tx_pool` to Grandpa as well
      
      * Fix the nodes
      
      * Print some error when using the old warnings
      
      * Fix merge issues
      
      * Fix compilation
      
      * Rename `babe_link`
      
      * Rename to `offchain_tx_pool_factory`
      
      * Cleanup
      
      * FMT
      
      * Fix benchmark name
      
      * Fix `try-runtime`
      
      * Remove `--execution` CLI args
      
      * Make clippy happy
      
      * Forward bls functions
      
      * Fix docs
      
      * Update UI tests
      
      * Update client/api/src/execution_extensions.rs
      
      Co-authored-by: default avatarMichal Kucharczyk <[email protected]>
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarKoute <[email protected]>
      
      * Update client/cli/src/params/import_params.rs
      
      Co-authored-by: default avatarKoute <[email protected]>
      
      * Update client/api/src/execution_extensions.rs
      
      Co-authored-by: default avatarKoute <[email protected]>
      
      * Pass the offchain storage to the MMR RPC
      
      * Update client/api/src/execution_extensions.rs
      
      Co-authored-by: default avatarSebastian Kunert <[email protected]>
      
      * Review comments
      
      * Fixes
      
      ---------
      
      Co-authored-by: default avatarMichal Kucharczyk <[email protected]>
      Co-authored-by: default avatarKoute <[email protected]>
      Co-authored-by: default avatarSebastian Kunert <[email protected]>
      5eb816d7
  15. Jul 09, 2023
  16. Jul 06, 2023
  17. Jul 05, 2023
  18. Jul 03, 2023
  19. Jun 29, 2023
  20. Jun 28, 2023
  21. Jun 27, 2023
  22. Jun 26, 2023
  23. Jun 24, 2023
  24. Jun 22, 2023