Skip to content
Snippets Groups Projects
  1. Jan 23, 2024
    • Niklas Adolfsson's avatar
      rpc: backpressured RPC server (bump jsonrpsee 0.20) (#1313) · e16ef086
      Niklas Adolfsson authored
      This is a rather big change in jsonrpsee, the major things in this bump
      are:
      - Server backpressure (the subscription impls are modified to deal with
      that)
      - Allow custom error types / return types (remove jsonrpsee::core::Error
      and jsonrpee::core::CallError)
      - Bug fixes (graceful shutdown in particular not used by substrate
      anyway)
         - Less dependencies for the clients in particular
         - Return type requires Clone in method call responses
         - Moved to tokio channels
         - Async subscription API (not used in this PR)
      
      Major changes in this PR:
      - The subscriptions are now bounded and if subscription can't keep up
      with the server it is dropped
      - CLI: add parameter to configure the jsonrpc server bounded message
      buffer (default is 64)
      - Add our own subscription helper to deal with the unbounded streams in
      substrate
      
      The most important things in this PR to review is the added helpers
      functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
      much chore.
      
      Regarding the "bounded buffer limit" it may cause the server to handle
      the JSON-RPC calls
      slower than before.
      
      The message size limit is bounded by "--rpc-response-size" thus "by
      default 10MB * 64 = 640MB"
      but the subscription message size is not covered by this limit and could
      be capped as well.
      
      Hopefully the last release prior to 1.0, sorry in advance for a big PR
      
      Previous attempt: https://github.com/paritytech/substrate/pull/13992
      
      Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
      https://github.com/paritytech/polkadot-sdk/issues/627
  2. Jan 12, 2024
    • Dmitry Markin's avatar
      Extract warp sync strategy from `ChainSync` (#2467) · 5208bed7
      Dmitry Markin authored
      
      Extract `WarpSync` (and `StateSync` as part of warp sync) from
      `ChainSync` as independent syncing strategy called by `SyncingEngine`.
      Introduce `SyncingStrategy` enum as a proxy between `SyncingEngine` and
      specific syncing strategies.
      
      ## Limitations
      Gap sync is kept in `ChainSync` for now because it shares the same set
      of peers as block syncing implementation in `ChainSync`. Extraction of a
      common context responsible for peer management in syncing strategies
      able to run in parallel is planned for a follow-up PR.
      
      ## Further improvements
      A possibility of conversion of `SyncingStartegy` into a trait should be
      evaluated. The main stopper for this is that different strategies need
      to communicate different actions to `SyncingEngine` and respond to
      different events / provide different APIs (e.g., requesting
      justifications is only possible via `ChainSync` and not through
      `WarpSync`; `SendWarpProofRequest` action is only relevant to
      `WarpSync`, etc.)
      
      ---------
      
      Co-authored-by: default avatarAaro Altonen <48052676+altonen@users.noreply.github.com>
  3. Nov 30, 2023
    • Sebastian Kunert's avatar
      PoV Reclaim (Clawback) Node Side (#1462) · 9a650c46
      Sebastian Kunert authored
      This PR provides the infrastructure for the pov-reclaim mechanism
      discussed in #209. The goal is to provide the current proof size to the
      runtime so it can be used to reclaim storage weight.
      
      ## New Host Function
      - A new host function is provided
      [here](https://github.com/skunert/polkadot-sdk/blob/5b317fda
      
      /cumulus/primitives/pov-reclaim/src/lib.rs#L23).
      It returns the size of the current proof size to the runtime. If
      recording is not enabled, it returns 0.
      
      ## Implementation Overview
      - Implement option to enable proof recording during import in the
      client. This is currently enabled for `polkadot-parachain`,
      `parachain-template` and the cumulus test node.
      - Make the proof recorder ready for no-std. It was previously only
      enabled for std environments, but we need to record the proof size in
      `validate_block` too.
      - Provide a recorder implementation that only the records the size of
      incoming nodes and does not store the nodes itself.
      - Fix benchmarks that were broken by async backing changes
      - Provide new externalities extension that is registered by default if
      proof recording is enabled.
      - I think we should discuss the naming, pov-reclaim was more intuitive
      to me, but we could also go with clawback like in the issue.
      
      ## Impact of proof recording during import
      With proof recording: 6.3058 Kelem/s
      Without proof recording: 6.3427 Kelem/s
      
      The measured impact on the importing performance is quite low on my
      machine using the block import benchmark. With proof recording I am
      seeing a performance hit of 0.585%.
      
      ---------
      
      Co-authored-by: command-bot <>
      Co-authored-by: default avatarDavide Galassi <davxy@datawok.net>
      Co-authored-by: default avatarBastian Köcher <git@kchr.de>
  4. Sep 27, 2023
    • Michal Kucharczyk's avatar
      `BlockId` removal: `tx-pool` refactor (#1678) · ab3a3bc2
      Michal Kucharczyk authored
      
      It changes following APIs:
      - trait `ChainApi`
      -- `validate_transaction`
      
      - trait `TransactionPool` 
      --`submit_at`
      --`submit_one`
      --`submit_and_watch`
      
      and some implementation details, in particular:
      - impl `Pool` 
      --`submit_at`
      --`resubmit_at`
      --`submit_one`
      --`submit_and_watch`
      --`prune_known`
      --`prune`
      --`prune_tags`
      --`resolve_block_number`
      --`verify`
      --`verify_one`
      
      - revalidation queue
      
      All tests are also adjusted.
      
      ---------
      
      Co-authored-by: command-bot <>
      Co-authored-by: default avatarBastian Köcher <git@kchr.de>
  5. Sep 05, 2023
  6. Aug 16, 2023
  7. Jul 25, 2023
    • Anton's avatar
      chore: update libp2p to 0.52.1 (#14429) · 59d8b864
      Anton authored
      * update libp2p to 0.52.0
      
      * proto name now must implement `AsRef<str>`
      
      * update libp2p version everywhere
      
      * ToSwarm, FromBehaviour, ToBehaviour
      
      also LocalProtocolsChange and RemoteProtocolsChange
      
      * new NetworkBehaviour invariants
      
      * replace `Vec<u8>` with `StreamProtocol`
      
      * rename ConnectionHandlerEvent::Custom to NotifyBehaviour
      
      * remove DialError & ListenError invariants
      
      also fix pending_events
      
      * use connection_limits::Behaviour
      
      See https://github.com/libp2p/rust-libp2p/pull/3885
      
      * impl `void::Void` for `BehaviourOut`
      
      also use `Behaviour::with_codec`
      
      * KademliaHandler no longer public
      
      * fix StreamProtocol construction
      
      * update libp2p-identify to 0.2.0
      
      * remove non-existing methods from PollParameters
      
      rename ConnectionHandlerUpgrErr to StreamUpgradeError
      
      * `P2p` now contains `PeerId`, not `Multihash`
      
      * use multihash-codetable crate
      
      * update Cargo.lock
      
      * reformat text
      
      * comment out tests for now
      
      * remove `.into()` from P2p
      
      * confirm observed addr manually
      
      See https://github.com/libp2p/rust-libp2p/blob/master/protocols/identify/CHANGELOG.md#0430
      
      * remove SwarmEvent::Banned
      
      since we're not using `ban_peer_id`, this can be safely removed.
      we may want to introduce `libp2p::allow_block_list` module in the future.
      
      * fix imports
      
      * replace `libp2p` with smaller deps in network-gossip
      
      * bring back tests
      
      * finish rewriting tests
      
      * uncomment handler tests
      
      * Revert "uncomment handler tests"
      
      This reverts commit 720a06815887f4e10767c62b58864a7ec3a48e50.
      
      * add a fixme
      
      * update Cargo.lock
      
      * remove extra From
      
      * make void uninhabited
      
      * fix discovery test
      
      * use autonat protocols
      
      confirming external addresses manually is unsafe in open networks
      
      * fix SyncNotificationsClogged invariant
      
      * only set server mode manually in tests
      
      doubt that we need to set it on node since we're adding public addresses
      
      * address @dmitry-markin comments
      
      * remove autonat
      
      * removed unused var
      
      * fix EOL
      
      * update smallvec and sha2
      
      in attempt to compile polkadot
      
      * bump k256
      
      in attempt to build cumulus
      
      ---------
      
      Co-authored-by: parity-processbot <>
      59d8b864
  8. Jul 17, 2023
  9. Jul 11, 2023
    • Bastian Köcher's avatar
      Removal of execution strategies (#14387) · 5eb816d7
      Bastian Köcher authored
      * Start
      
      * More work!
      
      * Moar
      
      * More changes
      
      * More fixes
      
      * More worrk
      
      * More fixes
      
      * More fixes to make it compile
      
      * Adds `NoOffchainStorage`
      
      * Pass the extensions
      
      * Small basti making small progress
      
      * Fix merge errors and remove `ExecutionContext`
      
      * Move registration of `ReadRuntimeVersionExt` to `ExecutionExtension`
      
      Instead of registering `ReadRuntimeVersionExt` in `sp-state-machine` it is moved to
      `ExecutionExtension` which provides the default extensions.
      
      * Fix compilation
      
      * Register the global extensions inside runtime api instance
      
      * Fixes
      
      * Fix `generate_initial_session_keys` by passing the keystore extension
      
      * Fix the grandpa tests
      
      * Fix more tests
      
      * Fix more tests
      
      * Don't set any heap pages if there isn't an override
      
      * Fix small fallout
      
      * FMT
      
      * Fix tests
      
      * More tests
      
      * Offchain worker custom extensions
      
      * More fixes
      
      * Make offchain tx pool creation reusable
      
      Introduces an `OffchainTransactionPoolFactory` for creating offchain trans...
      5eb816d7
  10. May 31, 2023
  11. May 07, 2023
  12. May 04, 2023
    • Michal Kucharczyk's avatar
      substrate-test-runtime migrated to "pure" frame runtime (#13737) · 6a295e7c
      Michal Kucharczyk authored
      * substrate-test-runtime migrated to pure-frame based
      
      * test block builder: helpers added
      
      * simple renaming
      
      * basic_authorship test adjusted
      
      * block_building storage_proof test adjusted
      
      * babe: tests: should_panic expected added
      
      * babe: tests adjusted
      
      ConsensusLog::NextEpochData is now added by pallet_babe as
      pallet_babe::SameAuthoritiesForever trigger is used in runtime config.
      
      * beefy: tests adjusted
      
      test-substrate-runtime is now using frame::executive to finalize the
      block. during finalization the digests stored during block execution are
      checked against header digests:
      https://github.com/paritytech/substrate/blob/91bb2d29/frame/executive/src/lib.rs#L585-L591
      
      It makes impossible to directly manipulate header's digets, w/o
      depositing logs into system pallet storage `Digest<T: Config>`.
      
      Instead of this dedicated extrinsic allowing to store logs items
      (MmrRoot / AuthoritiesChange) is used.
      
      * grandpa: tests adjusted
      
      test-substrate-runtime is now using frame::executive to finalize the
      block. during finalization the digest logs stored during block execution are
      checked against header digest logs:
      https://github.com/paritytech/substrate/blob/91bb2d29
      
      /frame/executive/src/lib.rs#L585-L591
      
      It makes impossible to directly manipulate header's digets, w/o
      depositing logs into system pallet storage `Digest<T: Config>`.
      
      Instead of this dedicated extrinsic allowing to store logs items
      (ScheduledChange / ForcedChange and DigestItem::Other) is used.
      
      * network:bitswap: test adjusted
      
      The size of unchecked extrinsic was increased. The pattern used in test will
      be placed at the end of scale-encoded buffer.
      
      * runtime apis versions adjusted
      
      * storage keys used in runtime adjusted
      
      * wasm vs native tests removed
      
      * rpc tests: adjusted
      
      Transfer transaction processing was slightly improved, test was
      adjusted.
      
      * tests: sizes adjusted
      
      Runtime extrinsic size was increased. Size of data read during block
      execution was also increased due to usage of new pallets in runtime.
      
      Sizes were adjusted in tests.
      
      * cargo.lock update
      
      cargo update -p substrate-test-runtime -p substrate-test-runtime-client
      
      * warnings fixed
      
      * builders cleanup: includes / std
      
      * extrinsic validation cleanup
      
      * txpool: benches performance fixed
      
      * fmt
      
      * spelling
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarDavide Galassi <davxy@datawok.net>
      
      * Apply code review suggestions
      
      * Apply code review suggestions
      
      * get rid of 1063 const
      
      * renaming: UncheckedExtrinsic -> Extrinsic
      
      * test-utils-runtime: further step to pure-frame
      
      * basic-authorship: tests OK
      
      * CheckSubstrateCall added + tests fixes
      
      * test::Transfer call removed
      
      * priority / propagate / no sudo+root-testing
      
      * fixing warnings + format
      
      * cleanup: build2/nonce + format
      
      * final tests fixes
      
      all tests are passing
      
      * logs/comments removal
      
      * should_not_accept_old_signatures test removed
      
      * make txpool benches work again
      
      * Cargo.lock reset
      
      * format
      
      * sudo hack removed
      
      * txpool benches fix+cleanup
      
      * .gitignore reverted
      
      * rebase fixing + unsigned cleanup
      
      * Cargo.toml/Cargo.lock cleanup
      
      * force-debug feature removed
      
      * mmr tests fixed
      
      * make cargo-clippy happy
      
      * network sync test uses unsigned extrinsic
      
      * cleanup
      
      * ".git/.scripts/commands/fmt/fmt.sh"
      
      * push_storage_change signed call remove
      
      * GenesisConfig cleanup
      
      * fix
      
      * fix
      
      * GenesisConfig simplified
      
      * storage_keys_works: reworked
      
      * storage_keys_works: expected keys in vec
      
      * storage keys list moved to substrate-test-runtime
      
      * substrate-test: some sanity tests + GenesisConfigBuilder rework
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <git@kchr.de>
      
      * Apply suggestions from code review
      
      * Review suggestions
      
      * fix
      
      * fix
      
      * beefy: generate_blocks_and_sync block_num sync with actaul value
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarDavide Galassi <davxy@datawok.net>
      
      * Update test-utils/runtime/src/genesismap.rs
      
      Co-authored-by: default avatarDavide Galassi <davxy@datawok.net>
      
      * cargo update -p sc-rpc -p sc-transaction-pool
      
      * Review suggestions
      
      * fix
      
      * doc added
      
      * slot_duration adjusted for Babe::slot_duration
      
      * small doc fixes
      
      * array_bytes::hex used instead of hex
      
      * tiny -> medium name fix
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarSebastian Kunert <skunert49@gmail.com>
      
      * TransferData::try_from_unchecked_extrinsic -> try_from
      
      * Update Cargo.lock
      
      ---------
      
      Co-authored-by: parity-processbot <>
      Co-authored-by: default avatarDavide Galassi <davxy@datawok.net>
      Co-authored-by: default avatarBastian Köcher <git@kchr.de>
      Co-authored-by: default avatarSebastian Kunert <skunert49@gmail.com>
      6a295e7c
  13. May 03, 2023
  14. Apr 09, 2023
    • yjh's avatar
      refactor(sc-executor): use wasm executor builder instead of old apis (#13740) · d5e460b3
      yjh authored
      * refactor: use builder api for all executors
      
      * improve a lot
      
      * remove unused args
      
      * cleanup deps
      
      * fix inconsistency about heap alloc
      
      * add `heap_pages` back to try-runtime
      
      * fix
      
      * chore: reduce duplicated code for sc-service-test
      
      * cleanup code
      
      * fmt
      
      * improve test executor
      
      * improve
      
      * use #[deprecated]
      
      * set runtime_cache_size: 4
      
      * fix and improve
      
      * refactor builder
      
      * fix
      
      * fix bench
      
      * fix tests
      
      * fix warnings
      
      * fix warnings
      
      * fix
      
      * fix
      
      * update by suggestions
      
      * update name
      d5e460b3
  15. Mar 14, 2023
    • Aaro Altonen's avatar
      Move code from `sc-network-common` back to `sc-network` (#13592) · 9ced14e2
      Aaro Altonen authored
      
      * Move service tests to `client/network/tests`
      
      These tests depend on `sc-network` and `sc-network-sync` so they should
      live outside the crate.
      
      * Move some configs from `sc-network-common` to `sc-network`
      
      * Move `NetworkService` traits to `sc-network`
      
      * Move request-responses to `sc-network`
      
      * Remove more stuff
      
      * Remove rest of configs from `sc-network-common` to `sc-network`
      
      * Remove more stuff
      
      * Fix warnings
      
      * Update client/network/src/request_responses.rs
      
      Co-authored-by: default avatarDmitry Markin <dmitry@markin.tech>
      
      * Fix cargo doc
      
      ---------
      
      Co-authored-by: default avatarDmitry Markin <dmitry@markin.tech>
      9ced14e2
  16. Mar 06, 2023
    • Liu-Cheng Xu's avatar
      Expose `new_full_parts_with_genesis_builder` from sc-service (#13539) · 58d1d9e1
      Liu-Cheng Xu authored
      I forgot this in https://github.com/paritytech/substrate/pull/12291
      58d1d9e1
    • Aaro Altonen's avatar
      Extract syncing protocol from `sc-network` (#12828) · 1a7f5be0
      Aaro Altonen authored
      
      * Move import queue out of `sc-network`
      
      Add supplementary asynchronous API for the import queue which means
      it can be run as an independent task and communicated with through
      the `ImportQueueService`.
      
      This commit removes removes block and justification imports from
      `sc-network` and provides `ChainSync` with a handle to import queue so
      it can import blocks and justifications. Polling of the import queue is
      moved complete out of `sc-network` and `sc_consensus::Link` is
      implemented for `ChainSyncInterfaceHandled` so the import queue
      can still influence the syncing process.
      
      * Move stuff to SyncingEngine
      
      * Move `ChainSync` instanation to `SyncingEngine`
      
      Some of the tests have to be rewritten
      
      * Move peer hashmap to `SyncingEngine`
      
      * Let `SyncingEngine` to implement `ChainSyncInterface`
      
      * Introduce `SyncStatusProvider`
      
      * Move `sync_peer_(connected|disconnected)` to `SyncingEngine`
      
      * Implement `SyncEventStream`
      
      Remove `SyncConnected`/`SyncDisconnected` events from
      `NetworkEvenStream` and provide those events through
      `ChainSyncInterface` instead.
      
      Modify BEEFY/GRANDPA/transactions protocol and `NetworkGossip` to take
      `SyncEventStream` object which they listen to for incoming sync peer
      events.
      
      * Introduce `ChainSyncInterface`
      
      This interface provides a set of miscellaneous functions that other
      subsystems can use to query, for example, the syncing status.
      
      * Move event stream polling to `SyncingEngine`
      
      Subscribe to `NetworkStreamEvent` and poll the incoming notifications
      and substream events from `SyncingEngine`.
      
      The code needs refactoring.
      
      * Make `SyncingEngine` into an asynchronous runner
      
      This commits removes the last hard dependency of syncing from
      `sc-network` meaning the protocol now lives completely outside of
      `sc-network`, ignoring the hardcoded peerset entry which will be
      addressed in the future.
      
      Code needs a lot of refactoring.
      
      * Fix warnings
      
      * Code refactoring
      
      * Use `SyncingService` for BEEFY
      
      * Use `SyncingService` for GRANDPA
      
      * Remove call delegation from `NetworkService`
      
      * Remove `ChainSyncService`
      
      * Remove `ChainSync` service tests
      
      They were written for the sole purpose of verifying that `NetworWorker`
      continues to function while the calls are being dispatched to
      `ChainSync`.
      
      * Refactor code
      
      * Refactor code
      
      * Update client/finality-grandpa/src/communication/tests.rs
      
      Co-authored-by: default avatarAnton <anton.kalyaev@gmail.com>
      
      * Fix warnings
      
      * Apply review comments
      
      * Fix docs
      
      * Fix test
      
      * cargo-fmt
      
      * Update client/network/sync/src/engine.rs
      
      Co-authored-by: default avatarAnton <anton.kalyaev@gmail.com>
      
      * Update client/network/sync/src/engine.rs
      
      Co-authored-by: default avatarAnton <anton.kalyaev@gmail.com>
      
      * Add missing docs
      
      * Refactor code
      
      ---------
      
      Co-authored-by: default avatarAnton <anton.kalyaev@gmail.com>
      1a7f5be0
  17. Feb 28, 2023
  18. Feb 21, 2023
    • Vivek Pandya's avatar
      Remove years from copyright notes. (#13415) · bc53b9a0
      Vivek Pandya authored
      * Change copyright year to 2023 from 2022
      
      * Fix incorrect update of copyright year
      
      * Remove years from copy right header
      
      * Fix remaining files
      
      * Fix typo in a header and remove update-copyright.sh
      bc53b9a0
  19. Feb 20, 2023
    • Dmitry Markin's avatar
      Use async/await instead of manual polling of `NetworkWorker` (#13219) · 8d033b6d
      Dmitry Markin authored
      
      * Convert `NetworkWorker::poll()` into async `next_action()`
      
      * Use `NetworkWorker::next_action` instead of `poll` in `sc-network-test`
      
      * Revert "Use `NetworkWorker::next_action` instead of `poll` in `sc-network-test`"
      
      This reverts commit 4b5d851ec864f78f9d083a18a618fbe117c896d2.
      
      * Fix `sc-network-test` to poll `NetworkWorker::next_action`
      
      * Fix `sc_network::service` tests to poll `NetworkWorker::next_action`
      
      * Fix docs
      
      * kick CI
      
      * Factor out `next_worker_message()` & `next_swarm_event()`
      
      * Error handling: replace `futures::pending!()` with `expect()`
      
      * Simplify stream polling in `select!`
      
      * Replace `NetworkWorker::next_action()` with `run()`
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <git@kchr.de>
      
      * minor: comment
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <git@kchr.de>
      
      * Print debug log when network future is shut down
      
      * Evaluate `NetworkWorker::run()` future once before the loop
      
      * Fix client code to match new `NetworkService` interfaces
      
      * Make clippy happy
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <git@kchr.de>
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <git@kchr.de>
      
      * Revert "Apply suggestions from code review"
      
      This reverts commit 9fa646d0ed613e5f8623d3d37d1d59ec0a535850.
      
      * Make `NetworkWorker::run()` consume `self`
      
      * Terminate system RPC future if RPC rx stream has terminated.
      
      * Rewrite with let-else
      
      * Fix comments
      
      * Get `best_seen_block` and call `on_block_finalized` via `ChainSync` instead of `NetworkService`
      
      * rustfmt
      
      * make clippy happy
      
      * Tests: schedule wake if `next_action()` returned true
      
      * minor: comment
      
      * minor: fix `NetworkWorker` rustdoc
      
      * minor: amend the rustdoc
      
      * Fix bug that caused `on_demand_beefy_justification_sync` test to hang
      
      * rustfmt
      
      * Apply review suggestions
      
      ---------
      
      Co-authored-by: default avatarBastian Köcher <git@kchr.de>
      8d033b6d
  20. Feb 14, 2023
  21. Dec 28, 2022
    • Bastian Köcher's avatar
      Print "stalled" task on shutdown (#13022) · 0a94112c
      Bastian Köcher authored
      * Print "stalled" task on shutdown
      
      When the node is shutting down, we give the Tokio runtime 60 seconds to shutdown. If after these 60
      seconds there are still running tasks, we now print these tasks. This should help debugging nodes
      that have stalled tasks.
      
      This pr introduces a `TaskRegistry` that keeps track of all running tasks. Each task registers and
      unregisters itself in this `TaskRegistry`.
      
      * Fix rustdoc
      
      * Update client/service/src/lib.rs
      0a94112c
  22. Dec 19, 2022
    • Liu-Cheng Xu's avatar
      Support custom genesis block (#12291) · b92aa3db
      Liu-Cheng Xu authored
      * Set genesis block data using the built genesis block
      
      * Make resolve_state_version_from_wasm a separate function and some small refactorings
      
      Useful for the commit following.
      
      * Introduce trait BuildGenesisBlock
      
      Substrate users can use this trait to implement their custom genesis block when constructing the
      client.
      
      * Make call_executor test compile
      
      * cargo +nightly fmt --all
      
      * Fix test
      
      * Remove unnecessary clone
      
      * FMT
      
      * Apply review suggestions
      
      * Revert changes to new_full_client() and new_full_parts() signature
      
      * Remove needless `Block` type in `resolve_state_version_from_wasm`
      b92aa3db
  23. Dec 12, 2022
    • Niklas Adolfsson's avatar
      rpc server with HTTP/WS on the same socket (#12663) · 84303ca7
      Niklas Adolfsson authored
      * jsonrpsee v0.16
      
      add backwards compatibility
      
      run old http server on http only
      
      * cargo fmt
      
      * update jsonrpsee 0.16.1
      
      * less verbose cors log
      
      * fix nit in log: WS -> HTTP
      
      * revert needless changes in Cargo.lock
      
      * remove unused features in tower
      
      * fix nits; add client-core feature
      
      * jsonrpsee v0.16.2
      84303ca7
  24. Sep 26, 2022
  25. Aug 29, 2022
  26. Aug 09, 2022
    • Nazar Mokrynskyi's avatar
      Network sync refactoring (part 6) (#11940) · a685582b
      Nazar Mokrynskyi authored
      * Extract `NetworkKVProvider` trait in `sc-authority-discovery` and remove unnecessary dependency
      
      * Extract `NetworkSyncForkRequest` trait in `sc-finality-grandpa`
      
      * Relax requirements on `SyncOracle` trait, remove extra native methods from `NetworkService` that are already provided by trait impls
      
      * Move `NetworkSigner` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`
      
      * Move `NetworkKVProvider` trait from `sc-authority-discovery` into `sc-network-common` and de-duplicate methods on `NetworkService`
      
      * Minimize `sc-authority-discovery` dependency on `sc-network`
      
      * Move `NetworkSyncForkRequest` trait from `sc-finality-grandpa` to `sc-network-common` and de-duplicate methods in `NetworkService`
      
      * Extract `NetworkStatusProvider` trait and de-duplicate methods on `NetworkService`
      
      * Extract `NetworkPeers` trait and de-duplicate methods on `NetworkService`
      
      * Extract `NetworkEventStream` trait and de-duplicate methods on `NetworkService`
      
      * Move more methods from `NetworkService` into `NetworkPeers` trait
      
      * Move `NetworkStateInfo` trait into `sc-network-common`
      
      * Extract `NetworkNotification` trait and de-duplicate methods on `NetworkService`
      
      * Extract `NetworkRequest` trait and de-duplicate methods on `NetworkService`
      
      * Remove `NetworkService::local_peer_id()`, it is already provided by `NetworkStateInfo` impl
      
      * Extract `NetworkTransaction` trait and de-duplicate methods on `NetworkService`
      
      * Extract `NetworkBlock` trait and de-duplicate methods on `NetworkService`
      
      * Remove dependencies on `NetworkService` from most of the methods of `sc-service`
      
      * Address simple review comments
      a685582b
  27. Aug 08, 2022
  28. Aug 05, 2022
    • Bastian Köcher's avatar
      system_syncState: Always return highest block (#11979) · a3144848
      Bastian Köcher authored
      Before `highestBlock` was an optional that was omitted when it was `None`. We recently changed the
      way the `highestBlock` is determined, this resulted in having this value in 99.99% of the time being
      `None` when the node is syncing blocks at the tip. Now we always return a block for `highestBlock`.
      If sync doesn't return us any best seen block, we return our own local best block as `highestBlock`.
      This should mainly reflect the same behavior to before we changed the way the best seen block is determined.
      a3144848
  29. Aug 02, 2022
  30. Jul 21, 2022
  31. May 21, 2022
  32. May 10, 2022
    • David's avatar
      jsonrpsee integration (#8783) · 29c0c6a4
      David authored
      
      * Add tokio
      
      * No need to map CallError to CallError
      
      * jsonrpsee proc macros (#9673)
      
      * port error types to `JsonRpseeError`
      
      * migrate chain module to proc macro api
      
      * make it compile with proc macros
      
      * update branch
      
      * update branch
      
      * update to jsonrpsee master
      
      * port system rpc
      
      * port state rpc
      
      * port childstate & offchain
      
      * frame system rpc
      
      * frame transaction payment
      
      * bring back CORS hack to work with polkadot UI
      
      * port babe rpc
      
      * port manual seal rpc
      
      * port frame mmr rpc
      
      * port frame contracts rpc
      
      * port finality grandpa rpc
      
      * port sync state rpc
      
      * resolve a few TODO + no jsonrpc deps
      
      * Update bin/node/rpc-client/src/main.rs
      
      * Update bin/node/rpc-client/src/main.rs
      
      * Update bin/node/rpc-client/src/main.rs
      
      * Update bin/node/rpc-client/src/main.rs
      
      * Port over system_ rpc tests
      
      * Make it compile
      
      * Use prost 0.8
      
      * Use prost 0.8
      
      * Make it compile
      
      * Ignore more failing tests
      
      * Comment out WIP tests
      
      * fix nit in frame system api
      
      * Update lockfile
      
      * No more juggling tokio versions
      
      * No more wait_for_stop ?
      
      * Remove browser-testing
      
      * Arguments must be arrays
      
      * Use same argument names
      
      * Resolve todo: no wait_for_stop for WS server
      Add todo: is parse_rpc_result used?
      Cleanup imports
      
      * fmt
      
      * log
      
      * One test passes
      
      * update jsonrpsee
      
      * update jsonrpsee
      
      * cleanup rpc-servers crate
      
      * jsonrpsee: add host and origin filtering (#9787)
      
      * add access control in the jsonrpsee servers
      
      * use master
      
      * fix nits
      
      * rpc runtime_version safe
      
      * fix nits
      
      * fix grumbles
      
      * remove unused files
      
      * resolve some todos
      
      * jsonrpsee more cleanup (#9803)
      
      * more cleanup
      
      * resolve TODOs
      
      * fix some unwraps
      
      * remove type hints
      
      * update jsonrpsee
      
      * downgrade zeroize
      
      * pin jsonrpsee rev
      
      * remove unwrap nit
      
      * Comment out more tests that aren't ported
      
      * Comment out more tests
      
      * Fix tests after merge
      
      * Subscription test
      
      * Invalid nonce test
      
      * Pending exts
      
      * WIP removeExtrinsic test
      
      * Test remove_extrinsic
      
      * Make state test: should_return_storage work
      
      * Uncomment/fix the other non-subscription related state tests
      
      * test: author_insertKey
      
      * test: author_rotateKeys
      
      * Get rest of state tests passing
      
      * asyncify a little more
      
      * Add todo to note #msg change
      
      * Crashing test for has_session_keys
      
      * Fix error conversion to avoid stack overflows
      Port author_hasSessionKeys test
      fmt
      
      * test author_hasKey
      
      * Add two missing tests
      Add a check on the return type
      Add todos for James's concerns
      
      * RPC tests for state, author and system (#9859)
      
      * Fix test runner
      
      * Impl Default for SubscriptionTaskExecutor
      
      * Keep the minimul amount of code needed to compile tests
      
      * Re-instate `RpcSession` (for now)
      
      * cleanup
      
      * Port over RPC tests
      
      * Add tokio
      
      * No need to map CallError to CallError
      
      * Port over system_ rpc tests
      
      * Make it compile
      
      * Use prost 0.8
      
      * Use prost 0.8
      
      * Make it compile
      
      * Ignore more failing tests
      
      * Comment out WIP tests
      
      * Update lockfile
      
      * No more juggling tokio versions
      
      * No more wait_for_stop ?
      
      * Remove browser-testing
      
      * Arguments must be arrays
      
      * Use same argument names
      
      * Resolve todo: no wait_for_stop for WS server
      Add todo: is parse_rpc_result used?
      Cleanup imports
      
      * fmt
      
      * log
      
      * One test passes
      
      * Comment out more tests that aren't ported
      
      * Comment out more tests
      
      * Fix tests after merge
      
      * Subscription test
      
      * Invalid nonce test
      
      * Pending exts
      
      * WIP removeExtrinsic test
      
      * Test remove_extrinsic
      
      * Make state test: should_return_storage work
      
      * Uncomment/fix the other non-subscription related state tests
      
      * test: author_insertKey
      
      * test: author_rotateKeys
      
      * Get rest of state tests passing
      
      * asyncify a little more
      
      * Add todo to note #msg change
      
      * Crashing test for has_session_keys
      
      * Fix error conversion to avoid stack overflows
      Port author_hasSessionKeys test
      fmt
      
      * test author_hasKey
      
      * Add two missing tests
      Add a check on the return type
      Add todos for James's concerns
      
      * offchain rpc tests
      
      * Address todos
      
      * fmt
      
      Co-authored-by: default avatarJames Wilson <james@jsdw.me>
      
      * fix drop in state test
      
      * update jsonrpsee
      
      * fix ignored system test
      
      * fix chain tests
      
      * remove some boiler plate
      
      * Port BEEFY RPC (#9883)
      
      * Merge master
      
      * Port beefy RPC (ty @niklas!)
      
      * trivial changes left over from merge
      
      * Remove unused code
      
      * Update jsonrpsee
      
      * fix build
      
      * make tests compile again
      
      * beefy update jsonrpsee
      
      * fix: respect rpc methods policy
      
      * update cargo.lock
      
      * update jsonrpsee
      
      * update jsonrpsee
      
      * downgrade error logs
      
      * update jsonrpsee
      
      * Fix typo
      
      * remove unused file
      
      * Better name
      
      * Port Babe RPC tests
      
      * Put docs back
      
      * Resolve todo
      
      * Port tests for System RPCs
      
      * Resolve todo
      
      * fix build
      
      * Updated jsonrpsee to current master
      
      * fix: port finality grandpa rpc tests
      
      * Move .into() outside of the match
      
      * more review grumbles
      
      * jsonrpsee: add `rpc handlers` back (#10245)
      
      * add back RpcHandlers
      
      * cargo fmt
      
      * fix docs
      
      * fix grumble: remove needless alloc
      
      * resolve TODO
      
      * fmt
      
      * Fix typo
      
      * grumble: Use constants based on BASE_ERROR
      
      * grumble: DRY whitelisted listening addresses
      grumble: s/JSONRPC/JSON-RPC/
      
      * cleanup
      
      * grumbles: Making readers aware of the possibility of gaps
      
      * review grumbles
      
      * grumbles
      
      * remove notes from niklasad1
      
      * Update `jsonrpsee`
      
      * fix: jsonrpsee features
      
      * jsonrpsee: fallback to random port in case the specified port failed (#10304)
      
      * jsonrpsee: fallback to random port
      
      * better comment
      
      * Update client/rpc-servers/src/lib.rs
      
      Co-authored-by: default avatarMaciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
      
      * Update client/rpc-servers/src/lib.rs
      
      Co-authored-by: default avatarMaciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
      
      * address grumbles
      
      * cargo fmt
      
      * addrs already slice
      
      Co-authored-by: default avatarMaciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
      
      * Update jsonrpsee to 092081a0a2b8904c6ebd2cd99e16c7bc13ffc3ae
      
      * lockfile
      
      * update jsonrpsee
      
      * fix warning
      
      * Don't fetch jsonrpsee from crates
      
      * make tests compile again
      
      * fix rpc tests
      
      * remove unused deps
      
      * update tokio
      
      * fix rpc tests again
      
      * fix: test runner
      
      `HttpServerBuilder::builder` fails unless it's called within tokio runtime
      
      * cargo fmt
      
      * grumbles: fix subscription aliases
      
      * make clippy happy
      
      * update remaining subscriptions alias
      
      * cleanup
      
      * cleanup
      
      * fix chain subscription: less boiler plate (#10285)
      
      * fix chain subscription: less boiler plate
      
      * fix bad merge
      
      * cargo fmt
      
      * Switch to jsonrpsee 0.5
      
      * fix build
      
      * add missing features
      
      * fix nit: remove needless Box::pin
      
      * Integrate jsonrpsee metrics (#10395)
      
      * draft metrics impl
      
      * Use latest api
      
      * Add missing file
      
      * Http server metrics
      
      * cleanup
      
      * bump jsonrpsee
      
      * Remove `ServerMetrics` and use a single middleware for both connection counting (aka sessions) and call metrics.
      
      * fix build
      
      * remove needless Arc::clone
      
      * Update to jsonrpsee 0.6
      
      * lolz
      
      * fix metrics
      
      * Revert "lolz"
      
      This reverts commit eed6c6a56e78d8e307b4950f4c52a1c3a2322ba1.
      
      * fix: in-memory rpc support subscriptions
      
      * commit Cargo.lock
      
      * Update tests to 0.7
      
      * fix TODOs
      
      * ws server: generate subscriptionIDs as Strings
      
      Some libraries seems to expect the subscription IDs to be Strings, let's not break
      this in this PR.
      
      * Increase timeout
      
      * Port over tests
      
      * cleanup
      
      * Using error codes from the spec
      
      * fix clippy
      
      * cargo fmt
      
      * update jsonrpsee
      
      * fix nits
      
      * fix: rpc_query
      
      * enable custom subid gen through spawn_tasks
      
      * remove unsed deps
      
      * unify tokio deps
      
      * Revert "enable custom subid gen through spawn_tasks"
      
      This reverts commit 5c5eb70328fe39d154fdb55c56e637b4548cf470.
      
      * fix bad merge of `test-utils`
      
      * fix more nits
      
      * downgrade wasm-instrument to 0.1.0
      
      * [jsonrpsee]: enable custom RPC subscription ID generatation (#10731)
      
      * enable custom subid gen through spawn_tasks
      
      * fix nits
      
      * Update client/service/src/builder.rs
      
      Co-authored-by: default avatarDavid <dvdplm@gmail.com>
      
      * add Poc; needs jsonrpsee pr
      
      * update jsonrpsee
      
      * add re-exports
      
      * add docs
      
      Co-authored-by: default avatarDavid <dvdplm@gmail.com>
      
      * cargo fmt
      
      * fmt
      
      * port RPC-API dev
      
      * Remove unused file
      
      * fix nit: remove async trait
      
      * fix doc links
      
      * fix merge nit: remove jsonrpc deps
      
      * kill namespace on rpc apis
      
      * companion for jsonrpsee v0.10 (#11158)
      
      * companion for jsonrpsee v0.10
      
      * update versions v0.10.0
      
      * add some fixes
      
      * spelling
      
      * fix spaces
      
      Co-authored-by: default avatarNiklas Adolfsson <niklasadolfsson1@gmail.com>
      
      * send error before subs are closed
      
      * fix unsubscribe method names: chain
      
      * fix tests
      
      * jsonrpc server: print binded local address
      
      * grumbles: kill SubscriptionTaskExecutor
      
      * Update client/sync-state-rpc/src/lib.rs
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Update client/rpc/src/chain/chain_full.rs
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Update client/rpc/src/chain/chain_full.rs
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * sync-state-rpc: kill anyhow
      
      * no more anyhow
      
      * remove todo
      
      * jsonrpsee:  fix bad params in subscriptions. (#11251)
      
      * update jsonrpsee
      
      * fix error responses
      
      * revert error codes
      
      * dont do weird stuff in drop impl
      
      * rpc servers: remove needless clone
      
      * Remove silly constants
      
      * chore: update jsonrpsee v0.12
      
      * commit Cargo.lock
      
      * deps: downgrade git2
      
      * feat: CLI flag max subscriptions per connection
      
      * metrics: use old logging format
      
      * fix: read WS address from substrate output (#11379)
      
      Co-authored-by: default avatarNiklas Adolfsson <niklasadolfsson1@gmail.com>
      Co-authored-by: default avatarJames Wilson <james@jsdw.me>
      Co-authored-by: default avatarMaciej Hirsz <hello@maciej.codes>
      Co-authored-by: default avatarMaciej Hirsz <1096222+maciejhirsz@users.noreply.github.com>
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      29c0c6a4
  33. Apr 30, 2022
  34. Apr 29, 2022
    • Nazar Mokrynskyi's avatar
      Network sync refactoring (part 1) (#11303) · af6773ab
      Nazar Mokrynskyi authored
      * Remove unnecessary imports, move one internal re-export into where it is actually used, make one import explicit
      
      * Move a few data structures down into modules
      
      * Use generic parameters in `sc-network` instead of `chain::Client` trait
      
      * Remove unnecessary bound
      af6773ab
  35. Mar 04, 2022
  36. Feb 09, 2022
    • Koute's avatar
      Add a new host function for reporting fatal errors; make WASM backtraces... · 9a31b2c3
      Koute authored
      Add a new host function for reporting fatal errors; make WASM backtraces readable when printing out errors (#10741)
      
      * Add a new host function for reporting fatal errors
      
      * Fix one of the wasmtime executor tests
      
      * Have `#[runtime_interface(wasm_only)]` actually mean WASM-only, and not no_std-only
      
      * Print out errors through `Display` instead of `Debug`
      
      * Switch one more trait to require `Error` for its error instead of only `Debug`
      
      * Align to review comments
      9a31b2c3
  37. Jan 27, 2022
    • Davide Galassi's avatar
      Add stale branches heads to finality notifications (#10639) · 7f3bb8d0
      Davide Galassi authored
      
      * Add stale branches heads to finality notifications
      
      Warning. Previous implementation was sending a notification for
      each block between the previous (explicitly) finalized block and
      the new finalized one (with an hardcoded limit of 256).
      
      Now finality notification is sent only for the new finalized head and it
      contains the hash of the new finalized head, new finalized head header,
      a list of all the implicitly finalized blocks and a list of stale
      branches heads (i.e. the branches heads that are not part of the
      canonical chain anymore).
      
      * Add implicitly finalized blocks list to `ChainEvent::Finalized` message
      
      The list contains all the blocks between the previously finalized block
      up to the parent of the currently finalized one, sorted by block number.
      
      `Finalized` messages handler, part of the `MaintainedTransactionPool`
      implementation for `BasicPool`, still propagate full set of finalized
      blocks to the txpool by iterating over implicitly finalized blocks list.
      
      * Rust fmt
      
      * Greedy evaluation of `stale_heads` during finalization
      
      * Fix outdated assumption in a comment
      
      * Removed a test optimization that is no more relevant
      
      The loop was there to prevent sending to
      `peer.network.on_block_finalized` the full list of finalized blocks.
      
      Now only the finalized heads are received.
      
      * Last finalized block lookup not required anymore
      
      * Tests for block finality notifications payloads
      
      * Document a bit tricky condition to avoid duplicate finalization notifications
      
      * More idiomatic way to skip an iterator entry
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Cargo fmt iteration
      
      * Typo fix
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Fix potential failure when a finalized orphan block is imported
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      7f3bb8d0
  38. Jan 03, 2022