1. Mar 13, 2024
  2. Mar 12, 2024
    • Koute's avatar
      Add a PolkaVM-based executor (#3458) · b0f34e4b
      Koute authored
      This PR adds a new PolkaVM-based executor to Substrate.
      
      - The executor can now be used to actually run a PolkaVM-based runtime,
      and successfully produces blocks.
      - The executor is always compiled-in, but is disabled by default.
      - The `SUBSTRATE_ENABLE_POLKAVM` environment variable must be set to `1`
      to enable the executor, in which case the node will accept both WASM and
      PolkaVM program blobs (otherwise it'll default to WASM-only). This is
      deliberately undocumented and not explicitly exposed anywhere (e.g. in
      the command line arguments, or in the API) to disincentivize anyone from
      enabling it in production. If/when we'll move this into production usage
      I'll remove the environment variable and do it "properly".
      - I did not use our legacy runtime allocator for the PolkaVM executor,
      so currently every allocation inside of the runtime will leak guest
      memory until that particular instance is destroyed. The idea here is
      that I will work on the https://github.com/polkadot-fellows/RFCs/pull/4
      which will remove the need for the legacy allocator under WASM, and that
      will also allow us to use a proper non-leaking allocator under PolkaVM.
      - I also did some minor cleanups of the WASM executor and deleted some
      dead code.
      
      No prdocs included since this is not intended to be an end-user feature,
      but an unofficial experiment, and shouldn't affect any current
      production user. Once this is production-ready a full Polkadot
      Fellowship RFC will be necessary anyway.
      b0f34e4b
  3. Mar 11, 2024
  4. Mar 07, 2024
  5. Mar 04, 2024
    • Gavin Wood's avatar
      FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280) · fd5f9292
      Gavin Wood authored
      Closes #2160
      
      First part of [Extrinsic
      Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415
      
      )
      
      Introduces a new trait `TransactionExtension` to replace
      `SignedExtension`. Introduce the idea of transactions which obey the
      runtime's extensions and have according Extension data (né Extra data)
      yet do not have hard-coded signatures.
      
      Deprecate the terminology of "Unsigned" when used for
      transactions/extrinsics owing to there now being "proper" unsigned
      transactions which obey the extension framework and "old-style" unsigned
      which do not. Instead we have __*General*__ for the former and
      __*Bare*__ for the latter. (Ultimately, the latter will be phased out as
      a type of transaction, and Bare will only be used for Inherents.)
      
      Types of extrinsic are now therefore:
      - Bare (no hardcoded signature, no Extra data; used to be known as
      "Unsigned")
      - Bare transactions (deprecated): Gossiped, validated with
      `ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
      `TransactionExtension` (deprecated).
        - Inherents: Not gossiped, validated with `ProvideInherent`.
      - Extended (Extra data): Gossiped, validated via `TransactionExtension`.
        - Signed transactions (with a hardcoded signature).
        - General transactions (without a hardcoded signature).
      
      `TransactionExtension` differs from `SignedExtension` because:
      - A signature on the underlying transaction may validly not be present.
      - It may alter the origin during validation.
      - `pre_dispatch` is renamed to `prepare` and need not contain the checks
      present in `validate`.
      - `validate` and `prepare` is passed an `Origin` rather than a
      `AccountId`.
      - `validate` may pass arbitrary information into `prepare` via a new
      user-specifiable type `Val`.
      - `AdditionalSigned`/`additional_signed` is renamed to
      `Implicit`/`implicit`. It is encoded *for the entire transaction* and
      passed in to each extension as a new argument to `validate`. This
      facilitates the ability of extensions to acts as underlying crypto.
      
      There is a new `DispatchTransaction` trait which contains only default
      function impls and is impl'ed for any `TransactionExtension` impler. It
      provides several utility functions which reduce some of the tedium from
      using `TransactionExtension` (indeed, none of its regular functions
      should now need to be called directly).
      
      Three transaction version discriminator ("versions") are now
      permissible:
      - 0b000000100: Bare (used to be called "Unsigned"): contains Signature
      or Extra (extension data). After bare transactions are no longer
      supported, this will strictly identify an Inherents only.
      - 0b100000100: Old-school "Signed" Transaction: contains Signature and
      Extra (extension data).
      - 0b010000100: New-school "General" Transaction: contains Extra
      (extension data), but no Signature.
      
      For the New-school General Transaction, it becomes trivial for authors
      to publish extensions to the mechanism for authorizing an Origin, e.g.
      through new kinds of key-signing schemes, ZK proofs, pallet state,
      mutations over pre-authenticated origins or any combination of the
      above.
      
      ## Code Migration
      
      ### NOW: Getting it to build
      
      Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
      accompanied by renaming your aggregate type in line with the new
      terminology. E.g. Before:
      
      ```rust
      /// The SignedExtension to the basic transaction logic.
      pub type SignedExtra = (
      	/* snip */
      	MySpecialSignedExtension,
      );
      /// Unchecked extrinsic type as expected by this runtime.
      pub type UncheckedExtrinsic =
      	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
      ```
      
      After:
      
      ```rust
      /// The extension to the basic transaction logic.
      pub type TxExtension = (
      	/* snip */
      	AsTransactionExtension<MySpecialSignedExtension>,
      );
      /// Unchecked extrinsic type as expected by this runtime.
      pub type UncheckedExtrinsic =
      	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
      ```
      
      You'll also need to alter any transaction building logic to add a
      `.into()` to make the conversion happen. E.g. Before:
      
      ```rust
      fn construct_extrinsic(
      		/* snip */
      ) -> UncheckedExtrinsic {
      	let extra: SignedExtra = (
      		/* snip */
      		MySpecialSignedExtension::new(/* snip */),
      	);
      	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
      	let signature = payload.using_encoded(|e| sender.sign(e));
      	UncheckedExtrinsic::new_signed(
      		/* snip */
      		Signature::Sr25519(signature),
      		extra,
      	)
      }
      ```
      
      After:
      
      ```rust
      fn construct_extrinsic(
      		/* snip */
      ) -> UncheckedExtrinsic {
      	let tx_ext: TxExtension = (
      		/* snip */
      		MySpecialSignedExtension::new(/* snip */).into(),
      	);
      	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
      	let signature = payload.using_encoded(|e| sender.sign(e));
      	UncheckedExtrinsic::new_signed(
      		/* snip */
      		Signature::Sr25519(signature),
      		tx_ext,
      	)
      }
      ```
      
      ### SOON: Migrating to `TransactionExtension`
      
      Most `SignedExtension`s can be trivially converted to become a
      `TransactionExtension`. There are a few things to know.
      
      - Instead of a single trait like `SignedExtension`, you should now
      implement two traits individually: `TransactionExtensionBase` and
      `TransactionExtension`.
      - Weights are now a thing and must be provided via the new function `fn
      weight`.
      
      #### `TransactionExtensionBase`
      
      This trait takes care of anything which is not dependent on types
      specific to your runtime, most notably `Call`.
      
      - `AdditionalSigned`/`additional_signed` is renamed to
      `Implicit`/`implicit`.
      - Weight must be returned by implementing the `weight` function. If your
      extension is associated with a pallet, you'll probably want to do this
      via the pallet's existing benchmarking infrastructure.
      
      #### `TransactionExtension`
      
      Generally:
      - `pre_dispatch` is now `prepare` and you *should not reexecute the
      `validate` functionality in there*!
      - You don't get an account ID any more; you get an origin instead. If
      you need to presume an account ID, then you can use the trait function
      `AsSystemOriginSigner::as_system_origin_signer`.
      - You get an additional ticket, similar to `Pre`, called `Val`. This
      defines data which is passed from `validate` into `prepare`. This is
      important since you should not be duplicating logic from `validate` to
      `prepare`, you need a way of passing your working from the former into
      the latter. This is it.
      - This trait takes two type parameters: `Call` and `Context`. `Call` is
      the runtime call type which used to be an associated type; you can just
      move it to become a type parameter for your trait impl. `Context` is not
      currently used and you can safely implement over it as an unbounded
      type.
      - There's no `AccountId` associated type any more. Just remove it.
      
      Regarding `validate`:
      - You get three new parameters in `validate`; all can be ignored when
      migrating from `SignedExtension`.
      - `validate` returns a tuple on success; the second item in the tuple is
      the new ticket type `Self::Val` which gets passed in to `prepare`. If
      you use any information extracted during `validate` (off-chain and
      on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
      pass it through with this. For the tuple's last item, just return the
      `origin` argument.
      
      Regarding `prepare`:
      - This is renamed from `pre_dispatch`, but there is one change:
      - FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
      `validate`!!
      - (This is different to `SignedExtension` which was required to run the
      same checks in `pre_dispatch` as in `validate`.)
      
      Regarding `post_dispatch`:
      - Since there are no unsigned transactions handled by
      `TransactionExtension`, `Pre` is always defined, so the first parameter
      is `Self::Pre` rather than `Option<Self::Pre>`.
      
      If you make use of `SignedExtension::validate_unsigned` or
      `SignedExtension::pre_dispatch_unsigned`, then:
      - Just use the regular versions of these functions instead.
      - Have your logic execute in the case that the `origin` is `None`.
      - Ensure your transaction creation logic creates a General Transaction
      rather than a Bare Transaction; this means having to include all
      `TransactionExtension`s' data.
      - `ValidateUnsigned` can still be used (for now) if you need to be able
      to construct transactions which contain none of the extension data,
      however these will be phased out in stage 2 of the Transactions Horizon,
      so you should consider moving to an extension-centric design.
      
      ## TODO
      
      - [x] Introduce `CheckSignature` impl of `TransactionExtension` to
      ensure it's possible to have crypto be done wholly in a
      `TransactionExtension`.
      - [x] Deprecate `SignedExtension` and move all uses in codebase to
      `TransactionExtension`.
        - [x] `ChargeTransactionPayment`
        - [x] `DummyExtension`
        - [x] `ChargeAssetTxPayment` (asset-tx-payment)
        - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment)
        - [x] `CheckWeight`
        - [x] `CheckTxVersion`
        - [x] `CheckSpecVersion`
        - [x] `CheckNonce`
        - [x] `CheckNonZeroSender`
        - [x] `CheckMortality`
        - [x] `CheckGenesis`
        - [x] `CheckOnlySudoAccount`
        - [x] `WatchDummy`
        - [x] `PrevalidateAttests`
        - [x] `GenericSignedExtension`
        - [x] `SignedExtension` (chain-polkadot-bulletin)
        - [x] `RefundSignedExtensionAdapter`
      - [x] Implement `fn weight` across the board.
      - [ ] Go through all pre-existing extensions which assume an account
      signer and explicitly handle the possibility of another kind of origin.
      - [x] `CheckNonce` should probably succeed in the case of a non-account
      origin.
      - [x] `CheckNonZeroSender` should succeed in the case of a non-account
      origin.
      - [x] `ChargeTransactionPayment` and family should fail in the case of a
      non-account origin.
        - [ ] 
      - [x] Fix any broken tests.
      
      ---------
      
      Signed-off-by: default avatargeorgepisaltu <[email protected]>
      Signed-off-by: default avatarAlexandru Vasile <[email protected]>
      Signed-off-by: default avatardependabot[bot] <[email protected]>
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      Signed-off-by: default avatarAlexandru Gheorghe <[email protected]>
      Signed-off-by: default avatarAndrei Sandu <[email protected]>
      Co-authored-by: default avatarNikhil Gupta <[email protected]>
      Co-authored-by: default avatargeorgepisaltu <[email protected]>
      Co-authored-by: default avatarChevdor <[email protected]>
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      Co-authored-by: default avatarMaciej <[email protected]>
      Co-authored-by: default avatarJavier Viola <[email protected]>
      Co-authored-by: default avatarMarcin S. <[email protected]>
      Co-authored-by: default avatarTsvetomir Dimitrov <[email protected]>
      Co-authored-by: default avatarJavier Bullrich <[email protected]>
      Co-authored-by: default avatarKoute <[email protected]>
      Co-authored-by: default avatarAdrian Catangiu <[email protected]>
      Co-authored-by: Vladimir Istyufeev's avatarVladimir Istyufeev <[email protected]>
      Co-authored-by: default avatarRoss Bulat <[email protected]>
      Co-authored-by: default avatarGonçalo Pestana <[email protected]>
      Co-authored-by: default avatarLiam Aharon <[email protected]>
      Co-authored-by: default avatarSvyatoslav Nikolsky <[email protected]>
      Co-authored-by: default avatarAndré Silva <[email protected]>
      Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: default avatars0me0ne-unkn0wn <[email protected]>
      Co-authored-by: default avatarordian <[email protected]>
      Co-authored-by: default avatarSebastian Kunert <[email protected]>
      Co-authored-by: default avatarAaro Altonen <[email protected]>
      Co-authored-by: default avatarDmitry Markin <[email protected]>
      Co-authored-by: default avatarAlexandru Vasile <[email protected]>
      Co-authored-by: default avatarAlexander Samusev <[email protected]>
      Co-authored-by: default avatarJulian Eager <[email protected]>
      Co-authored-by: default avatarMichal Kucharczyk <[email protected]>
      Co-authored-by: default avatarDavide Galassi <[email protected]>
      Co-authored-by: default avatarDónal Murray <[email protected]>
      Co-authored-by: default avataryjh <[email protected]>
      Co-authored-by: default avatarTom Mi <[email protected]>
      Co-authored-by: default avatardependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
      Co-authored-by: default avatarWill | Paradox | ParaNodes.io <[email protected]>
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      Co-authored-by: default avatarJoshy Orndorff <[email protected]>
      Co-authored-by: default avatarJoshy Orndorff <[email protected]>
      Co-authored-by: default avatarPG Herveou <[email protected]>
      Co-authored-by: default avatarAlexander Theißen <[email protected]>
      Co-authored-by: default avatarKian Paimani <[email protected]>
      Co-authored-by: default avatarJuan Girini <[email protected]>
      Co-authored-by: default avatarbader y <[email protected]>
      Co-authored-by: default avatarJames Wilson <[email protected]>
      Co-authored-by: default avatarjoe petrowski <[email protected]>
      Co-authored-by: default avatarasynchronous rob <[email protected]>
      Co-authored-by: default avatarParth <[email protected]>
      Co-authored-by: default avatarAndrew Jones <[email protected]>
      Co-authored-by: default avatarJonathan Udd <[email protected]>
      Co-authored-by: default avatarSerban Iorga <[email protected]>
      Co-authored-by: default avatarEgor_P <[email protected]>
      Co-authored-by: default avatarBranislav Kontur <[email protected]>
      Co-authored-by: default avatarEvgeny Snitko <[email protected]>
      Co-authored-by: default avatarJust van Stam <[email protected]>
      Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
      Co-authored-by: default avatargupnik <[email protected]>
      Co-authored-by: default avatardzmitry-lahoda <[email protected]>
      Co-authored-by: default avatarzhiqiangxu <[email protected]>
      Co-authored-by: default avatarNazar Mokrynskyi <[email protected]>
      Co-authored-by: default avatarAnwesh <[email protected]>
      Co-authored-by: default avatarcheme <[email protected]>
      Co-authored-by: default avatarSam Johnson <[email protected]>
      Co-authored-by: default avatarkianenigma <[email protected]>
      Co-authored-by: default avatarJegor Sidorenko <[email protected]>
      Co-authored-by: default avatarMuharem <[email protected]>
      Co-authored-by: default avatarjoepetrowski <[email protected]>
      Co-authored-by: default avatarAlexandru Gheorghe <[email protected]>
      Co-authored-by: default avatarGabriel Facco de Arruda <[email protected]>
      Co-authored-by: default avatarSquirrel <[email protected]>
      Co-authored-by: default avatarAndrei Sandu <[email protected]>
      Co-authored-by: default avatargeorgepisaltu <[email protected]>
      Co-authored-by: command-bot <>
      fd5f9292
  6. Feb 28, 2024
  7. Feb 26, 2024
  8. Feb 22, 2024
    • Koute's avatar
      Fix `wasm-builder` not exiting if compilation fails (#3439) · 82208280
      Koute authored
      This PR fixes a subtle bug in `wasm-builder` first introduced in
      https://github.com/paritytech/polkadot-sdk/pull/1851 (sorry, my bad! I
      should have caught this during review) where the status code of the
      `cargo` subprocess is not properly checked, which results in builds
      silently succeeding when they shouldn't (that is: if we successfully
      build a runtime blob, and then modify the code so that it won't compile,
      and recompile it again, then the build will succeed and silently use the
      *old* blob).
      
      cc @athei This is the bug you were seeing.
      
      [edit]Also fixes a similar PolkaVM-specific bug where I accidentally
      used the wrong comparison operator.[/edit]
      82208280
  9. Feb 21, 2024
    • Dastan's avatar
      benchmarking-cli: add --list-pallets and --all options (#3395) · 165d075a
      Dastan authored
      
      
      closes #2844 
      
      - adds `list-pallets` option which prints all unique available pallets
      for benchmarking
      ```bash
      ./target/release/node benchmark pallet --list=pallets
      ```
      - adds `all` option which runs benchmarks for all available pallets and
      extrinsics (equivalent to `--pallet * --extrinsic *`)
      ```bash
      ./target/release/node benchmark pallet --all
      ```
      
      - use the `list=pallets` syntax in `run_all_benchmarks.sh` script
      
      cc ggwpez
      
      ---------
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      165d075a
  10. Feb 20, 2024
    • Oliver Tale-Yazdi's avatar
      Lift dependencies to the workspace (Part 2/x) (#3366) · e89d0fca
      Oliver Tale-Yazdi authored
      
      
      Lifting some more dependencies to the workspace. Just using the
      most-often updated ones for now.
      It can be reproduced locally.
      
      ```sh
      # First you can check if there would be semver incompatible bumps (looks good in this case):
      $ zepter transpose dependency lift-to-workspace --ignore-errors syn quote thiserror "regex:^serde.*"
      
      # Then apply the changes:
      $ zepter transpose dependency lift-to-workspace --version-resolver=highest syn quote thiserror "regex:^serde.*" --fix
      
      # And format the changes:
      $ taplo format --config .config/taplo.toml
      ```
      
      ---------
      
      Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
      e89d0fca
  11. Feb 19, 2024
  12. Feb 16, 2024
  13. Feb 14, 2024
    • Niklas Adolfsson's avatar
      rpc: bump jsonrpsee v0.22 and fix race in `rpc v2 chain_head` (#3230) · c7c4fe01
      Niklas Adolfsson authored
      Close #2992 
      
      Breaking changes:
      - rpc server grafana metric `substrate_rpc_requests_started` is removed
      (not possible to implement anymore)
      - rpc server grafana metric `substrate_rpc_requests_finished` is removed
      (not possible to implement anymore)
      - rpc server ws ping/pong not ACK:ed within 30 seconds more than three
      times then the connection will be closed
      
      Added
      - rpc server grafana metric `substrate_rpc_sessions_time` is added to
      get the duration for each websocket session
      c7c4fe01
  14. Feb 13, 2024
  15. Feb 12, 2024
  16. Feb 08, 2024
  17. Feb 06, 2024
    • Koute's avatar
      Build more runtimes targeting PolkaVM (#3209) · 402b64ca
      Koute authored
      This PR improves compatibility with RISC-V and PolkaVM, allowing more
      runtimes to successfully compile.
      
      In particular, it makes the following changes:
      
      - The `sp-mmr-primitives` and `sp-consensus-beefy` crates
      unconditionally required an `std`-only dependency; now they only require
      those dependencies when the `std` feature is actually enabled. (Our
      RISC-V target is, unlike WASM, a true `no_std` target where you can't
      accidentally use stuff from `std` anymore.)
      - One of our dependencies (the `bitvec` trace) uses a crate called
      `radium` which doesn't compile under RISC-V due to incomplete
      autodetection logic in their `build.rs` file. The good news is that this
      is already fixed in the newest upstream version of `radium`, and the
      newest version of `bitvec` uses it. The bad news is that the newest
      version of `bitvec` is not currently released on crates.io, so we can't
      use it. I've [created an
      issue](https://github.com/ferrilab/ferrilab/issues/5) asking for a new
      release, but in the meantime I forked the currently used `radium` 0.7,
      [fixed the faulty
      logic](https://github.com/paritytech/radium-0.7-fork/commit/ed66c8a294b138c67f93499644051d97d4c7fbda)
      and used cargo's patching capabilities to use it for the RISC-V runtime
      builds. This might be a little hacky, but it is the least intrusive way
      to fix the problem, doesn't affect WASM builds at all, and we can
      trivially remove it once a new `bitvec` is released.
      - The new runtimes are added to the CI to make sure their compilation
      doesn't break.
      402b64ca
  18. Feb 05, 2024
  19. Feb 03, 2024
    • Koute's avatar
      Initial support for building RISC-V runtimes targeting PolkaVM (#3179) · e349fc9e
      Koute authored
      This PR adds initial support for building RISC-V runtimes targeting
      PolkaVM.
      
      - Setting the `SUBSTRATE_RUNTIME_TARGET=riscv` environment variable will
      now build a RISC-V runtime instead of a WASM runtime.
      - This only adds support for *building* runtimes; running them will need
      a PolkaVM-based executor, which I will add in a future PR.
      - Only building the minimal runtime is supported (building the Polkadot
      runtime doesn't work *yet* due to one of the dependencies).
      - The builder now sets a `substrate_runtime` cfg flag when building the
      runtimes, with the idea being that instead of doing `#[cfg(not(feature =
      "std"))]` or `#[cfg(target_arch = "wasm32")]` to detect that we're
      building a runtime you'll do `#[cfg(substrate_runtime)]`. (Switching the
      whole codebase to use this will be done in a future PR; I deliberately
      didn't do this here to keep this PR minimal and reviewable.)
      - Further renaming of things (e.g. types, environment variables and proc
      macro attributes having "wasm" in their name) to be target-agnostic will
      also be done in a future refactoring PR (while keeping backwards
      compatibility where it makes sense; I don't intend to break anyone's
      workflow or create unnecessary churn).
      - This PR also fixes two bugs in the `wasm-builder` crate:
      * The `RUSTC` environment variable is now removed when invoking the
      compiler. This prevents the toolchain version from being overridden when
      called from a `build.rs` script.
      * When parsing the `rustup toolchain list` output the `(default)` is now
      properly stripped and not treated as part of the version.
      - I've also added a minimal CI job that makes sure this doesn't break in
      the future. (cc @paritytech/ci)
      
      cc @athei
      
      
      
      ------
      
      Also, just a fun little tidbit: quickly comparing the size of the built
      runtimes it seems that the PolkaVM runtime is slightly smaller than the
      WASM one. (`production` build, with the `names` section substracted from
      the WASM's size to keep things fair, since for the PolkaVM runtime we're
      currently stripping out everything)
      
      - `.wasm`: 625505 bytes
      - `.wasm` (after wasm-opt -O3): 563205 bytes
      - `.wasm` (after wasm-opt -Os): 562987 bytes
      - `.wasm` (after wasm-opt -Oz): 536852 bytes
      - `.polkavm`: ~~580338 bytes~~ 550476 bytes (after enabling extra target
      features; I'll add those in another PR once we have an executor working)
      
      ---------
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      e349fc9e
  20. Jan 26, 2024
  21. Jan 23, 2024
    • Niklas Adolfsson's avatar
      rpc: backpressured RPC server (bump jsonrpsee 0.20) (#1313) · e16ef086
      Niklas Adolfsson authored
      This is a rather big change in jsonrpsee, the major things in this bump
      are:
      - Server backpressure (the subscription impls are modified to deal with
      that)
      - Allow custom error types / return types (remove jsonrpsee::core::Error
      and jsonrpee::core::CallError)
      - Bug fixes (graceful shutdown in particular not used by substrate
      anyway)
         - Less dependencies for the clients in particular
         - Return type requires Clone in method call responses
         - Moved to tokio channels
         - Async subscription API (not used in this PR)
      
      Major changes in this PR:
      - The subscriptions are now bounded and if subscription can't keep up
      with the server it is dropped
      - CLI: add parameter to configure the jsonrpc server bounded message
      buffer (default is 64)
      - Add our own subscription helper to deal with the unbounded streams in
      substrate
      
      The most important things in this PR to review is the added helpers
      functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
      much chore.
      
      Regarding the "bounded buffer limit" it may cause the server to handle
      the JSON-RPC calls
      slower than before.
      
      The message size limit is bounded by "--rpc-response-size" thus "by
      default 10MB * 64 = 640MB"
      but the subscription message size is not covered by this limit and could
      be capped as well.
      
      Hopefully the last release prior to 1.0, sorry in advance for a big PR
      
      Previous attempt: https://github.com/paritytech/substrate/pull/13992
      
      Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
      https://github.com/paritytech/polkadot-sdk/issues/627
      e16ef086
  22. Jan 22, 2024
  23. Jan 20, 2024
  24. Jan 18, 2024
  25. Jan 16, 2024
  26. Jan 15, 2024
  27. Jan 13, 2024
  28. Jan 10, 2024
  29. Jan 09, 2024
  30. Jan 08, 2024
  31. Jan 07, 2024
  32. Jan 04, 2024
  33. Dec 20, 2023
  34. Dec 18, 2023
  35. Dec 13, 2023
  36. Dec 11, 2023
  37. Dec 08, 2023
    • Sam Johnson's avatar
      Tasks: general system for recognizing and executing service work (#1343) · ac3f14d2
      Sam Johnson authored
      `polkadot-sdk` version of original tasks PR located here:
      https://github.com/paritytech/substrate/pull/14329
      
      Fixes #206
      
      ## Status
      - [x] Generic `Task` trait
      - [x] `RuntimeTask` aggregated enum, compatible with
      `construct_runtime!`
      - [x] Casting between `Task` and `RuntimeTask` without needing `dyn` or
      `Box`
      - [x] Tasks Example pallet
      - [x] Runtime tests for Tasks example pallet
      - [x] Parsing for task-related macros
      - [x] Retrofit parsing to make macros optional
      - [x] Expansion for task-related macros
      - [x] Adds support for args in tasks
      - [x] Retrofit tasks example pallet to use macros instead of manual
      syntax
      - [x] Weights
      - [x] Cleanup
      - [x] UI tests
      - [x] Docs
      
      ## Target Syntax
      Adapted from
      https://github.com/paritytech/polkadot-sdk/issues/206#issue-1865172283
      
      
      
      ```rust
      // NOTE: this enum is optional and is auto-generated by the other macros if not present
      #[pallet::task]
      pub enum Task<T: Config> {
          AddNumberIntoTotal {
              i: u32,
          }
      }
      
      /// Some running total.
      #[pallet::storage]
      pub(super) type Total<T: Config<I>, I: 'static = ()> =
      StorageValue<_, (u32, u32), ValueQuery>;
      
      /// Numbers to be added into the total.
      #[pallet::storage]
      pub(super) type Numbers<T: Config<I>, I: 'static = ()> =
      StorageMap<_, Twox64Concat, u32, u32, OptionQuery>;
      
      #[pallet::tasks_experimental]
      impl<T: Config<I>, I: 'static> Pallet<T, I> {
      	/// Add a pair of numbers into the totals and remove them.
      	#[pallet::task_list(Numbers::<T, I>::iter_keys())]
      	#[pallet::task_condition(|i| Numbers::<T, I>::contains_key(i))]
      	#[pallet::task_index(0)]
      	pub fn add_number_into_total(i: u32) -> DispatchResult {
      		let v = Numbers::<T, I>::take(i).ok_or(Error::<T, I>::NotFound)?;
      		Total::<T, I>::mutate(|(total_keys, total_values)| {
      			*total_keys += i;
      			*total_values += v;
      		});
      		Ok(())
      	}
      }
      ```
      
      ---------
      
      Co-authored-by: default avatarNikhil Gupta <[email protected]>
      Co-authored-by: default avatarkianenigma <[email protected]>
      Co-authored-by: Nikhil Gupta <>
      Co-authored-by: default avatarGavin Wood <[email protected]>
      Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: default avatargupnik <[email protected]>
      ac3f14d2