Skip to content
  1. Apr 03, 2024
    • Alexandru Vasile's avatar
      chainHead: Ensure reasonable distance between leaf and finalized block (#3562) · 287b116c
      Alexandru Vasile authored 
      This PR ensure that the distance between any leaf and the finalized
block is within a reasonable distance.

For a new subscription, the chainHead has to provide all blocks between
the leaves of the chain and the finalized block.
 When the distance between a leaf and the finalized block is large:
 - The tree route is costly to compute
 - We could deliver an unbounded number of blocks (potentially millions)
(For more details see
https://github.com/paritytech/polkadot-sdk/pull/3445#discussion_r1507210283

)

The configuration of the ChainHead is extended with:
- suspend on lagging distance: When the distance between any leaf and
the finalized block is greater than this number, the subscriptions are
suspended for a given duration.
- All active subscriptions are terminated with the `Stop` event, all
blocks are unpinned and data discarded.
- For incoming subscriptions, until the suspended period expires the
subscriptions will immediately receive the `Stop` event.
    - Defaults to 128 blocks
- suspended duration: The amount of time for which subscriptions are
suspended
    - Defaults to 30 seconds
 
 
 cc @paritytech/subxt-team

---------

Signed-off-by: default avatarAlexandru Vasile <[email protected]>
Co-authored-by: default avatarSebastian Kunert <[email protected]>
      287b116c
  3. Apr 02, 2024
  5. Apr 01, 2024
  7. Mar 31, 2024
  9. Mar 28, 2024
  11. Mar 27, 2024
  13. Mar 26, 2024
    • Pavel Orlov's avatar
      XCM Fee Payment Runtime API (#3607) · 3c972fc1
      Pavel Orlov authored 
      The PR provides API for obtaining:
- the weight required to execute an XCM message,
- a list of acceptable `AssetId`s for message execution payment,
- the cost of the weight in the specified acceptable `AssetId`.

It is meant to address an issue where one has to guess how much fee to
pay for execution. Also, at the moment, a client has to guess which
assets are acceptable for fee execution payment.
See the related issue
https://github.com/paritytech/polkadot-sdk/issues/690.
With this API, a client is supposed to query the list of the supported
asset IDs (in the XCM version format the client understands), weigh the
XCM program the client wants to execute and convert the weight into one
of the acceptable assets. Note that the client is supposed to know what
program will be executed on what chains. However, having a small
companion JS library for the pallet-xcm and xtokens should be enough to
determine what XCM programs will be executed and where (since these
pallets compose a known small set of programs).
```Rust
pub trait XcmPaymentApi<Call>
	where
		Call: Codec,
	{
		/// Returns a list of acceptable payment assets.
		///
		/// # Arguments
		///
		/// * `xcm_version`: Version.
		fn query_acceptable_payment_assets(xcm_version: Version) -> Result<Vec<VersionedAssetId>, Error>;
		/// Returns a weight needed to execute a XCM.
		///
		/// # Arguments
		///
		/// * `message`: `VersionedXcm`.
		fn query_xcm_weight(message: VersionedXcm<Call>) -> Result<Weight, Error>;
		/// Converts a weight into a fee for the specified `AssetId`.
		///
		/// # Arguments
		///
		/// * `weight`: convertible `Weight`.
		/// * `asset`: `VersionedAssetId`.
		fn query_weight_to_asset_fee(weight: Weight, asset: VersionedAssetId) -> Result<u128, Error>;
		/// Get delivery fees for sending a specific `message` to a `destination`.
		/// These always come in a specific asset, defined by the chain.
		///
		/// # Arguments
		/// * `message`: The message that'll be sent, necessary because most delivery fees are based on the
		///   size of the message.
		/// * `destination`: The destination to send the message to. Different destinations may use
		///   different senders that charge different fees.
		fn query_delivery_fees(destination: VersionedLocation, message: VersionedXcm<()>) -> Result<VersionedAssets, Error>;
	}
```
An
[example](https://gist.github.com/PraetorP/4bc323ff85401abe253897ba990ec29d

)
of a client side code.

---------

Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
Co-authored-by: default avatarAdrian Catangiu <[email protected]>
Co-authored-by: default avatarDaniel Shiposha <[email protected]>
      3c972fc1
    • Andrei Eres's avatar
      [subsystem-benchmarks] Save results to json (#3829) · fd79b3b0
      Andrei Eres authored 
      Here we add the ability to save subsystem benchmark results in JSON
format to display them as graphs

To draw graphs, CI team will use
[github-action-benchmark](https://github.com/benchmark-action/github-action-benchmark).
Since we are using custom benchmarks, we need to prepare [a specific
data
type](https://github.com/benchmark-action/github-action-benchmark?tab=readme-ov-file#examples):
```
[
    {
        "name": "CPU Load",
        "unit": "Percent",
        "value": 50
    }
]
```

Then we'll get graphs like this: 

![example](https://raw.githubusercontent.com/rhysd/ss/master/github-action-benchmark/main.png)

[A live page with
graphs](https://benchmark-action.github.io/github-action-benchmark/dev/bench/

)

---------

Co-authored-by: default avatarordian <[email protected]>
      fd79b3b0
  15. Mar 22, 2024
  17. Mar 20, 2024
  19. Mar 19, 2024
  21. Mar 18, 2024
    • Squirrel's avatar
      sp-std removal from substrate/primitives (#3274) · 1b5f4243
      Squirrel authored 
      

This PR removes sp-std crate from substrate/primitives sub-directories.

For now crates that have `pub use` of sp-std or export macros that would
necessitate users of the macros to `extern crate alloc` have been
excluded from this PR.

There should be no breaking changes in this PR.

---------

Co-authored-by: default avatarKoute <[email protected]>
      1b5f4243
  23. Mar 17, 2024
  25. Mar 15, 2024
  27. Mar 14, 2024
    • Ignacio Palacios's avatar
      Improve Penpal runtime + emulated tests (#3543) · cfc4050d
      Ignacio Palacios authored 
      Issues addressed in this PR:
- Improve *Penpal* runtime:
- Properly handled received assets. Previously, it treated `(1, Here)`
as the local native currency, whereas it should be treated as a
`ForeignAsset`. This wasn't a great example of standard Parachain
behaviour, as no Parachain treats the system asset as the local
currency.
- Remove `AllowExplicitUnpaidExecutionFrom` the system. Again, this
wasn't a great example of standard Parachain behaviour.
- Move duplicated
`ForeignAssetFeeAsExistentialDepositMultiplierFeeCharger` to
`assets_common` crate.
- Improve emulated tests:
  - Update *Penpal* tests to new runtime.
- To simplify tests, register the reserve transferred, teleported, and
system assets in *Penpal* and *AssetHub* genesis. This saves us from
having to create the assets repeatedly for each test
- Add missing test case:
`reserve_transfer_assets_from_para_to_system_para`.
  - Cleanup.
- Prevent integration tests crates imports from being re-exported, as
they were polluting the `polkadot-sdk` docs.

There is still a test case missing for reserve transfers:
- Reserve transfer of system asset from *Parachain* to *Parachain*
trough *AssetHub*.
- This is not yet possible with `pallet-xcm` due to the reasons
explained in https://github.com/paritytech/polkadot-sdk/pull/3339

---------

Co-authored-by: command-bot <>
      cfc4050d
  29. Mar 13, 2024
  31. Mar 12, 2024
  33. Mar 11, 2024
  35. Mar 09, 2024
  37. Mar 07, 2024
  39. Mar 06, 2024
    • Branislav Kontur's avatar
      Make `penpal-runtime`'s `TrustedReserves` more connfigurable (#3564) · 117a9433
      Branislav Kontur authored 
      

The current `penpal` runtime utilizes the `EthereumLocation` parameter,
which is employed for XCM emulated integration tests concerning the
Rococo <> ETH bridge. It includes a hard-coded chainId for the Ethereum
testnet utilized in Rococo. The `EthereumLocation` serves the purpose of
aligning with the `TrustedReserves`. However, due to this hard-coded
configuration, reusing `penpal` for testing various environments such as
Kusama/Polkadot versus Ethereum bridge becomes unfeasible.

This PR introduces the capability to easily customize the asset location
for `TrustedReserves` without needing to know anything about Ethereum.


## TODO
- [x] fix integration tests with
`System::set_storage(CustomizableAssetFromSystemAssetHub::key(),
<whatever-location-is-needed>)` @claravanstaden
- [ ] ~~maybe add some helper function/macro to support `set_storage`
for other runtimes (that we could reuse)~~
- [ ] Release patch for: `penpal-runtime` + emulated crate with
`set_storage` support (if needed)
  - [ ] backport to 1.7.0
  - [ ] backport to 1.8.0

---------

Co-authored-by: default avatarClara van Staden <[email protected]>
      117a9433
  41. Mar 05, 2024
    • Kian Paimani's avatar
      Repot all templates into a single directory (#3460) · 4c810609
      Kian Paimani authored 
      The first step towards
https://github.com/paritytech/polkadot-sdk/issues/3155

Brings all templates under the following structure

```
templates
|   parachain
|   |   polkadot-launch
|   |   runtime              --> parachain-template-runtime
|   |   pallets              --> pallet-parachain-template
|   |   node                 --> parachain-template-node
|   minimal
|   |   runtime              --> minimal-template-runtime
|   |   pallets              --> pallet-minimal-template
|   |   node                 --> minimal-template-node
|   solochain
|   |   runtime              --> solochain-template-runtime
|   |   pallets              --> pallet-template (the naming is not consistent here)
|   |   node                 --> solochain-template-node
```

The only note-worthy changes in this PR are: 

- More `Cargo.toml` fields are forwarded to use the one from the
workspace.
- parachain template now has weights and benchmarks
- adds a shell pallet to the minimal template
- remove a few unused deps 


A list of possible follow-ups: 

- [ ] Unify READMEs, create a parent README for all
- [ ] remove references to `docs.substrate.io` in templates
- [ ] make all templates use `#[derive_impl]`
- [ ] update and unify all licenses
- [ ] Remove polkadot launch, use
https://github.com/paritytech/polkadot-sdk/blob/35349df993ea2e7c4769914ef5d199e787b23d4c/cumulus/zombienet/examples/small_network.toml
instead.
      4c810609
    • Niklas Adolfsson's avatar
      rpc server: add prometheus label `is_rate_limited` (#3504) · efcea0ed
      Niklas Adolfsson authored 
      After some discussion with @kogeler

 after the we added the rate-limit
middleware it may slow down
the rpc call timings metrics significantly because it works as follows:

1. The rate limit guard is checked when the call comes and if a slot is
available -> process the call
2. If no free spot is available then the call will be sleeping
`jitter_delay + min_time_rate_guard` then woken up and checked at most
ten times
3. If no spot is available after 10 iterations -> the call is rejected
(this may take tens of seconds)

Thus, this PR adds a label "is_rate_limited" to filter those out on the
metrics "substrate_rpc_calls_time" and "substrate_rpc_calls_finished".

I had to merge two middleware layers Metrics and RateLimit to avoid
shared state in a hacky way.

---------

Co-authored-by: default avatarJames Wilson <[email protected]>
      efcea0ed
  43. Mar 04, 2024
  45. Mar 03, 2024
  47. Mar 01, 2024
    • Alin Dima's avatar
      provisioner: allow multiple cores assigned to the same para (#3233) · 62b78a16
      Alin Dima authored 
      https://github.com/paritytech/polkadot-sdk/issues/3130

builds on top of https://github.com/paritytech/polkadot-sdk/pull/3160

Processes the availability cores and builds a record of how many
candidates it should request from prospective-parachains and their
predecessors.
Tries to supply as many candidates as the runtime can back. Note that
the runtime changes to back multiple candidates per para are not yet
done, but this paves the way for it.

The following backing/inclusion policy is assumed:
1. the runtime will never back candidates of the same para which don't
form a chain with the already backed candidates. Even if the others are
still pending availability. We're optimistic that they won't time out
and we don't want to back parachain forks (as the complexity would be
huge).
2. if a candidate is timed out of the core before being included, all of
its successors occupying a core will be evicted.
3. only the candidates which are made available and form a chain
starting from the on-chain para head may be included/enacted and cleared
from the cores. In other words, if para head is at A and the cores are
occupied by B->C->D, and B and D are made available, only B will be
included and its core cleared. C and D will remain on the cores awaiting
for C to be made available or timed out. As point (2) above already
says, if C is timed out, D will also be dropped.
4. The runtime will deduplicate candidates which form a cycle. For
example if the provisioner supplies candidates A->B->A, the runtime will
only back A (as the state output will be the same)

Note that if a candidate is timed out, we don't guarantee that in the
next relay chain block the block author will be able to fill all of the
timed out cores of the para. That increases complexity by a lot.
Instead, the provisioner will supply N candidates where N is the number
of candidates timed out, but doesn't include their successors which will
be also deleted by the runtime. This'll be backfilled in the next relay
chain block.

Adjacent changes:
- Also fixes: https://github.com/paritytech/polkadot-sdk/issues/3141
- For non prospective-parachains, don't supply multiple candidates per
para (we can't have elastic scaling without prospective parachains
enabled). paras_inherent should already sanitise this input but it's
more efficient this way.

Note: all of these changes are backwards-compatible with the
non-elastic-scaling scenario (one core per para).
      62b78a16
    • Andrei Eres's avatar
      subsystem-bench: add regression tests for availability read and write (#3311) · f0e589d7
      Andrei Eres authored 
      ### What's been done
- `subsystem-bench` has been split into two parts: a cli benchmark
runner and a library.
- The cli runner is quite simple. It just allows us to run `.yaml` based
test sequences. Now it should only be used to run benchmarks during
development.
- The library is used in the cli runner and in regression tests. Some
code is changed to make the library independent of the runner.
- Added first regression tests for availability read and write that
replicate existing test sequences.

### How we run regression tests
- Regression tests are simply rust integration tests without the
harnesses.
- They should only be compiled under the `subsystem-benchmarks` feature
to prevent them from running with other tests.
- This doesn't work when running tests with `nextest` in CI, so
additional filters have been added to the `nextest` runs.
- Each benchmark run takes a different time in the beginning, so we
"warm up" the tests until their CPU usage differs by only 1%.
- After the warm-up, we run the benchmarks a few more times and compare
the average with the exception using a precision.

### What is still wrong?
- I haven't managed to set up approval voting tests. The spread of their
results is too large and can't be narrowed down in a reasonable amount
of time in the warm-up phase.
- The tests start an unconfigurable prometheus endpoint inside, which
causes errors because they use the same 9999 port. I disable it with a
flag, but I think it's better to extract the endpoint launching outside
the test, as we already do with `valgrind` and `pyroscope`. But we still
use `prometheus` inside the tests.

### Future work
* https://github.com/paritytech/polkadot-sdk/issues/3528
* https://github.com/paritytech/polkadot-sdk/issues/3529
* https://github.com/paritytech/polkadot-sdk/issues/3530
* https://github.com/paritytech/polkadot-sdk/issues/3531



---------

Co-authored-by: default avatarAlexander Samusev <[email protected]>
      f0e589d7
  49. Feb 29, 2024