Skip to content
  1. Mar 22, 2024
    • girazoki's avatar
      [pallet-xcm] fix transport fees for remote reserve transfers (#3792) · 9a04ebbf
      girazoki authored 
      Currently `transfer_assets` from pallet-xcm covers 4 main different
transfer types:
- `localReserve`
- `DestinationReserve`
- `Teleport`
- `RemoteReserve`

For the first three, the local execution and the remote message sending
are separated, and fees are deducted in pallet-xcm itself:
https://github.com/paritytech/polkadot-sdk/blob/3410dfb3929462da88be2da813f121d8b1cf46b3/polkadot/xcm/pallet-xcm/src/lib.rs#L1758

.

For the 4th case `RemoteReserve`, pallet-xcm is still relying on the
xcm-executor itself to send the message (through the
`initiateReserveWithdraw` instruction). In this case, if delivery fees
need to be charged, it is not possible to do so because the
`jit_withdraw` mode has not being set.

This PR proposes to still use the `initiateReserveWithdraw` but
prepending a `setFeesMode { jit_withdraw: true }` to make sure delivery
fees can be paid.

A test-case is also added to present the aforementioned case

---------

Co-authored-by: default avatarAdrian Catangiu <[email protected]>
      9a04ebbf
    • PG Herveou's avatar
      XCM remove extra QueryId types from traits (#3763) · 2f59e9ef
      PG Herveou authored 
      We do not need to make these traits generic over QueryId type, we can
just use the QueryId alias everywhere
      2f59e9ef
    • Dmitry Markin's avatar
      Make public addresses go first in authority discovery DHT records (#3757) · 9d2963c2
      Dmitry Markin authored 
      Make sure explicitly set by the operator public addresses go first in
the authority discovery DHT records.

Also update `Discovery` behavior to eliminate duplicates in the returned
addresses.

This PR should improve situation with
https://github.com/paritytech/polkadot-sdk/issues/3519.

Obsoletes https://github.com/paritytech/polkadot-sdk/pull/3657.
      9d2963c2
    • Will | Paradox | ParaNodes.io's avatar
      Adding LF's bootnodes to relay and system chains (#3514) · ea5f4e9a
      Will | Paradox | ParaNodes.io authored 
      

Good day,

I'm seeking to add the following bootnodes for Kusama and Polkadot's
relay and system chains. The following commands can be used to test
connectivity. All node keys are backed up.

Polkadot:
```
polkadot --chain polkadot --base-path /tmp/node --name "Boot" --reserved-only --reserved-nodes "/dns/boot-polkadot.luckyfriday.io/tcp/443/wss/p2p/12D3KooWAdyiVAaeGdtBt6vn5zVetwA4z4qfm9Fi2QCSykN1wTBJ" --no-hardware-benchmarks
```


Assethub-Polkadot:

```
polkadot-parachain --chain asset-hub-polkadot --base-path /tmp/node --name "Boot" --reserved-only --reserved-nodes "/dns/boot-polkadot-assethub.luckyfriday.io/tcp/443/wss/p2p/12D3KooWDR9M7CjV1xdjCRbRwkFn1E7sjMaL4oYxGyDWxuLrFc2J" --no-hardware-benchmarks

```

Bridgehub-Polkadot:

```
polkadot-parachain --chain bridge-hub-polkadot --base-path /tmp/node --name "Boot" --reserved-only --reserved-nodes "/dns/boot-polkadot-bridgehub.luckyfriday.io/tcp/443/wss/p2p/12D3KooWKf3mBXHjLbwtPqv1BdbQuwbFNcQQYxASS7iQ25264AXH" --no-hardware-benchmarks

```
Collectives-Polkadot

```
polkadot-parachain --chain collectives-polkadot --base-path /tmp/node --name "Boot" --reserved-only --reserved-nodes "/dns/boot-polkadot-collectives.luckyfriday.io/tcp/443/wss/p2p/12D3KooWCzifnPooTt4kvTnXT7FTKTymVL7xn7DURQLsS2AKpf6w" --no-hardware-benchmarks

```
Kusama:

```
polkadot --chain kusama --base-path /tmp/node --name "Boot" --reserved-only --reserved-nodes "/dns/boot-kusama.luckyfriday.io/tcp/443/wss/p2p/12D3KooWS1Lu6DmK8YHSvkErpxpcXmk14vG6y4KVEFEkd9g62PP8" --no-hardware-benchmarks

```
Assethub-Kusama:

```
polkadot-parachain --chain asset-hub-kusama --base-path /tmp/node --name "Boot" --reserved-only --reserved-nodes "/dns/boot-kusama-assethub.luckyfriday.io/tcp/443/wss/p2p/12D3KooWSwaeFs6FNgpgh54fdoxSDAA4nJNaPE3PAcse2GRrG7b3" --no-hardware-benchmarks
```

Bridgehub-Kusama:

```
polkadot-parachain --chain bridge-hub-kusama --base-path /tmp/node --name "Boot" --reserved-only --reserved-nodes "/dns/boot-kusama-bridgehub.luckyfriday.io/tcp/443/wss/p2p/12D3KooWQybw6AFmAvrFfwUQnNxUpS12RovapD6oorh2mAJr4xyd" --no-hardware-benchmarks
```

Co-authored-by: default avatarBastian Köcher <[email protected]>
      ea5f4e9a
  3. Mar 21, 2024
  5. Mar 20, 2024
    • eskimor's avatar
      Fix algorithmic complexity of on-demand scheduler with regards to number of cores. (#3190) · b74353d3
      eskimor authored 
      

We witnessed really poor performance on Rococo, where we ended up with
50 on-demand cores. This was due to the fact that for each core the full
queue was processed. With this change full queue processing will happen
way less often (most of the time complexity is O(1) or O(log(n))) and if
it happens then only for one core (in expectation).

Also spot price is now updated before each order to ensure economic back
pressure.


TODO:

- [x] Implement
- [x] Basic tests
- [x] Add more tests (see todos)
- [x] Run benchmark to confirm better performance, first results suggest
> 100x faster.
- [x] Write migrations
- [x] Bump scale-info version and remove patch in Cargo.toml
- [x] Write PR docs: on-demand performance improved, more on-demand
cores are now non problematic anymore. If need by also the max queue
size can be increased again. (Maybe not to 10k)

Optional: Performance can be improved even more, if we called
`pop_assignment_for_core()`, before calling `report_processed` (Avoid
needless affinity drops). The effect gets smaller the larger the claim
queue and I would only go for it, if it does not add complexity to the
scheduler.

---------

Co-authored-by: default avatareskimor <[email protected]>
Co-authored-by: default avatarantonva <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: default avatarAnton Vilhelm Ásgeirsson <[email protected]>
Co-authored-by: default avatarordian <[email protected]>
      b74353d3
    • Tsvetomir Dimitrov's avatar
      Expose `ClaimQueue` via a runtime api and use it in `collation-generation` (#3580) · e58e854a
      Tsvetomir Dimitrov authored 
      The PR adds two things:
1. Runtime API exposing the whole claim queue
2. Consumes the API in `collation-generation` to fetch the next
scheduled `ParaEntry` for an occupied core.

Related to https://github.com/paritytech/polkadot-sdk/issues/1797
      e58e854a
  7. Mar 19, 2024
  9. Mar 18, 2024
  11. Mar 17, 2024
  13. Mar 15, 2024
  15. Mar 14, 2024
  17. Mar 13, 2024
  19. Mar 12, 2024
    • Alexandru Gheorghe's avatar
      Add api-name in `cannot query the runtime API version` warning (#3653) · 1ead5977
      Alexandru Gheorghe authored 
      

Sometimes we see nodes printing this warning:
```
cannot query the runtime API version: Api called for an unknown Block: State already discarded for
```

The log is harmless, but let's print the api we got this for, so that we
can track its call site and truly confirm it is harmless or fix it.

Signed-off-by: default avatarAlexandru Gheorghe <[email protected]>
      1ead5977
    • Koute's avatar
      Add a PolkaVM-based executor (#3458) · b0f34e4b
      Koute authored 
      This PR adds a new PolkaVM-based executor to Substrate.

- The executor can now be used to actually run a PolkaVM-based runtime,
and successfully produces blocks.
- The executor is always compiled-in, but is disabled by default.
- The `SUBSTRATE_ENABLE_POLKAVM` environment variable must be set to `1`
to enable the executor, in which case the node will accept both WASM and
PolkaVM program blobs (otherwise it'll default to WASM-only). This is
deliberately undocumented and not explicitly exposed anywhere (e.g. in
the command line arguments, or in the API) to disincentivize anyone from
enabling it in production. If/when we'll move this into production usage
I'll remove the environment variable and do it "properly".
- I did not use our legacy runtime allocator for the PolkaVM executor,
so currently every allocation inside of the runtime will leak guest
memory until that particular instance is destroyed. The idea here is
that I will work on the https://github.com/polkadot-fellows/RFCs/pull/4
which will remove the need for the legacy allocator under WASM, and that
will also allow us to use a proper non-leaking allocator under PolkaVM.
- I also did some minor cleanups of the WASM executor and deleted some
dead code.

No prdocs included since this is not intended to be an end-user feature,
but an unofficial experiment, and shouldn't affect any current
production user. Once this is production-ready a full Polkadot
Fellowship RFC will be necessary anyway.
      b0f34e4b
  21. Mar 11, 2024
  23. Mar 08, 2024
  25. Mar 07, 2024
  27. Mar 05, 2024
  29. Mar 04, 2024
    • Gavin Wood's avatar
      FRAME: Create `TransactionExtension` as a replacement for `SignedExtension` (#2280) · fd5f9292
      Gavin Wood authored 
      Closes #2160

First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415

)

Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for the latter. (Ultimately, the latter will be phased out as
a type of transaction, and Bare will only be used for Inherents.)

Types of extrinsic are now therefore:
- Bare (no hardcoded signature, no Extra data; used to be known as
"Unsigned")
- Bare transactions (deprecated): Gossiped, validated with
`ValidateUnsigned` (deprecated) and the `_bare_compat` bits of
`TransactionExtension` (deprecated).
  - Inherents: Not gossiped, validated with `ProvideInherent`.
- Extended (Extra data): Gossiped, validated via `TransactionExtension`.
  - Signed transactions (with a hardcoded signature).
  - General transactions (without a hardcoded signature).

`TransactionExtension` differs from `SignedExtension` because:
- A signature on the underlying transaction may validly not be present.
- It may alter the origin during validation.
- `pre_dispatch` is renamed to `prepare` and need not contain the checks
present in `validate`.
- `validate` and `prepare` is passed an `Origin` rather than a
`AccountId`.
- `validate` may pass arbitrary information into `prepare` via a new
user-specifiable type `Val`.
- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`. It is encoded *for the entire transaction* and
passed in to each extension as a new argument to `validate`. This
facilitates the ability of extensions to acts as underlying crypto.

There is a new `DispatchTransaction` trait which contains only default
function impls and is impl'ed for any `TransactionExtension` impler. It
provides several utility functions which reduce some of the tedium from
using `TransactionExtension` (indeed, none of its regular functions
should now need to be called directly).

Three transaction version discriminator ("versions") are now
permissible:
- 0b000000100: Bare (used to be called "Unsigned"): contains Signature
or Extra (extension data). After bare transactions are no longer
supported, this will strictly identify an Inherents only.
- 0b100000100: Old-school "Signed" Transaction: contains Signature and
Extra (extension data).
- 0b010000100: New-school "General" Transaction: contains Extra
(extension data), but no Signature.

For the New-school General Transaction, it becomes trivial for authors
to publish extensions to the mechanism for authorizing an Origin, e.g.
through new kinds of key-signing schemes, ZK proofs, pallet state,
mutations over pre-authenticated origins or any combination of the
above.

## Code Migration

### NOW: Getting it to build

Wrap your `SignedExtension`s in `AsTransactionExtension`. This should be
accompanied by renaming your aggregate type in line with the new
terminology. E.g. Before:

```rust
/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
	/* snip */
	MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
```

After:

```rust
/// The extension to the basic transaction logic.
pub type TxExtension = (
	/* snip */
	AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;
```

You'll also need to alter any transaction building logic to add a
`.into()` to make the conversion happen. E.g. Before:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let extra: SignedExtra = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */),
	);
	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		extra,
	)
}
```

After:

```rust
fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let tx_ext: TxExtension = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */).into(),
	);
	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		tx_ext,
	)
}
```

### SOON: Migrating to `TransactionExtension`

Most `SignedExtension`s can be trivially converted to become a
`TransactionExtension`. There are a few things to know.

- Instead of a single trait like `SignedExtension`, you should now
implement two traits individually: `TransactionExtensionBase` and
`TransactionExtension`.
- Weights are now a thing and must be provided via the new function `fn
weight`.

#### `TransactionExtensionBase`

This trait takes care of anything which is not dependent on types
specific to your runtime, most notably `Call`.

- `AdditionalSigned`/`additional_signed` is renamed to
`Implicit`/`implicit`.
- Weight must be returned by implementing the `weight` function. If your
extension is associated with a pallet, you'll probably want to do this
via the pallet's existing benchmarking infrastructure.

#### `TransactionExtension`

Generally:
- `pre_dispatch` is now `prepare` and you *should not reexecute the
`validate` functionality in there*!
- You don't get an account ID any more; you get an origin instead. If
you need to presume an account ID, then you can use the trait function
`AsSystemOriginSigner::as_system_origin_signer`.
- You get an additional ticket, similar to `Pre`, called `Val`. This
defines data which is passed from `validate` into `prepare`. This is
important since you should not be duplicating logic from `validate` to
`prepare`, you need a way of passing your working from the former into
the latter. This is it.
- This trait takes two type parameters: `Call` and `Context`. `Call` is
the runtime call type which used to be an associated type; you can just
move it to become a type parameter for your trait impl. `Context` is not
currently used and you can safely implement over it as an unbounded
type.
- There's no `AccountId` associated type any more. Just remove it.

Regarding `validate`:
- You get three new parameters in `validate`; all can be ignored when
migrating from `SignedExtension`.
- `validate` returns a tuple on success; the second item in the tuple is
the new ticket type `Self::Val` which gets passed in to `prepare`. If
you use any information extracted during `validate` (off-chain and
on-chain, non-mutating) in `prepare` (on-chain, mutating) then you can
pass it through with this. For the tuple's last item, just return the
`origin` argument.

Regarding `prepare`:
- This is renamed from `pre_dispatch`, but there is one change:
- FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM
`validate`!!
- (This is different to `SignedExtension` which was required to run the
same checks in `pre_dispatch` as in `validate`.)

Regarding `post_dispatch`:
- Since there are no unsigned transactions handled by
`TransactionExtension`, `Pre` is always defined, so the first parameter
is `Self::Pre` rather than `Option<Self::Pre>`.

If you make use of `SignedExtension::validate_unsigned` or
`SignedExtension::pre_dispatch_unsigned`, then:
- Just use the regular versions of these functions instead.
- Have your logic execute in the case that the `origin` is `None`.
- Ensure your transaction creation logic creates a General Transaction
rather than a Bare Transaction; this means having to include all
`TransactionExtension`s' data.
- `ValidateUnsigned` can still be used (for now) if you need to be able
to construct transactions which contain none of the extension data,
however these will be phased out in stage 2 of the Transactions Horizon,
so you should consider moving to an extension-centric design.

## TODO

- [x] Introduce `CheckSignature` impl of `TransactionExtension` to
ensure it's possible to have crypto be done wholly in a
`TransactionExtension`.
- [x] Deprecate `SignedExtension` and move all uses in codebase to
`TransactionExtension`.
  - [x] `ChargeTransactionPayment`
  - [x] `DummyExtension`
  - [x] `ChargeAssetTxPayment` (asset-tx-payment)
  - [x] `ChargeAssetTxPayment` (asset-conversion-tx-payment)
  - [x] `CheckWeight`
  - [x] `CheckTxVersion`
  - [x] `CheckSpecVersion`
  - [x] `CheckNonce`
  - [x] `CheckNonZeroSender`
  - [x] `CheckMortality`
  - [x] `CheckGenesis`
  - [x] `CheckOnlySudoAccount`
  - [x] `WatchDummy`
  - [x] `PrevalidateAttests`
  - [x] `GenericSignedExtension`
  - [x] `SignedExtension` (chain-polkadot-bulletin)
  - [x] `RefundSignedExtensionAdapter`
- [x] Implement `fn weight` across the board.
- [ ] Go through all pre-existing extensions which assume an account
signer and explicitly handle the possibility of another kind of origin.
- [x] `CheckNonce` should probably succeed in the case of a non-account
origin.
- [x] `CheckNonZeroSender` should succeed in the case of a non-account
origin.
- [x] `ChargeTransactionPayment` and family should fail in the case of a
non-account origin.
  - [ ] 
- [x] Fix any broken tests.

---------

Signed-off-by: default avatargeorgepisaltu <[email protected]>
Signed-off-by: default avatarAlexandru Vasile <[email protected]>
Signed-off-by: default avatardependabot[bot] <[email protected]>
Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
Signed-off-by: default avatarAlexandru Gheorghe <[email protected]>
Signed-off-by: default avatarAndrei Sandu <[email protected]>
Co-authored-by: default avatarNikhil Gupta <[email protected]>
Co-authored-by: default avatargeorgepisaltu <[email protected]>
Co-authored-by: default avatarChevdor <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
Co-authored-by: default avatarMaciej <[email protected]>
Co-authored-by: default avatarJavier Viola <[email protected]>
Co-authored-by: default avatarMarcin S. <[email protected]>
Co-authored-by: default avatarTsvetomir Dimitrov <[email protected]>
Co-authored-by: default avatarJavier Bullrich <[email protected]>
Co-authored-by: default avatarKoute <[email protected]>
Co-authored-by: default avatarAdrian Catangiu <[email protected]>
Co-authored-by: Vladimir Istyufeev's avatarVladimir Istyufeev <[email protected]>
Co-authored-by: default avatarRoss Bulat <[email protected]>
Co-authored-by: default avatarGonçalo Pestana <[email protected]>
Co-authored-by: default avatarLiam Aharon <[email protected]>
Co-authored-by: default avatarSvyatoslav Nikolsky <[email protected]>
Co-authored-by: default avatarAndré Silva <[email protected]>
Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatars0me0ne-unkn0wn <[email protected]>
Co-authored-by: default avatarordian <[email protected]>
Co-authored-by: default avatarSebastian Kunert <[email protected]>
Co-authored-by: default avatarAaro Altonen <[email protected]>
Co-authored-by: default avatarDmitry Markin <[email protected]>
Co-authored-by: default avatarAlexandru Vasile <[email protected]>
Co-authored-by: default avatarAlexander Samusev <[email protected]>
Co-authored-by: default avatarJulian Eager <[email protected]>
Co-authored-by: default avatarMichal Kucharczyk <[email protected]>
Co-authored-by: default avatarDavide Galassi <[email protected]>
Co-authored-by: default avatarDónal Murray <[email protected]>
Co-authored-by: default avataryjh <[email protected]>
Co-authored-by: default avatarTom Mi <[email protected]>
Co-authored-by: default avatardependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: default avatarWill | Paradox | ParaNodes.io <[email protected]>
Co-authored-by: default avatarBastian Köcher <[email protected]>
Co-authored-by: default avatarJoshy Orndorff <[email protected]>
Co-authored-by: default avatarJoshy Orndorff <[email protected]>
Co-authored-by: default avatarPG Herveou <[email protected]>
Co-authored-by: default avatarAlexander Theißen <[email protected]>
Co-authored-by: default avatarKian Paimani <[email protected]>
Co-authored-by: default avatarJuan Girini <[email protected]>
Co-authored-by: default avatarbader y <[email protected]>
Co-authored-by: default avatarJames Wilson <[email protected]>
Co-authored-by: default avatarjoe petrowski <[email protected]>
Co-authored-by: default avatarasynchronous rob <[email protected]>
Co-authored-by: default avatarParth <[email protected]>
Co-authored-by: default avatarAndrew Jones <[email protected]>
Co-authored-by: default avatarJonathan Udd <[email protected]>
Co-authored-by: default avatarSerban Iorga <[email protected]>
Co-authored-by: default avatarEgor_P <[email protected]>
Co-authored-by: default avatarBranislav Kontur <[email protected]>
Co-authored-by: default avatarEvgeny Snitko <[email protected]>
Co-authored-by: default avatarJust van Stam <[email protected]>
Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
Co-authored-by: default avatargupnik <[email protected]>
Co-authored-by: default avatardzmitry-lahoda <[email protected]>
Co-authored-by: default avatarzhiqiangxu <[email protected]>
Co-authored-by: default avatarNazar Mokrynskyi <[email protected]>
Co-authored-by: default avatarAnwesh <[email protected]>
Co-authored-by: default avatarcheme <[email protected]>
Co-authored-by: default avatarSam Johnson <[email protected]>
Co-authored-by: default avatarkianenigma <[email protected]>
Co-authored-by: default avatarJegor Sidorenko <[email protected]>
Co-authored-by: default avatarMuharem <[email protected]>
Co-authored-by: default avatarjoepetrowski <[email protected]>
Co-authored-by: default avatarAlexandru Gheorghe <[email protected]>
Co-authored-by: default avatarGabriel Facco de Arruda <[email protected]>
Co-authored-by: default avatarSquirrel <[email protected]>
Co-authored-by: default avatarAndrei Sandu <[email protected]>
Co-authored-by: default avatargeorgepisaltu <[email protected]>
Co-authored-by: command-bot <>
      fd5f9292