Skip to content
  1. Jan 09, 2024
  3. Jan 07, 2024
  5. Jan 04, 2024
  7. Dec 22, 2023
  9. Dec 21, 2023
  11. Dec 14, 2023
    • Svyatoslav Nikolsky's avatar
      Add Rococo People <> Rococo Bulletin bridge support to Rococo Bridge Hub (#2540) · 097308e3
      Svyatoslav Nikolsky authored 
      This PR adds [Rococo
People](https://github.com/paritytech/polkadot-sdk/pull/2281) <> [Rococo
Bulletin](https://github.com/zdave-parity/polkadot-bulletin-chain) to
the Rococo Bridge Hub code. There's a couple of things left to do here:
- [x] add remaining tests - it'd need some refactoring in the
`bridge-hub-test-utils` - will do in a separate PR;
- [x] actually run benchmarks for new messaging pallet (do we have bot
nowadays?).

The reason why I'm opening it before this ^^^ is ready, is that I'd like
to hear others opinion on how to deal with hacks with that bridge.
Initially I was assuming that Rococo Bulletin will be the 1:1 copy of
the Polkadot Bulletin (to avoid maintaining multiple
runtimes/releases/...), so you can see many `PolkadotBulletin` mentions
in this PR, even though we are going to bridge with the parallel chain
(`RococoBulletin`). That's because e.g. pallet names from
`construct_runtime` are affecting runtime storage keys and bridges are
using runtime storage proofs => it is important to use names that the
Bulletin chain expects.

But in the end, this hack won't work - we can't use Polkadot Bulletin
runtime to bridge with Rococo Bridge Hub, because Polkadot Bulletin
expects Polkadot Bridge hub to use `1002` parachain id and Rococo Bridge
Hub seats on the `1013`. This also affects storage keys using in
bridging, so I had to add the [`rococo`
feature](https://github.com/svyatonik/polkadot-bulletin-chain/blob/add-bridge-pallets/runtime/Cargo.toml#L198)
to the Bulletin chain. So now we can actually alter its runtime and
adapt it for Rococo.

So the question here is - what's better for us here
- to leave everything as is (seems hacky and non-trivial);
- change Bulletin chain runtime when `rococo` feature is used - e.g. use
proper names there (`WithPolkadotGrandpa` -> `WithRococoGrandpa`, ...)
- add another set of pallets to the Bulletin chain runtime to bridge
with Rococo and never use them in production. Similar to hack that we
had in Rococo/Wococo

cc @acatangiu @bkontur @serban300 

also cc @joepetrowski

 as the main "client" of this bridge

---

A couple words on how this bridge is different from the Rococo <>
Westend bridge:
- it is a bridge with a chain that uses GRANDPA finality, not the
parachain finality (hence the tests needs to be changed);
- it is a fee-free bridge. So
`AllowExplicitUnpaidExecutionFrom<Equals<SiblingPeople>>` + we are not
paying any rewards to relayers (apart from compensating transaction
costs).

---------

Signed-off-by: default avatardependabot[bot] <[email protected]>
Signed-off-by: default avatarAndrei Sandu <[email protected]>
Co-authored-by: default avatarAdrian Catangiu <[email protected]>
Co-authored-by: default avatardependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: default avatarAndrei Sandu <[email protected]>
Co-authored-by: default avatarEgor_P <[email protected]>
Co-authored-by: command-bot <>
      097308e3
  13. Dec 13, 2023
  15. Dec 06, 2023
  17. Dec 01, 2023
  19. Nov 30, 2023
  21. Nov 24, 2023
  23. Nov 15, 2023
  25. Nov 08, 2023
  27. Nov 02, 2023
    • Oliver Tale-Yazdi's avatar
      Use `Message Queue` as DMP and XCMP dispatch queue (#1246) · e1c033eb
      Oliver Tale-Yazdi authored 
      (imported from https://github.com/paritytech/cumulus/pull/2157)

## Changes

This MR refactores the XCMP, Parachains System and DMP pallets to use
the [MessageQueue](https://github.com/paritytech/substrate/pull/12485)
for delayed execution of incoming messages. The DMP pallet is entirely
replaced by the MQ and thereby removed. This allows for PoV-bounded
execution and resolves a number of issues that stem from the current
work-around.

All System Parachains adopt this change.  
The most important changes are in `primitives/core/src/lib.rs`,
`parachains/common/src/process_xcm_message.rs`,
`pallets/parachain-system/src/lib.rs`, `pallets/xcmp-queue/src/lib.rs`
and the runtime configs.

### DMP Queue Pallet

The pallet got removed and its logic refactored into parachain-system.
Overweight message management can be done directly through the MQ
pallet.

Final undeployment migrations are provided by
`cumulus_pallet_dmp_queue::UndeployDmpQueue` and `DeleteDmpQueue` that
can be configured with an aux config trait like:

```rust
parameter_types! {
	pub const DmpQueuePalletName: &'static str = \"DmpQueue\" < CHANGE ME;
	pub const RelayOrigin: AggregateMessageOrigin = AggregateMessageOrigin::Parent;
}

impl cumulus_pallet_dmp_queue::MigrationConfig for Runtime {
	type PalletName = DmpQueuePalletName;
	type DmpHandler = frame_support::traits::EnqueueWithOrigin<MessageQueue, RelayOrigin>;
	type DbWeight = <Runtime as frame_system::Config>::DbWeight;
}

// And adding them to your Migrations tuple:
pub type Migrations = (
	...
	cumulus_pallet_dmp_queue::UndeployDmpQueue<Runtime>,
	cumulus_pallet_dmp_queue::DeleteDmpQueue<Runtime>,
);
```

### XCMP Queue pallet

Removed all dispatch queue functionality. Incoming XCMP messages are now
either: Immediately handled if they are Signals, enqueued into the MQ
pallet otherwise.

New config items for the XCMP queue pallet:
```rust
/// The actual queue implementation that retains the messages for later processing.
type XcmpQueue: EnqueueMessage<ParaId>;

/// How a XCM over HRMP from a sibling parachain should be processed.
type XcmpProcessor: ProcessMessage<Origin = ParaId>;

/// The maximal number of suspended XCMP channels at the same time.
#[pallet::constant]
type MaxInboundSuspended: Get<u32>;
```

How to configure those:

```rust
// Use the MessageQueue pallet to store messages for later processing. The `TransformOrigin` is needed since
// the MQ pallet itself operators on `AggregateMessageOrigin` but we want to enqueue `ParaId`s.
type XcmpQueue = TransformOrigin<MessageQueue, AggregateMessageOrigin, ParaId, ParaIdToSibling>;

// Process XCMP messages from siblings. This is type-safe to only accept `ParaId`s. They will be dispatched
// with origin `Junction::Sibling(…)`.
type XcmpProcessor = ProcessFromSibling<
	ProcessXcmMessage<
		AggregateMessageOrigin,
		xcm_executor::XcmExecutor<xcm_config::XcmConfig>,
		RuntimeCall,
	>,
>;

// Not really important what to choose here. Just something larger than the maximal number of channels.
type MaxInboundSuspended = sp_core::ConstU32<1_000>;
```

The `InboundXcmpStatus` storage item was replaced by
`InboundXcmpSuspended` since it now only tracks inbound queue suspension
and no message indices anymore.

Now only sends the most recent channel `Signals`, as all prio ones are
out-dated anyway.

### Parachain System pallet

For `DMP` messages instead of forwarding them to the `DMP` pallet, it
now pushes them to the configured `DmpQueue`. The message processing
which was triggered in `set_validation_data` is now being done by the MQ
pallet `on_initialize`.

XCMP messages are still handed off to the `XcmpMessageHandler`
(XCMP-Queue pallet) - no change here.

New config items for the parachain system pallet:
```rust
/// Queues inbound downward messages for delayed processing. 
///
/// Analogous to the `XcmpQueue` of the XCMP queue pallet.
type DmpQueue: EnqueueMessage<AggregateMessageOrigin>;
``` 

How to configure:
```rust
/// Use the MQ pallet to store DMP messages for delayed processing.
type DmpQueue = MessageQueue;
``` 

## Message Flow

The flow of messages on the parachain side. Messages come in from the
left via the `Validation Data` and finally end up at the `Xcm Executor`
on the right.

![Untitled
(1)](https://github.com/paritytech/cumulus/assets/10380170/6cf8b377-88c9-4aed-96df-baace266e04d)

## Further changes

- Bumped the default suspension, drop and resume thresholds in
`QueueConfigData::default()`.
- `XcmpQueue::{suspend_xcm_execution, resume_xcm_execution}` errors when
they would be a noop.
- Properly validate the `QueueConfigData` before setting it.
- Marked weight files as auto-generated so they wont auto-expand in the
MR files view.
- Move the `hypothetical` asserts to `frame_support` under the name
`experimental_hypothetically`

Questions:
- [ ] What about the ugly `#[cfg(feature = \"runtime-benchmarks\")]` in
the runtimes? Not sure how to best fix. Just having them like this makes
tests fail that rely on the real message processor when the feature is
enabled.
- [ ] Need a good weight for `MessageQueueServiceWeight`. The scheduler
already takes 80% so I put it to 10% but that is quite low.

TODO:
- [x] Remove c&p code after
https://github.com/paritytech/polkadot/pull/6271
- [x] Use `HandleMessage` once it is public in Substrate
- [x] fix `runtime-benchmarks` feature
https://github.com/paritytech/polkadot/pull/6966


- [x] Benchmarks
- [x] Tests
- [ ] Migrate `InboundXcmpStatus` to `InboundXcmpSuspended`
- [x] Possibly cleanup Migrations (DMP+XCMP)
- [x] optional: create `TransformProcessMessageOrigin` in Substrate and
replace `ProcessFromSibling`
- [ ] Rerun weights on ref HW

---------

Signed-off-by: default avatarOliver Tale-Yazdi <[email protected]>
Co-authored-by: default avatarLiam Aharon <[email protected]>
Co-authored-by: default avatarjoe petrowski <[email protected]>
Co-authored-by: default avatarKian Paimani <[email protected]>
Co-authored-by: command-bot <>
      e1c033eb
  29. Nov 01, 2023
  31. Oct 31, 2023
  33. Oct 27, 2023
  35. Oct 20, 2023
    • Bastian Köcher's avatar
      `xcm`: Change `TypeInfo::path` to not include `staging` (#1948) · f3bf5c1a
      Bastian Köcher authored 
      The `xcm` crate was renamed to `staging-xcm` to be able to publish it to
crates.io as someone as squatted `xcm`. The problem with this rename is
that the `TypeInfo` includes the crate name which ultimately lands in
the metadata. The metadata is consumed by downstream users like
`polkadot-js` or people building on top of `polkadot-js`. These people
are using the entire `path` to find the type in the type registry. Thus,
their code would break as the type path would now be [`staging_xcm`,
`VersionedXcm`] instead of [`xcm`, `VersionedXcm`]. This pull request
fixes this by renaming the path segment `staging_xcm` to `xcm`.

This requires: https://github.com/paritytech/scale-info/pull/197



---------

Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
      f3bf5c1a
  37. Oct 18, 2023
  39. Sep 27, 2023
  41. Sep 14, 2023
  43. Sep 11, 2023
    • Serban Iorga's avatar
      Update bridges subtree (#1392) · 142a11ad
      Serban Iorga authored 
      * Move the bridges subtree under root

* Squashed 'bridges/' changes from 277f0d5496..e50398d1c5

e50398d1c5 bridges subtree fixes (#2528)
99af07522d Markdown linter (#1309) (#2526)
733ff0fe7a `polkadot-staging` branch: Use polkadot-sdk dependencies (#2524)
e8a59f141e Fix benchmark with new XCM::V3 `MAX_INSTRUCTIONS_TO_DECODE` (#2514)
62b185de15 Backport `polkadot-sdk` changes to `polkadot-staging` (#2518)
d9658f4d5b Fix equivocation detection containers startup (#2516) (#2517)
d65db28a8f Backport: building images from locally built binaries (#2513)
5fdbaf45f6 Start the equivocation detection loop from the complex relayer (#2507) (#2512)
7fbb67de46 Backport: Implement basic equivocations detection loop (#2375)
cb7efe245c Manually update deps in polkadot staging (#2371)
d17981fc33 #2351 to polkadot-staging (#2359)

git-subtree-dir: bridges
git-subtree-split: e50398d1c594e4e96df70b0bd376e565d17e8558

* Reapply diener workspacify

* Fix Cargo.toml

* Fix test

* Adjustments
      142a11ad
  45. Aug 31, 2023
    • Bastian Köcher's avatar
      Rename `polkadot-parachain` to `polkadot-parachain-primitives` (#1334) · a33d7922
      Bastian Köcher authored 
      * Rename `polkadot-parachain` to `polkadot-parachain-primitives`

While doing this it also fixes some last `rustdoc` issues and fixes
another Cargo warning related to `pallet-paged-list`.

* Fix compilation

* ".git/.scripts/commands/fmt/fmt.sh"

* Fix XCM docs

---------

Co-authored-by: command-bot <>
      a33d7922
  47. Aug 30, 2023
  49. Aug 29, 2023
  51. Aug 28, 2023
  53. Aug 25, 2023
  55. Aug 24, 2023
  57. Aug 23, 2023
    • juangirini's avatar
      Companion: restructure macro related exports (#3015) · 8349c8d1
      juangirini authored 
      * restructure macro related exports

* restructure macro related exports

* wip

* wip

* update cargo lock

* refactor RuntimeDebug on unincluded segment

* fmt

* Companion: restructure `benchmarking` macro related exports (#3039)

* wip

* wip

* restructure benchmarking macro related exports

* add cargo lock

---------

Co-authored-by: parity-processbot <>
      8349c8d1
  59. Aug 18, 2023
    • Chris Sosnin's avatar
      Asynchronous backing PR (#2300) · 6ef11176
      Chris Sosnin authored 
      

* Update substrate & polkadot

* min changes to make async backing compile

* (async backing) parachain-system: track limitations for unincluded blocks (#2438)

* unincluded segment draft

* read para head from storage proof

* read_para_head -> read_included_para_head

* Provide pub interface

* add errors

* fix unincluded segment update

* BlockTracker -> Ancestor

* add a dmp limit

* Read para head depending on the storage switch

* doc comments

* storage items docs

* add a sanity check on block initialize

* Check watermark

* append to the segment on block finalize

* Move segment update into set_validation_data

* Resolve para head todo

* option watermark

* fix comment

* Drop dmq check

* fix weight

* doc-comments on inherent invariant

* Remove TODO

* add todo

* primitives tests

* pallet tests

* doc comments

* refactor unincluded segment length into a ConsensusHook (#2501)

* refactor unincluded segment length into a ConsensusHook

* add docs

* refactor bandwidth_out calculation

Co-authored-by: default avatarChris Sosnin <[email protected]>

* test for limits from impl

* fmt

* make tests compile

* update comment

* uncomment test

* fix collator test by adding parent to state proof

* patch HRMP watermark rules for unincluded segment

* get consensus-common tests to pass, using unincluded segment

* fix unincluded segment tests

* get all tests passing

* fmt

* rustdoc CI

* aura-ext: limit the number of authored blocks per slot (#2551)

* aura_ext consensus hook

* reverse dependency

* include weight into hook

* fix tests

* remove stray println

Co-authored-by: default avatarChris Sosnin <[email protected]>

* fix test warning

* fix doc link

---------

Co-authored-by: default avatarChris Sosnin <[email protected]>
Co-authored-by: default avatarChris Sosnin <[email protected]>

* parachain-system: ignore go ahead signal once upgrade is processed (#2594)

* handle goahead signal for unincluded segment

* doc comment

* add test

* parachain-system: drop processed messages from inherent data (#2590)

* implement `drop_processed_messages`

* drop messages based on relay parent number

* adjust tests

* drop changes to mqc

* fix comment

* drop test

* drop more dead code

* clippy

* aura-ext: check slot in consensus hook and remove all `CheckInherents` logic (#2658)

* aura-ext: check slot in consensus hook

* convert relay chain slot

* Make relay chain slot duration generic

* use fixed velocity hook for pallets with aura

* purge timestamp inherent

* fix warning

* adjust runtime tests

* fix slots in tests

* Make `xcm-emulator` test pass for new consensus hook (#2722)

* add pallets on_initialize

* tests pass

* add AuraExt on_init

* ".git/.scripts/commands/fmt/fmt.sh"

---------

Co-authored-by: command-bot <>

---------

Co-authored-by: default avatarIgnacio Palacios <[email protected]>

* update polkadot git refs

* CollationGenerationConfig closure is now optional (#2772)

* CollationGenerationConfig closure is now optional

* fix test

* propagate network-protocol-staging feature (#2899)

* Feature Flagging Consensus Hook Type Parameter (#2911)

* First pass

* fmt

* Added as default feature in tomls

* Changed to direct dependency feature

* Dealing with clippy error

* Update pallets/parachain-system/src/lib.rs

Co-authored-by: default avatarasynchronous rob <[email protected]>

---------

Co-authored-by: default avatarasynchronous rob <[email protected]>

* fmt

* bump deps and remove warning

* parachain-system: update RelevantMessagingState according to the unincluded segment (#2948)

* mostly address 2471 with a bug introduced

* adjust relevant messaging state after computing total

* fmt

* max -> min

* fix test implementation of xcmp source

* add test

* fix test message sending logic

* fix + test

* add more to unincluded segment test

* fmt

---------

Co-authored-by: default avatarChris Sosnin <[email protected]>

* Integrate new Aura / Parachain Consensus Logic in Parachain-Template / Polkadot-Parachain (#2864)

* add a comment

* refactor client/service utilities

* deprecate start_collator

* update parachain-template

* update test-service in the same way

* update polkadot-parachain crate

* fmt

* wire up new SubmitCollation message

* some runtime utilities for implementing unincluded segment runtime APIs

* allow parachains to configure their level of sybil-resistance when starting the network

* make aura-ext compile

* update to specify sybil resistance levels

* fmt

* specify relay chain slot duration in milliseconds

* update Aura to explicitly produce Send futures

also, make relay_chain_slot_duration a Duration

* add authoring duration to basic collator and document params

* integrate new basic collator into parachain-template

* remove assert_send used for testing

* basic-aura: only author when parent included

* update polkadot-parachain-bin

* fmt

* some fixes

* fixes

* add a RelayNumberMonotonicallyIncreases

* add a utility function for initializing subsystems

* some logging for timestamp adjustment

* fmt

* some fixes for lookahead collator

* add a log

* update `find_potential_parents` to account for sessions

* bound the loop

* restore & deprecate old start_collator and start_full_node functions.

* remove unnecessary await calls

* fix warning

* clippy

* more clippy

* remove unneeded logic

* ci

* update comment

Co-authored-by: default avatarMarcin S. <[email protected]>

* (async backing) restore `CheckInherents` for backwards-compatibility (#2977)

* bring back timestamp

* Restore CheckInherents

* revert to empty CheckInherents

* make CheckInherents optional

* attempt

* properly end system blocks

* add some more comments

* ignore failing system parachain tests

* update refs after main feature branch merge

* comment out the offending tests because CI runs ignored tests

* fix warnings

* fmt

* revert to polkadot master

* cargo update -p polkadot-primitives -p sp-io

---------

Co-authored-by: default avatarasynchronous rob <[email protected]>
Co-authored-by: default avatarIgnacio Palacios <[email protected]>
Co-authored-by: default avatarBradley Olson <[email protected]>
Co-authored-by: default avatarMarcin S. <[email protected]>
Co-authored-by: default avatareskimor <[email protected]>
Co-authored-by: default avatarAndronik <[email protected]>
      6ef11176
  61. Aug 15, 2023
  63. Aug 10, 2023
    • Branislav Kontur's avatar
      Bridges subtree sync (#2991) · e077bdd9
      Branislav Kontur authored 
      * Squashed 'bridges/' changes from 0417308a48..278119fec2

278119fec2 Grandpa: Store the authority set changes (#2336) (#2337)
19f9c8ffdb remove message sender origin (#2322)
3c7c271d2e GRANDPA module: store accepted justifications (#2298) (#2301)
fb7d12e793 GRANDPA justifications: equivocation detection primitives (#2295) (#2297)
d03a2ed450 More backports from Cumulus subtree to polkadot-staging (#2283)
3c4ada921b Update dependecies (#2277) (#2281)
3e195c9e76 GRANDPA: optimize votes_ancestries when needed (#2262) (#2264)
7065bbabc6 Implement RuntimeDebug for GrandpaJustification (#2254)
8c9e59bcbc Define generate_grandpa_key_ownership_proof() (#2247) (#2248)
0b46956df7 Deduplicate Grandpa consensus log reading logic (#2245) (#2246)
96c9701710 Fix deps from Cumulus (#2244)

git-subtree-dir: bridges
git-subtree-split: 278119fec2b45990cf1271999b0c21befe7003d9

* Subtree update

* Squashed 'bridges/' changes from 278119fec2..edf33a2c85

edf33a2c85 Backport fix (for wasm `std` env) (#2339)

git-subtree-dir: bridges
git-subtree-split: edf33a2c85399d366e008228f2d9e63e8a492d95
      e077bdd9
  65. Aug 08, 2023
  67. Jul 27, 2023
  69. Jul 25, 2023
  71. Jul 11, 2023
  73. Jul 10, 2023
  75. Jul 08, 2023