Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

## Multi Block Election Pallet

This PR adds the first iteration of the multi-block staking pallet. 

From this point onwards, the staking and its election provider pallets
are being customized to work in AssetHub. While usage in solo-chains is
still possible, it is not longer the main focus of this pallet. For a
safer usage, please fork and user an older version of this pallet.

---

## Replaces

- [x] https://github.com/paritytech/polkadot-sdk/pull/6034 
- [x] https://github.com/paritytech/polkadot-sdk/pull/5272

## Related PRs: 

- [x] https://github.com/paritytech/polkadot-sdk/pull/7483
- [ ] https://github.com/paritytech/polkadot-sdk/pull/7357
- [ ] https://github.com/paritytech/polkadot-sdk/pull/7424
- [ ] https://github.com/paritytech/polkadot-staking-miner/pull/955

This branch can be periodically merged into
https://github.com/paritytech/polkadot-sdk/pull/7358 ->
https://github.com/paritytech/polkadot-sdk/pull/6996

## TODOs: 

- [x] rebase to master 
- Benchmarking for staking critical path
  - [x] snapshot
  - [x] election result
- Benchmarking for EPMB critical path
  - [x] snapshot
  - [x] verification
  - [x] submission
  - [x] unsigned submission
  - [ ] election results fetching
- [ ] Fix deletion weights. Either of
  - [ ] Garbage collector + lazy removal of all paged storage items
  - [ ] Confirm that deletion is small PoV footprint.
- [ ] Move election prediction to be push based. @tdimitrov 
- [ ] integrity checks for bounds 
- [ ] Properly benchmark this as a part of CI -- for now I will remove
them as they are too slow
- [x] add try-state to all pallets
- [x] Staking to allow genesis dev accounts to be created internally
- [x] Decouple miner config so @niklasad1 can work on the miner
72841b73
- [x] duplicate snapshot page reported by @niklasad1

 
- [ ] https://github.com/paritytech/polkadot-sdk/pull/6520 or equivalent
-- during snapshot, `VoterList` must be locked
- [ ] Move target snapshot to a separate block

---------

Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Changes:
- Expose a `force_set_head` function from the `MessageQueue` pallet via
a new trait: `ForceSetHead`. This can be used to force the MQ pallet to
process this queue next.
- The change only exposes an internal function through a trait, no audit
is required.

## Context

For the Asset Hub Migration (AHM) we need a mechanism to prioritize the
inbound upward messages and the inbound downward messages on the AH. To
achieve this, a minimal (and no breaking) change is done to the MQ
pallet in the form of adding the `force_set_head` function.

An example use of how to achieve prioritization is then demonstrated in
`integration_test.rs::AhmPrioritizer`. Normally, all queues are
scheduled round-robin like this:

`| Relay | Para(1) | Para(2) | ... | Relay | ... `

The prioritizer listens to changes to its queue and triggers if either:
- The queue processed in the last block (to keep the general round-robin
scheduling)
- The queue did not process since `n` blocks (to prevent starvation if
there are too many other queues)

In either situation, it schedules the queue for a streak of three
consecutive blocks, such that it would become:

`| Relay | Relay | Relay | Para(1) | Para(2) | ... | Relay | Relay |
Relay | ... `

It basically transforms the round-robin into an elongated round robin.
Although different strategies can be injected into the pallet at
runtime, this one seems to strike a good balance between general service
level and prioritization.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: muharem <ismailov.m.h@gmail.com>

Implementation of on-demand credits as described in
[RFC-1](https://github.com/polkadot-fellows/RFCs/blob/main/text/0001-agile-coretime.md#instantaneous-coretime)

---------

Co-authored-by: ordian <write@reusable.software>
Co-authored-by: Dónal Murray <donalm@seadanda.dev>

Part of https://github.com/paritytech/polkadot-sdk/issues/5079.

Removes all usage of the static async backing params, replacing them
with dynamically computed equivalent values (based on the claim queue
and scheduling lookahead).

Adds a new runtime API for querying the scheduling lookahead value. If
not present, falls back to 3 (the default value that is backwards
compatible with values we have on production networks for
allowed_ancestry_len)

Also resolves most of
https://github.com/paritytech/polkadot-sdk/issues/4447, removing code
that handles async backing not yet being enabled.
While doing this, I removed the support for collation protocol version 1
on collators, as it only worked for leaves not supporting async backing
(which are none).
I also unhooked the legacy v1 statement-distribution (for the same
reason as above). That subsystem is basically dead code now, so I had to
remove some of its tests as they would no longer pass (since the
subsystem no longer sends messages to the legacy variant). I did not
remove the entire legacy subsystem yet, as that would pollute this PR
too much. We can remove the entire v1 and v2 validation protocols in a
follow up PR.

In another PR: remove test files with names `prospective_parachains`
(it'd pollute this PR if we do now)

TODO:
- [x] add deprecation warnings
- [x] prdoc

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

This should fix the error log related to PoV pre-dispatch weight being
lower than post-dispatch for `ParasInherent`:
```
ERROR tokio-runtime-worker runtime::frame-support: Post dispatch weight is greater than pre dispatch weight. Pre dispatch weight may underestimating the actual weight. Greater post dispatch weight components are ignored.
                                        Pre dispatch weight: Weight { ref_time: 47793353978, proof_size: 1019 },
                                        Post dispatch weight: Weight { ref_time: 5030321719, proof_size: 135395 }
```

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Part of: https://github.com/paritytech/polkadot-sdk/issues/6906

Currently the `para_backing_state` API is used only by the prospective
parachains subsystems and returns 2 things: the constraints for
parachain blocks and the candidates pending availability.

This PR deprecates `para_backing_state` and introduces a new
`backing_constraints` API that can be used together with
`candidates_pending_availability` to get the same information provided
by `para_backing_state`.

TODO:
- [x] PRDoc

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: command-bot <>

Migrate inclusion benchmark to v2.

---

Polkadot address: 156HGo9setPcU2qhFMVWLkcmtCEGySLwNqa3DaEiYSWtte4Y

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

It doesn't make sense to only reorder the features array.

For example:

This makes it hard for me to compare the dependencies and features,
especially some crates have a really really long dependencies list.
```toml​
[dependencies]
c = "*"
a = "*"
b = "*"

[features]
std = [
  "a",
  "b",
  "c",
]
```

This makes my life easier.
```toml​
[dependencies]
a = "*"
b = "*"
c = "*"

[features]
std = [
  "a",
  "b",
  "c",
]
```

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>

[polkadot-runtime-parachains] migrate disputes and disputes/slashing to benchmarking to bench v2 syntax (#6577)

[polkadot-runtime-parachains] migrate disputes and disputes/slashing to
benchmarking to bench v2 syntax

Part of:
* #6202

---------

Co-authored-by: Giuseppe Re <giuseppe.re@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

[polkadot-runtime-parachains] migrate paras module to benchmarking v2
syntax

Part of:
* #6202

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: command-bot <>

Closes: https://github.com/paritytech/polkadot-sdk/issues/6585

Removing the `require_weight_at_most` parameter in V5 Transact had only
one problem. Converting a message from V5 to V4 to send to chains that
didn't upgrade yet. The conversion would not know what weight to give to
the Transact, since V4 and below require it.

To fix this, I added back the weight in the form of an `Option<Weight>`
called `fallback_max_weight`. This can be set to `None` if you don't
intend to deal with a chain that hasn't upgraded yet. If you set it to
`Some(_)`, the behaviour is the same. The plan is to totally remove this
in V6 since there will be a good conversion path from V6 to V5.

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>

# Context

This PR aims to introduce XCMv5, for now it's in progress and will be
updated over time.
This branch will serve as a milestone branch for merging in all features
we want to add to XCM, roughly outlined
[here](https://github.com/polkadot-fellows/xcm-format/issues/60).
More features could be added.

## TODO
- [x] Migrate foreign assets from v3 to v4
- [x] Setup v5 skeleton
- [x] Remove XCMv2
- [x] https://github.com/paritytech/polkadot-sdk/pull/5390
- [x] https://github.com/paritytech/polkadot-sdk/pull/5585
- [x] https://github.com/paritytech/polkadot-sdk/pull/5420
- [x] https://github.com/paritytech/polkadot-sdk/pull/5876
- [x] https://github.com/paritytech/polkadot-sdk/pull/5971
- [x] https://github.com/paritytech/polkadot-sdk/pull/6148
- [x] https://github.com/paritytech/polkadot-sdk/pull/6228

Fixes #3434 
Fixes https://github.com/paritytech/polkadot-sdk/issues/4190
Fixes https://github.com/paritytech/polkadot-sdk/issues/5209
Fixes https://github.com/paritytech/polkadot-sdk/issues/5241
Fixes https://github.com/paritytech/polkadot-sdk/issues/4284

---------

Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Andrii <ndk@parity.io>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Joseph Zhao <65984904+programskillforverification@users.noreply.github.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: command-bot <>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Serban Iorga <serban@parity.io>

Relax requirements for `assign_core` so that it accepts updates for the
last scheduled entry.

Fixes #6102

---------

Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: GitHub Action <action@github.com>

Resolves https://github.com/paritytech/polkadot-sdk/issues/6179

Reported in
https://github.com/paritytech/polkadot-sdk/issues/6161#issuecomment-2432097120

Fixes a bug introduced in
https://github.com/paritytech/polkadot-sdk/pull/5461, where the claim
queue would contain entries even if the validator groups storage is
empty (which happens during the first session).

This PR sets the claim queue core count to be the minimum between the
num_cores param and the number of validator groups

TODO:
- [x] prdoc
- [x] unit test

Resolves https://github.com/paritytech/polkadot-sdk/issues/5970

Removes the code of the legacy parachains assigner, which was used prior
to coretime. Now that all networks are upgraded to use the coretime
assigner, we can remove it.

on top of https://github.com/paritytech/polkadot-sdk/pull/5423

This PR implements the plumbing work required for
https://github.com/paritytech/polkadot-sdk/issues/5047 . I also added
additional helper methods gated by feature "test" in primitives.

TODO:
- [x] PRDoc

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: GitHub Action <action@github.com>

Resolves https://github.com/paritytech/polkadot-sdk/issues/4776

This will enable proper core-sharing between paras, even if one of them
is not producing blocks.

TODO:
- [x] duplicate first entry in the claim queue if the queue used to be
empty
- [x] don't back anything if at the end of the block there'll be a
session change
- [x] write migration for removing the availability core storage
- [x] update and write unit tests
- [x] prdoc
- [x] add zombienet test for synchronous backing
- [x] add zombienet test for core-sharing paras where one of them is not
producing any blocks

_Important note:_
The `ttl` and `max_availability_timeouts` fields of the
HostConfiguration are not removed in this PR, due to #64.
Adding the workaround with the storage version check for every use of
the active HostConfiguration in all runtime APIs would be insane, as
it's used in almost all runtime APIs.

So even though the ttl and max_availability_timeouts fields will now be
unused, they will remain part of the host configuration.

These will be removed in a separate PR once #64 is fixed. Tracked by
https://github.com/paritytech/polkadot-sdk/issues/6067

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: command-bot <>

Original PR https://github.com/paritytech/polkadot-sdk/pull/2280
reverted in https://github.com/paritytech/polkadot-sdk/pull/3665

This PR reintroduces the reverted functionality with additional changes,
related effort
[here](https://github.com/paritytech/polkadot-sdk/pull/3623).
Description is copied over from the original PR

First part of [Extrinsic
Horizon](https://github.com/paritytech/polkadot-sdk/issues/2415)

Introduces a new trait `TransactionExtension` to replace
`SignedExtension`. Introduce the idea of transactions which obey the
runtime's extensions and have according Extension data (né Extra data)
yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for
transactions/extrinsics owing to there now being "proper" unsigned
transactions which obey the extension framework and "old-style" unsigned
which do not. Instead we have __*General*__ for the former and
__*Bare*__ for ...

Closes https://github.com/paritytech/polkadot-sdk/issues/5045 and
https://github.com/paritytech/polkadot-sdk/issues/5046

<del>On top of
https://github.com/paritytech/polkadot-sdk/pull/5362</del>

TODO:
- [x] storage migration for allowed relay parents tracker 
- [x] check session index
- [x] PRdoc
- [x] tests
- [x] ensure UMP queue cannot be abused with this change
- [x] Zombienet runtime upgrade test

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Quoting @bkchr (from
[here](https://github.com/paritytech/polkadot-sdk/issues/5334#issuecomment-2383815078)):

> That the hardcoded limit is used there again, is IMO not correct. The
`max_pov_size` should be controlled by the `HostConfiguration` and not
limited by some "random" constant.

This PR aims to change the hard limit to a not-so-random constant,
allowing more room for maneuvering in the future.

cc https://github.com/paritytech/ci_cd/issues/1035

cc https://github.com/paritytech/ci_cd/issues/1023

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Maksym H <1177472+mordamax@users.noreply.github.com>
Co-authored-by: gui <gui.thiolliere@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>

A change that I missed to add in
https://github.com/paritytech/polkadot-sdk/pull/5525 .

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

the range should always contain at least 2 values for the benchmark to
work
closes #5680

Follow-up to #5270. The baseline numbers for Westend were too high to be
representative of the reality as it seemed to do with how multi-variate
linear regression is calculated.

---------

Co-authored-by: command-bot <>

closes https://github.com/paritytech/polkadot-sdk/issues/5044

This PR switches the runtime to the new receipts format (vstaging
primitives). I've implemented `From` to convert from new primitives to
`v7` primitives and used them in the node runtime api client
implementation. Until we implement the support in the node, it will
continue e to use the v7 primitives but the runtime apis already use the
new primitives.


An expected downside of RFC103 is decoding V2 receipts shows garbage
values if the input is V1:

_![ima_9ce77de](https://github.com/user-attachments/assets/71d80e78-e238-4518-8cd1-548ae0d74b70)_

TODO:
- [x] fix tests
- [x] A few more tests for the new primitives
- [x] PRDoc

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

# Description

The error message should be logged out when the check method returns an
error.

Because specific information is lost when `UmpAcceptanceCheckErr`,
`ProcessedDownwardMessagesAcceptanceErr`, `HrmpWatermarkAcceptanceErr`,
`OutboundHrmpAcceptanceErr` are converted to `AcceptanceCheckErr`, a log
is added to each check.

## Integration

## Review Notes

# Checklist

* [ ] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [ ] My PR follows the [labeling requirements](

https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/CONTRIBUTING.md#Process
) of this project (at minimum one label for `T` required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)
* [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

As Runtime release 1.3.0 includes all of the remaining staging
primitives and APIs we can now release primitives version 8.
No other changes other than renaming/moving done here.

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

On top of #5082.

## Background

Previously, before #3479, we would
[include](https://github.com/paritytech/polkadot-sdk/blame/75074952/polkadot/runtime/parachains/src/builder.rs#L508C12-L508C44)
the cost enacting the candidate into the cost of processing a single
bitfield.
[Now](https://github.com/paritytech/polkadot-sdk/blame/dd48544a/polkadot/runtime/parachains/src/builder.rs#L529)
it is different, although the benchmarks seems to be not-up-to date.
Including the cost of enacting a candidate into a processing a single
bitfield cost was incorrect, since we multiple that by the number of
bitfields we have. Instead, we should separate calculate the cost of
processing a single bitfield without enactment, and multiple the cost of
enactment by the actual number of processed candidates (which is limited
by the number cores, not validators).

## Bench

Previously, the weight of `enact_candidate` was calculated manually
(without a benchmark) and then neglected:
https://github.com/paritytech/polkadot-sdk/blob/dd48544a

/polkadot/runtime/parachains/src/inclusion/mod.rs#L584

In this PR, we have a benchmark for it and it's based on the number of
ump and sent hrmp messages as well as whether the candidate has a
runtime upgrade (new_validation_code).
The differences from the previous attempt
https://github.com/paritytech/polkadot/pull/6929 are that
* we don't include the cost of enactment into the cost of processing a
backed candidate.
The reason for it is that enactment happens not in the same block as
backing (typically the next one), since we process bitfields before
backing votes.
* we don't take into account the size of the runtime upgrade, the
benchmark weight doesn't seem to depend much on it, but rather whether
there was one or not.

Similarly to the previous attempt, we don't account for dmp messages
(fixed cost). Also we don't account properly for received hrmp messages
(hrmp_watermark) because the cost of it depends on the runtime state and
can't be statically deduced in the benchmark (unless we pass the
information about channels as benchmark u32 arguments).

The total weight cost of processing a parainherent now includes the cost
of enactment of each candidate, but we don't do filtering based on that
(because we enact after processing bitfields and making other changes to
the storage).

## Numbers

```
Reads = 7 + (0 * u) + (3 * h) + (8 * c)
Writes = 10 + (1 * u) + (3 * h) + (7 * c)
```
In addition, there is a fixed cost of a few of ms (!) per candidate. 

This might result a full block slightly overflowing its weight with 200
enacted candidates, which in turn could prevent non-mandatory
transactions from being included in a block.

Given our modest limits on max ump and hrmp messages:
```
  maxUpwardMessageNumPerCandidate: 16
  hrmpMaxMessageNumPerCandidate: 10
```
and the fact that runtime upgrades are can't happen very frequently
(`validation_upgrade_cooldown`), we might only go over the limits in
case of many disputes.

TODOs:
- [x] Fix the overweight test
- [x] Generate the weights for Westend and Rococo
- [x] PRDoc

---------

Co-authored-by: command-bot <>
Co-authored-by: Alin Dima <alin@parity.io>

closes #849

## Context

For the background on this and the long-term fix, see
https://github.com/paritytech/polkadot-sdk/issues/849#issuecomment-2247895862.

## Changes

* The weigh files are renamed from `runtime_(parachains|common).*` to
`polkadot_runtime_(parachains|common).*`. The reason for it is the
renaming introduced in #4633. The new weight command and files are
generated now include `polkadot_` prefix.
* The WeightInfo for `paras_inherent` now includes `enter_empty` which
calculates the cost of processing an empty parachains inherent. This
cost is subtracted dynamically when calculating other weights (so the
other weights remain the same)

## Benefits

See
https://github.com/paritytech/polkadot-sdk/issues/849#issuecomment-2247895862,
but TL;DR is that we are not blocked on weights for scaling the number
of validators and cores further.

Resolved questions:
- [x] why new benchmarks for westend are doing fewer db IOPS?
Is it due polkadot-sdk update (db IOPS diff)?
or the bench setup is no longer valid?

https://github.com/polkadot-fellows/runtimes/blob/7723274a2c5cbb10213379271094d5180716ca7d/relay/polkadot/src/weights/runtime_parachains_paras_inherent.rs#L131-L196
Answer: see background section of #5270 

TODOs:
- [x] Rerun benchmarks for Rococo and Westend
- [x] PRDoc

---------

Co-authored-by: command-bot <>

## Description
This PR changes `PendingConfigs` storage item's visibility from crate to
public. This enable us to read it in our runtime.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>