Closes https://github.com/paritytech/polkadot-sdk/issues/2013

Fix for failed pipeline `test-doc`:
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4174859
I just wonder how could have other PR been merged after this was merged:
https://github.com/paritytech/polkadot-sdk/pull/1714/files#diff-1bde7bb2be0165cbe6db391e10a4a0b2f333348681373a86a0f8502d14d20d32R56

## Summary

Asset bridging support for AssetHub**Rococo** <-> AssetHub**Wococo** was
added [here](https://github.com/paritytech/polkadot-sdk/pull/1215), so
now we aim to bridge AssetHub**Rococo** and AssetHub**Westend**. (And
perhaps retire AssetHubWococo and the Wococo chains).

## Solution

**bridge-hub-westend-runtime**
- added new runtime as a copy of `bridge-hub-rococo-runtime`
- added support for bridging to `BridgeHubRococo`
- added tests and benchmarks

**bridge-hub-rococo-runtime**
- added support for bridging to `BridgeHubWestend`
- added tests and benchmarks
- internal refactoring by splitting bridge configuration per network,
e.g., `bridge_to_whatevernetwork_config.rs`.

**asset-hub-rococo-runtime**
- added support for asset bridging to `AssetHubWestend` (allows to
receive only WNDs)
- added new xcm router for `Westend`
- added tests and benchmarks

**asset-hub-westend-runtime**
- added support for asset bridging to `AssetHubRococo` (allows to
receive only ROCs)
- added new xcm router for `Rococo`
- added tests and benchmarks

## Deployment

All changes will be deployed as a part of
https://github.com/paritytech/polkadot-sdk/issues/1988.

## TODO

- [x] benchmarks for all pallet instances
- [x] integration tests
- [x] local run scripts


Relates to:
https://github.com/paritytech/parity-bridges-common/issues/2602
Relates to: https://github.com/paritytech/polkadot-sdk/issues/1988

---------

Co-authored-by: command-bot <>
Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: joe petrowski <[email protected]>

Changes:
- Change the fungible(s) logic to treat a self-transfer as No-OP (as
long as all pre-checks pass).

Note that the self-transfer case will not emit an event since no state
was changed.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

# Description

This PR updates the node-template-release generation binary as well as
the `node-template-release.sh` file so that we can automatically push
updates to the [substrate-node-template
repository](https://github.com/substrate-developer-hub/substrate-node-template).
I assume this part was not updated after the substrate project has been
moved into the polkadot-sdk mono repo.

# Adjustments
- extend the `node-template-release.sh` to support the substrate
child-folder
- update the `SUBSTRATE_GIT_URL`
- fix the Cargo.toml filter (so that it does not include any
non-relevant .toml files)
- set the workspace-edition to 2021

# Note
In order to auto-generate the artifacts [this
line](https://github.com/paritytech/polkadot-sdk/blob/master/.gitlab/pipeline/build.yml#L320C15-L320C15)
needs to be included in the build.yml script again. Since I do not have
access to the (probably) internal gitlab environment I hope that someone
with actual access can introduce that change.
I also do not know how the auto-publish feature works so that would be
another thing to add later on.

---------

Co-authored-by: Bastian Köcher <[email protected]>

This is a port (and hopefully a small improvement) of @Kianenigma

's PR
from the old Substrate repo:
https://github.com/paritytech/substrate/pull/13987. Following #1689 I
moved the documentation of all macros relevant to this PR from
`frame_support_procedural` to `pallet_macros` while including a hint for
RA users.

Question: Again with respect to #1689: Is there a good reason why we
should *not* enhance paths with links to our current rustdocs? For
example, instead of
```rust
/// **Rust-Analyzer users**: See the documentation of the Rust item in
/// `frame_support::pallet_macros::storage`.
```
we could write
```rust
/// **Rust-Analyzer users**: See the documentation of the Rust item in
/// [`frame_support::pallet_macros::storage`](https://paritytech.github.io/polkadot-sdk/master/frame_support/pallet_macros/attr.storage.html).
```
This results in a clickable link like this:
<img width="674" alt="image"
src="https://github.com/paritytech/polkadot-sdk/assets/10713977/c129e622-3942-4eeb-8acf-93ee4efdc99d">
I don't really expect the links to become outdated any time soon, but I
think this would be a great UX improvement over just having paths.

TODOs:
- [ ] Add documentation for `constant_name` macro
- [x] Add proper documentation for different `QueryKinds`, i.e.
`OptionQuery`, `ValueQuery`, `ResultQuery`. One example for each. Custom
`OnEmpty` should be moved to `QueryKinds` trait doc page.
- [ ] Rework `type_value` docs

---------

Co-authored-by: kianenigma <[email protected]>

This will remove `core-devs` from being required reviewers of PRs,

CI image has been updated in the shared snippet, reverting the variable
back.

Co-authored-by: Marcin S <[email protected]>

helps https://github.com/paritytech/polkadot-sdk/issues/439.
closes https://github.com/paritytech/polkadot-sdk/issues/473.

PR link in the older substrate repository:
https://github.com/paritytech/substrate/pull/13498.

# Context
Rewards payout is processed today in a single block and limited to
`MaxNominatorRewardedPerValidator`. This number is currently 512 on both
Kusama and Polkadot.

This PR tries to scale the nominators payout to an unlimited count in a
multi-block fashion. Exposures are stored in pages, with each page
capped to a certain number (`MaxExposurePageSize`). Starting out, this
number would be the same as `MaxNominatorRewardedPerValidator`, but
eventually, this number can be lowered through new runtime upgrades to
limit the rewardeable nominators per dispatched call instruction.

The changes in the PR are backward compatible.

## How payouts would work like after this change
Staking exposes two calls, 1) the existing `payout_stakers` and 2)
`payout_stakers_by_page`.

### payout_stakers
This remains backward compatible with no signature change. If for a
given era a validator has multiple pages, they can call `payout_stakers`
multiple times. The pages are executed in an ascending sequence and the
runtime takes care of preventing double claims.

### payout_stakers_by_page
Very similar to `payout_stakers` but also accepts an extra param
`page_index`. An account can choose to payout rewards only for an
explicitly passed `page_index`.

**Lets look at an example scenario**
Given an active validator on Kusama had 1100 nominators,
`MaxExposurePageSize` set to 512 for Era e. In order to pay out rewards
to all nominators, the caller would need to call `payout_stakers` 3
times.

- `payout_stakers(origin, stash, e)` => will pay the first 512
nominators.
- `payout_stakers(origin, stash, e)` => will pay the second set of 512
nominators.
- `payout_stakers(origin, stash, e)` => will pay the last set of 76
nominators.
...
- `payout_stakers(origin, stash, e)` => calling it the 4th time would
return an error `InvalidPage`.

The above calls can also be replaced by `payout_stakers_by_page` and
passing a `page_index` explicitly.

## Commission note
Validator commission is paid out in chunks across all the pages where
each commission chunk is proportional to the total stake of the current
page. This implies higher the total stake of a page, higher will be the
commission. If all the pages of a validator's single era are paid out,
the sum of commission paid to the validator across all pages should be
equal to what the commission would have been if we had a non-paged
exposure.

### Migration Note
Strictly speaking, we did not need to bump our storage version since
there is no migration of storage in this PR. But it is still useful to
mark a storage upgrade for the following reasons:

- New storage items are introduced in this PR while some older storage
items are deprecated.
- For the next `HistoryDepth` eras, the exposure would be incrementally
migrated to its corresponding paged storage item.
- Runtimes using staking pallet would strictly need to wait at least
`HistoryDepth` eras with current upgraded version (14) for the migration
to complete. At some era `E` such that `E >
era_at_which_V14_gets_into_effect + HistoryDepth`, we will upgrade to
version X which will remove the deprecated storage items.
In other words, it is a strict requirement that E<sub>x</sub> -
E<sub>14</sub> > `HistoryDepth`, where
E<sub>x</sub> = Era at which deprecated storages are removed from
runtime,
E<sub>14</sub> = Era at which runtime is upgraded to version 14.
- For Polkadot and Kusama, there is a [tracker
ticket](https://github.com/paritytech/polkadot-sdk/issues/433) to clean
up the deprecated storage items.

### Storage Changes

#### Added
- ErasStakersOverview
- ClaimedRewards
- ErasStakersPaged

#### Deprecated
The following can be cleaned up after 84 eras which is tracked
[here](https://github.com/paritytech/polkadot-sdk/issues/433).

- ErasStakers.
- ErasStakersClipped.
- StakingLedger.claimed_rewards, renamed to
StakingLedger.legacy_claimed_rewards.

### Config Changes
- Renamed MaxNominatorRewardedPerValidator to MaxExposurePageSize.

### TODO
- [x] Tracker ticket for cleaning up the old code after 84 eras.
- [x] Add companion.
- [x] Redo benchmarks before merge.
- [x] Add Changelog for pallet_staking.
- [x] Pallet should be configurable to enable/disable paged rewards.
- [x] Commission payouts are distributed across pages.
- [x] Review documentation thoroughly.
- [x] Rename `MaxNominatorRewardedPerValidator` ->
`MaxExposurePageSize`.
- [x] NMap for `ErasStakersPaged`.
- [x] Deprecate ErasStakers.
- [x] Integrity tests.

### Followup issues
[Runtime api for deprecated ErasStakers storage
item](https://github.com/paritytech/polkadot-sdk/issues/426)

---------

Co-authored-by: Javier Viola <[email protected]>
Co-authored-by: Ross Bulat <[email protected]>
Co-authored-by: command-bot <>

Run CI using new image with nightly 2023-11-01

cc https://github.com/paritytech/polkadot-sdk/issues/2113
cc https://github.com/paritytech/ci_cd/issues/896

This PR moves syncing-related code from `sc-network-common` to
`sc-network-sync`.

Unfortunately, some parts are tightly integrated with networking, so
they were left in `sc-network-common` for now:

1. `SyncMode` in `common/src/sync.rs` (used in `NetworkConfiguration`).
2. `BlockAnnouncesHandshake`, `BlockRequest`, `BlockResponse`, etc. in
`common/src/sync/message.rs` (used in `src/protocol.rs` and
`src/protocol/message.rs`).

More substantial refactoring is needed to decouple syncing and
networking completely, including getting rid of the hardcoded sync
protocol.

## Release notes

Move syncing-related code from `sc-network-common` to `sc-network-sync`.
Delete `ChainSync` trait as it's never used (the only implementation is
accessed directly from `SyncingEngine` and exposes a lot of public
methods that are not part of the trait). Some new trait(s) for syncing
will likely be introduced as part of Sync 2.0 refactoring to represent
syncing strategies.

This version includes paritytech/review-bot#97 which can assign
reviewers.

It will be the final step required to replace PRCR.

It also moves the secrets to the environment master.

Bumps
[chevdor/srtool-actions](https://github.com/chevdor/srtool-actions) from
0.8.0 to 0.9.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/chevdor/srtool-actions/releases">chevdor/srtool-actions's
releases</a>.</em></p>
<blockquote>
<h2>v0.9.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Add debugging info by <a
href="https://github.com/chevdor"><code>@​chevdor</code></a> in <a
href="https://redirect.github.com/chevdor/srtool-actions/pull/25">chevdor/srtool-actions#25</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/chevdor/srtool-actions/compare/v0.8.0...v0.9.0">https://github.com/chevdor/srtool-actions/compare/v0.8.0...v0.9.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/chevdor/srtool-actions/commit/d7c66d9766f633abab95127ee910dd96892f8e3b"><code>d7c66d9</code></a>
Add debugging info and fixes for the monorepo (<a
href="https://redirect.github.com/chevdor/srtool-actions/issues/25">#25</a>)</li>
<li>See full diff in <a
href="https://github.com/chevdor/srtool-actions/compare/v0.8.0...v0.9.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=chevdor/srtool-actions&package-manager=github_actions&previous-version=0.8.0&new-version=0.9.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: Sergejs Kostjucenko <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

# Description

The `trigger_defensive` call has been added to the `root-testing`
pallet. The idea is to have this pallet running on `Rococo/Westend` and
use it to verify if the runtime monitoring works end-to-end.

To accomplish this, `trigger_defensive` dispatches an event when it is
called.

Closes #1953

# Checklist

- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

You can remove the "Checklist" section once all have been checked. Thank
you for your contribution!

✄
-----------------------------------------------------------------------------

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

We shouldn't override with their default fields that have been added in
the previous version(v8), because we are going to lose whatever values have
been set.

Note, v8 & v9 seems to have landed at the same time on Rococo, probably
they will land at the same time on westend and other chains, so functionally
doesn't make much difference, but let's have this fixed for people that copy-paste
:D, like me.

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>

This version includes:
- Move `spot` usage in CI to 50%
- Fix `PodMonitor`, metrics will be relayed to grafana

Removed confusing and outdated `TODO`.

- Usage the new published
[arkworks-extensions](https://github.com/paritytech/arkworks-extensions)
crates.
  Hooks are internally defined to jump into the proper host functions.
- Conditional compilation of each curve (gated by feature with curve
name)
- Separation in smaller host functions sets, divided by curve (fits
nicely with prev point)

This pull request changes how `check-each-crate.py` is working. Instead
of passing the name of the crate via `-p`, we now jump into the
directory of the crate and call there `cargo check`. This should fix
issues like https://github.com/paritytech/polkadot-sdk/issues/2013 where
a crate is present twice in the `Cargo.lock`.

Besides that it also changes `core/Cargo.toml` to not always pull in
bandersnatch.

The change adds a test to show the failure scenario that caused #1812 to
be rolled back (more context:
https://github.com/paritytech/polkadot-sdk/issues/493#issuecomment-1772009924)

Summary of the scenario:
1. Node has finished downloading up to block 1000 from the peers, from
the canonical chain.
2. Peers are undergoing re-org around this time. One of the peers has
switched to a non-canonical chain, announces block 1001 from that chain
3. Node downloads 1001 from the peer, and tries to import which would
fail (as we don't have the parent block 1000 from the other chain)

---------

Co-authored-by: Dmitry Markin <[email protected]>

Restore fix from #2077

This ensures that upgrading a parachain code sends the same event &
digest as when using `set_code` on a standalone chain.

Close: https://github.com/paritytech/polkadot-sdk/issues/2049

Switch from: 
https://crates.io/crates/tiny-bip39
to:
https://crates.io/crates/bip39

Required for: https://github.com/paritytech/polkadot-sdk/pull/2044

Partial https://github.com/paritytech/polkadot-sdk/issues/226

`frame/transaction-storage`: replace `Currency` with `fungible::*`
traits

---------

Signed-off-by: Adrian Catangiu <[email protected]>
Co-authored-by: georgepisaltu <[email protected]>

Runtime migration CI is currently failing
(https://gitlab.parity.io/parity/mirrors/polkadot-sdk/builds/4122083)
for the contracts testnet due to unnecessary panicing in a `pre_upgrade`
hook.

Soon idempotency will be enforced
https://github.com/paritytech/try-runtime-cli/issues/42, in the mean
time we need to manually fix these issues as they arise.

---

also removes backticks from the string in `echo`, which caused a
'command not found' error in ci output

Closes https://github.com/paritytech/polkadot-sdk/issues/1970

Follow up issue to tackle, once the erroneous double
incrementing/decrementing has stopped:
https://github.com/paritytech/polkadot-sdk/issues/2037

Making some devex improvements as I audit our chains adherence to
try-state invariants, in preparation for automated try-state checks and
alerting.

Note to reviewer: while you're here, if you have time would be great to
get your eyes on https://github.com/paritytech/polkadot-sdk/pull/1297
also since it touches a similar file and I'd like to avoid merge
conflicts :P

## Devex Improvements

- Changes the log level of logs informing the user that try-state checks
are being run for a pallet from debug to info
- Improves how errors are communicated
- Errors are logged when they are encountered, rather than after
everything has been executed
- Exact pallet the error originated from is included with the error log
  - Clearly see all errors and how many there are, rather than only one
  - Closes #136 

### Example of new logs

<img width="1185" alt="Screenshot 2023-10-25 at 15 44 44"
src="https://github.com/paritytech/polkadot-sdk/assets/16665596/b75588a2-1c64-45df-bbc8-bcb8bf8b0fe0">

### Same but with old logs (run with RUST_LOG=debug)

Notice only informed of one of the errors, and it's unclear which pallet
it originated

<img width="1185" alt="Screenshot 2023-10-25 at 15 39 01"
src="https://github.com/paritytech/polkadot-sdk/assets/16665596/e3429cb1-489e-430a-9716-77c052e5dae6">
 

## Bug fix

When dry-running migrations and `checks.try_state()` is `true`, only run
`try_state` checks after migrations have been executed. Otherwise,
`try_state` checks that expect state to be in at a HIGHER storage
version than is on-chain could incorrectly fail.

---------

Co-authored-by: command-bot <>

Currently the CLI `-h/--help` commad output is almost unreadable as (for
some commands) it:
- doesn't provide a short brief of what the command does.
- doesn't separate the options description in smaller paragraphs.
- doesn't use a smart wrap strategy for lines longer than the number of
columns in the terminal.

Follow some pics taken with a 100 cols wide term

## Short help (./node -h)

### Before


![20231028-174531-grim](https://github.com/paritytech/polkadot-sdk/assets/8143589/11b62c3c-dcd5-43f4-ac58-f1b299e3f4b9)

### After


![20231028-175041-grim](https://github.com/paritytech/polkadot-sdk/assets/8143589/dc08f6fd-b287-40fb-8b33-71a185922104)


## Long help (./node --help)

### Before


![20231028-175257-grim](https://github.com/paritytech/polkadot-sdk/assets/8143589/9ebdc0ae-54ee-4760-b873-a7e813523cb6)

### After


![20231028-175155-grim](https://github.com/paritytech/polkadot-sdk/assets/8143589/69cbe5cb-eb2f-46a5-8ebf-76c0cf8c4bad)

---------

Co-authored-by: command-bot <>

In https://github.com/paritytech/substrate/pull/13740 the use of the
`heap-pages` param inside the `frame-benchmarking-cli` has been removed.
This results in running out of memory and this PR fixes the heap
allocation strategy for benchmarks wasm executor.

Closes https://github.com/paritytech/release-engineering/issues/6

Adds a new Github Workflow which on a new release being created, builds
and attaches all runtimes managed in this repository in two flavours:
- `dev-debug-build`: Built with the `try-runtime` feature and has
logging enabled
- `on-chain-release`: Built with the regular old `on-chain-release`
feature

The new Github Workflow could be extended in the future by the
@paritytech/release-engineering team to fully automate the release
process if they choose to, similar to how it is fully automated in the
Fellowship repo
(https://github.com/polkadot-fellows/runtimes/blob/main/.github/workflows/release.yml).

The `on-chain-release` did not exist for parachains, so I added it. 

---

Tested on my fork: 
- https://github.com/liamaharon/polkadot-sdk/actions/runs/6663773523
- https://github.com/liamaharon/polkadot-sdk/releases/tag/test-6

---------

Co-authored-by: Chevdor <[email protected]>
Co-authored-by: Dónal Murray <[email protected]>

Updates `docify` to 0.2.6, which fixes a bug that was preventing nesting
`#[docify::export]` within sub-items of items that already have
`#[docify::export]` attached to them from working properly.

Release notes here:
https://github.com/sam0x17/docify/releases/tag/v0.2.6

cc @ggwpez @Kianenigma

When warp syncing a node we import a header of the parachain around the
tip of the chain. This header is currently not imported as finalized
block (should be fixed at some point as well), the parent headers are
not yet present (still being synced) and thus, we run into a panic. Even
if there is a case where a leaf could not be found in the database, this
probably means that the db is broken and it will fail somewhere elese.