https://github.com/paritytech/polkadot-sdk/pull/3166 made private
functions used in `spawn_tasks()` public but forgot to add them in
exported functions of the crate.

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

The storage item ExtrinsicIndex is already taken before the `finalize()`
in `note_finished_extrinsics()`, rendering it's always 0 in the log.
This commit fixes it by using the proper API for extrinsics count.

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

This is for adding onfinality polkadot bootnode. Please correct me if
this is not the right place for adding a new bootnode

This is the second PR in preparation for
https://github.com/paritytech/polkadot-sdk/issues/454.

## Also see
- **Precursor** https://github.com/paritytech/polkadot-sdk/pull/3889.
- **Follow up** https://github.com/paritytech/polkadot-sdk/pull/3905.

Overall changes are documented here (lot more visual 😍

):
https://hackmd.io/@ak0n/454-np-governance

## Changes
### Delegation Interface
Provides delegation primitives for staking. 

Introduces two new roles:
- Agent: These are accounts who receive delegation from other accounts
(delegators) and stakes on behalf of them. The funds are held in
delegator accounts.
- Delegator: Accounts who delegate their funds to an agent authorising
them to use it for staking.

Supports
- A way for delegators to add or withdraw delegation to an agent.
- A way for an agent to slash a delegator during a slashing event.

### Pallet Delegated Staking
- Implements `DelegationInterface`.
- Lazy slashing: Any slashes to an Agent is posted in a ledger but not
immediately slashed. The agent can call
`DelegationInterface::delegator_slash` to slash the member and clear the
corresponding slash from its ledger.
- Consumes `StakingInterface` to provide `CoreStaking` features. In
reality, this will be `pallet-staking`.
- Ensures bookkeeping for agent and delegator are correct but leaves the
management of reward and slash logic upto the consumer of this pallet.
- While it does not expose any calls yet, it is written with the intent
of exposing these primitives via extrinsics.

## TODO
- [x] Improve unit tests in the pallet.
- [x] Separate slash reward perbill for rewarding the slash reporters?
- [x] Review if we should add more events.

---------

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Gonçalo Pestana <[email protected]>
Co-authored-by: georgepisaltu <[email protected]>

Link was dead

---------

Co-authored-by: Bastian Köcher <[email protected]>

Add small fixes, reorg debug definition and add more verbose debug logs
in k8s logs reading.
Thx!

Please add RadiumBlock bootnodes to the spec files for Westend Coretime
and Westend People

---------

Co-authored-by: Dónal Murray <[email protected]>

... to get rid of the macro debug logs like this from build output
```
[/home/alexggh/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fatality-proc-macro-0.1.0/src/types.rs:171:23] input = TokenStream [
    Literal {
        kind: Str,
        symbol: "Error while accessing runtime information",
        suffix: None,
        span: #0 bytes(6943..6986),
    },
]
```

Signed-off-by: Alexandru Gheorghe <[email protected]>

This PR changes the fork calculation and pruning algorithm to enable
future block header pruning. It's required because the previous
algorithm relied on the block header persistence. It follows the
[related
discussion](https://github.com/paritytech/polkadot-sdk/issues/1570)

The previous code contained this comment describing the situation:
```
	/// Note a block height finalized, displacing all leaves with number less than the finalized
	/// block's.
	///
	/// Although it would be more technically correct to also prune out leaves at the
	/// same number as the finalized block, but with different hashes, the current behavior
	/// is simpler and our assumptions about how finalization works means that those leaves
	/// will be pruned soon afterwards anyway.
	pub fn finalize_height(&mut self, number: N) -> FinalizationOutcome<H, N> {
```

The previous algorithm relied on the existing block headers to prune
forks later and to enable block header pruning we need to clear all
obsolete forks right after the block finalization to not depend on the
related block headers in the future.

---------

Co-authored-by: Bastian Köcher <[email protected]>

A small step towards
https://forum.polkadot.network/t/polkadot-parachain-omni-node-gathering-ideas-and-feedback/7823
and #4352.

Many parachains use `camelCase` and/or `PascalCase`ing for their chain
spec extension. Sometimes the only reason polkadot-parachain cannot sync
them is because it cannot parse the chain spec extension. This PR
relaxes the requirement for the extension to be camel case.

---------

Co-authored-by: Branislav Kontur <[email protected]>

Bumps [nix](https://github.com/nix-rust/nix) from 0.27.1 to 0.28.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/nix-rust/nix/blob/master/CHANGELOG.md">nix's
changelog</a>.</em></p>
<blockquote>
<h2>[0.28.0] - 2024-02-24</h2>
<h3>Added</h3>
<ul>
<li>
<p>Added <code>mkdtemp</code> wrapper (<a
href="https://redirect.github.com/nix-rust/nix/pull/1297">#1297</a>)</p>
</li>
<li>
<p>Add associated constants <code>UTIME_OMIT</code>
<code>UTIME_NOW</code> for <code>TimeSpec</code>
(<a
href="https://redirect.github.com/nix-rust/nix/pull/1879">#1879</a>)</p>
</li>
<li>
<p>Added <code>EventFd</code> type. (<a
href="https://redirect.github.com/nix-rust/nix/pull/1945">#1945</a>)</p>
</li>
<li>
<ul>
<li>Added <code>impl From&lt;Signal&gt; for SigSet</code>.</li>
<li>Added <code>impl std::ops::BitOr for SigSet</code>.</li>
<li>Added <code>impl std::ops::BitOr for Signal</code>.</li>
<li>Added <code>impl std::ops::BitOr&lt;Signal&gt; for
SigSet</code></li>
</ul>
<p>(<a
href="https://redirect.github.com/nix-rust/nix/pull/1959">#1959</a>)</p>
</li>
<li>
<p>Added <code>TlsGetRecordType</code> control message type and
corresponding enum for
linux (<a
href="https://redirect.github.com/nix-rust/nix/pull/2065">#2065</a>)</p>
</li>
<li>
<p>Added <code>Ipv6HopLimit</code> to
<code>::nix::sys::socket::ControlMessage</code> for Linux,
MacOS, FreeBSD, DragonflyBSD, Android, iOS and Haiku.
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2074">#2074</a>)</p>
</li>
<li>
<p>Added <code>Icmp</code> and <code>IcmpV6</code> to
<code>SockProtocol</code>
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2103">#2103</a>)</p>
</li>
<li>
<p>Added rfork support for FreeBSD in <code>unistd</code>
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2121">#2121</a>)</p>
</li>
<li>
<p>Added <code>MapFlags::map_hugetlb_with_size_log2</code> method for
Linux targets
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2125">#2125</a>)</p>
</li>
<li>
<p>Added <code>mmap_anonymous</code> function
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2127">#2127</a>)</p>
</li>
<li>
<p>Added <code>mips32r6</code> and <code>mips64r6</code> support for
signal, ioctl and ptrace
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2138">#2138</a>)</p>
</li>
<li>
<p>Added <code>F_GETPATH</code> FcntlFlags entry on
Apple/NetBSD/DragonflyBSD for
<code>::nix::fcntl</code>. (<a
href="https://redirect.github.com/nix-rust/nix/pull/2142">#2142</a>)</p>
</li>
<li>
<p>Added <code>F_KINFO</code> FcntlFlags entry on FreeBSD for
<code>::nix::fcntl</code>.
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2152">#2152</a>)</p>
</li>
<li>
<p>Added <code>F_GETPATH_NOFIRMLINK</code> and
<code>F_BARRIERFSYNC</code> FcntlFlags entry
on Apple for <code>::nix::fcntl</code>.
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2155">#2155</a>)</p>
</li>
<li>
<p>Added newtype <code>Flock</code> to automatically unlock a held flock
upon drop.
Added <code>Flockable</code> trait to represent valid types for
<code>Flock</code>.
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2170">#2170</a>)</p>
</li>
<li>
<p>Added <code>SetSockOpt</code> impls to enable Linux Kernel TLS on a
TCP socket and to
import TLS parameters. (<a
href="https://redirect.github.com/nix-rust/nix/pull/2175">#2175</a>)</p>
</li>
<li>
<ul>
<li>Added the <code>::nix::sys::socket::SocketTimestamp</code> enum for
configuring the
<code>TsClock</code> (a.k.a <code>SO_TS_CLOCK</code>) sockopt</li>
<li>Added FreeBSD's <code>ScmRealtime</code> and
<code>ScmMonotonic</code> as new options in
<code>::nix::sys::socket::ControlMessageOwned</code></li>
</ul>
<p>(<a
href="https://redirect.github.com/nix-rust/nix/pull/2187">#2187</a>)</p>
</li>
<li>
<p>Added new fanotify API: wrappers for <code>fanotify_init</code> and
<code>fanotify_mark</code>
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2194">#2194</a>)</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/nix-rust/nix/commit/21ab06ef23de214174ddb039be5b5f08750d19e6"><code>21ab06e</code></a>
chore: release 0.8.0</li>
<li><a
href="https://github.com/nix-rust/nix/commit/595c6eb2515723eae1f1db1b9103c95f339380ec"><code>595c6eb</code></a>
chore: changelog for 0.28.0 (<a
href="https://redirect.github.com/nix-rust/nix/issues/2303">#2303</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/79b86a7dff071a2a59858d1c98e202fdf38ebd06"><code>79b86a7</code></a>
mmsg revert signature simplification (<a
href="https://redirect.github.com/nix-rust/nix/issues/2227">#2227</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/08e05a5f66d14d4f06288c9a725e87f473eba41a"><code>08e05a5</code></a>
refactor: remove redundant imports to fix CI (<a
href="https://redirect.github.com/nix-rust/nix/issues/2320">#2320</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/197f55b3ccbce3273bf6ce119d1a8541b5df5d66"><code>197f55b</code></a>
refactor: update nix (<a
href="https://redirect.github.com/nix-rust/nix/issues/2311">#2311</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/f34d00d353a8ebcb16eb8810b00d613e4a294925"><code>f34d00d</code></a>
chore(deps): try libc 0.2.153 (<a
href="https://redirect.github.com/nix-rust/nix/issues/2309">#2309</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/062f6c0f0306743a401138fc2612d59e818d5768"><code>062f6c0</code></a>
test: clean clippy needless_borrow false positive issue (<a
href="https://redirect.github.com/nix-rust/nix/issues/2310">#2310</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/ca173ff30cc42e0fc7a7ba2cf7c1fcbe61dccfa5"><code>ca173ff</code></a>
Fix the mount tests (<a
href="https://redirect.github.com/nix-rust/nix/issues/2269">#2269</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/0dfcf3228489a5060a53e69783a1eea6f43b11a9"><code>0dfcf32</code></a>
test: tweak test_fanotify to monitor ONLY 1 file (<a
href="https://redirect.github.com/nix-rust/nix/issues/2294">#2294</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/0fb346031f6ddc87a57377101a6063133a25e443"><code>0fb3460</code></a>
Temporarily disable CI on GNU hurd (<a
href="https://redirect.github.com/nix-rust/nix/issues/2307">#2307</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/nix-rust/nix/compare/v0.27.1...v0.28.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=nix&package-manager=cargo&previous-version=0.27.1&new-version=0.28.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

related to
https://github.com/paritytech/parity-bridges-common/issues/2962
Usage example:
```sh
RUST_LOG=runtime=trace,rpc=trace,bridge=trace \
        ./target/release/substrate-relay relay-messages-range bridge-hub-rococo-to-bridge-hub-westend \
        --source-host localhost \
        --source-port 8943 \
        --source-version-mode Auto \
        --source-signer //Eve \
        --source-transactions-mortality 4 \
        --target-host localhost \
        --target-port 8945 \
        --target-version-mode Auto \
        --target-signer //Eve \
        --target-transactions-mortality 4 \
        --lane 00000002 \
        --at-source-block 34 \
        --messages-start 1 \
        --messages-end 1
INFO bridge Connecting to BridgeHubRococo node at ws://localhost:8943
INFO bridge Connecting to BridgeHubWestend node at ws://localhost:8945
TRACE bridge Refined weight of BridgeHubRococo->BridgeHubWestend message [0, 0, 0, 2]/1: at-source: Weight(ref_time: 0, proof_size: 0), at-target: Weight(ref_time: 452953993, proof_size: 0)
TRACE bridge Sent transaction to BridgeHubWestend node: 0x38552f4db6bc78baecb52ebd2f7d103b1c919c16b83129dc083bf01b7281955b
TRACE bridge BridgeHubWestend transaction 0x38552f4db6bc78baecb52ebd2f7d103b1c919c16b83129dc083bf01b7281955b has been included in block: (0x29a20bdca8726df0b32af9067290b7fc0a886908da3a30f3db60a6ea52be4604, 0)
TRACE bridge BridgeHubWestend transaction 0x38552f4db6bc78baecb52ebd2f7d103b1c919c16b83129dc083bf01b7281955b has been finalized at block: 0x29a20bdca8726df0b32af9067290b7fc0a886908da3a30f3db60a6ea52be4604
```

Github currently [doesn't support creating
subfolders](https://github.com/orgs/community/discussions/18055) in
`.github/workflows` folder. PR unifies naming in this folder. The idea
is that all files ideally should have naming
`<stage>-<short_descriptive_name>.yml`. The stage names are taken from
gitlab and needed only for logical division.
Changes in actions:
 - `check-licences` runs on self-hosted runners
- `check-workspace` and `check-markdown` are moved to `checks-quick.yml`
 - `test-linux-stable-int` moved to a new file `tests-linux-stable.yml`
- `fmt-check.yml` file is removed, formatting is checked in
`checks-quick.yml`

`*-host` and `*-port` are obsolete and we'll hopefully remove them in
the future (already WIP for Rococo <> Westend relayer)

Support async backing in `--dev` mode

This PR improve the relay mock `MockValidationDataInherentDataProvider`
to mach expectations of async backing runtimes.

* Add para_head in the mock relay proof
* Add relay slot in the mock relay proof 

fix https://github.com/paritytech/polkadot-sdk/issues/4437

This PR bumps `proc-macro-crate` to the latest version.

In order to test a runtime from
https://github.com/polkadot-fellows/runtimes/ with the latest version of
polkadot-sdk one needs to use `cargo vendor` to extract all runtime
dependencies, patch them by hand and then build the runtime.

However at the moment 'vendored' builds fail due to
https://github.com/bkchr/proc-macro-crate/issues/48. To fix this
`proc-macro-crate` should be updated to version `3.0.1` or higher.

---------

Co-authored-by: command-bot <>

- Moved check-runtime-migration from gitlab to github
- Moved `test-syscalls` and `cargo-check-all-benches` from gitlab to
github

Reworks prospective-parachains so that we allow a number of unconnected
candidates (for which we don't know the parent candidate yet). Needed
for elastic scaling:
https://github.com/paritytech/polkadot-sdk/issues/3541. Without this,
candidate B will not be validated and backed until candidate A (its
parent) is validated and a backing statement reaches the validator.

Due to the high complexity of the subsystem, I rewrote parts of it so
that we don't concern ourselves with candidates which form cycles or
which form parachain forks. We now have "Fragment chains" instead of
"Fragment trees". This greatly simplifies some of the code and is a
compromise we can make. We just need to make sure that cycle-producing
parachains don't brick the relay chain and that fork-producing
parachains can still make some progress (on one core at least). The only
forks that are allowed are those on the relay chain, obviously.

Unconnected candidates are kept in the `CandidateStorage` and whenever a
new candidate is introduced, we try to repopulate the chain with as many
candidates as we can.

Also fixes https://github.com/paritytech/polkadot-sdk/issues/3219

Guide changes will be done as part of:
https://github.com/paritytech/polkadot-sdk/issues/3699

TODOs:

- [x] see if we can replace the `Cow` over the candidate commitments
with an `Arc` over the entire `ProspectiveCandidate`. It's only being
overwritten in unit tests. We can work around that.
- [x] finish fragment_chain unit tests
- [x] add more prospective-parachains subsystem tests
- [x] test with zombienet what happens if a parachain is creating cycles
(it should not brick the relay chain).
- [x] test with zombienet a parachain that is creating forks. it should
keep producing blocks from time to time (one bad collator should not DOS
the parachain, even if throughput decreases)
- [x] add some more logs and metrics
- [x] add prdoc and remove the "silent" label

---------

Signed-off-by: Andrei Sandu <[email protected]>
Co-authored-by: Andrei Sandu <[email protected]>

PR adds forklift settings and forklift to test-github-actions

cc https://github.com/paritytech/ci_cd/issues/939

Currently the `CheckWeight` `SignedExtension` was tracking the size of
the proof and the extrinsic length separately. But in reality we need
one more check that ensures we don't hit the PoV limit with both
combined.

The rest of the logic remains unchanged. One scenario where the changes
make a difference is when we enter this branch:

https://github.com/paritytech/polkadot-sdk/blob/f34d8e3c

/substrate/frame/system/src/extensions/check_weight.rs#L185-L198

This was previously allowing to some extrinsics that is exceeding the
block limit but are withing the reserved area of `BlockWeights`. This
will now be caught by the later check I introduced. I think the new
behaviour makes sense, since the proof size dimension is designed for
parachains and they don't want to go over the limit and get rejected.


In the long run we should maybe get rid of `RuntimeBlockLength`
alltogether, however that would require a deprecation process and can
come at a later point.

---------

Co-authored-by: Adrian Catangiu <[email protected]>

Extracting the logic for generating and verifying ancestry proofs from
this PR: https://github.com/paritytech/polkadot-sdk/pull/1903 with small
adjustments

@Lederstrumpf I added you as a co-author to each commit. Please let me
know if you want me to also associate an e-mail address with your name

The state-trie migration is completed on Rococo Asset-Hub as
double-checked
[here](https://github.com/paritytech/polkadot-sdk/issues/4174#issuecomment-2097895275).
Undeploying now.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Library `substrate-frame-cli` seems unused.

Last non-dependabot update to the folder was over [two years
ago](https://github.com/paritytech/polkadot-sdk/commits/master/substrate/utils/frame/frame-utilities-cli).

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

#### Problem
`take()` consumes only 1 read worth of weight in
`single-block-migrations` example, while `take()`
[is](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/support/src/storage/unhashed.rs#L63)
`get() + kill()`, i.e should be 1 read + 1 write. I think this could
mislead developers who follow this example to write their migrations

---------

Co-authored-by: Bastian Köcher <[email protected]>

The block header is required to derive inherents for a relay chain next
block, this is useful in testing environments.

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

Closes: https://github.com/paritytech/polkadot-sdk/issues/4428

As per #3326, removes pallet::getter usage from the pallet
authority-discovery. The syntax `StorageItem::<T, I>::get()` should be
used instead.

cc @muraca

---------

Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

A part of #3326 

Removes all #[pallet::getter] usage from the contracts mock network
pallet. As the storage values were pub(super), read-only visibility was
lost external to the crate upon the removal of the macros. I have
implemented custom getters as a replacement, keeping the api the same.

If we care very much about consistency of the
storagevalue::<T>::get() syntax, the other option would be to set
the storage values to pub. Though I find preserving data authority
better myself.

@muraca

The fact that this takes two sessions to come into effect is not
obvious. Just added some docs to explain that.

Also tidied up uses of "broker chain" -> "coretime chain"

Closes #1961

Disable logging of rustls to get rid off the following log lines:
```
Sending fatal alert BadCertificate
```

Upstream also removed them: https://github.com/rustls/rustls/pull/1278


Closes: https://github.com/paritytech/polkadot-sdk/issues/3252

This PR adds two new CLI options to disable rate limiting for certain ip
addresses and whether to trust "proxy header".
After going back in forth I decided to use ip addr instead host because
we don't want rely on the host header which can be spoofed but another
solution is to resolve the ip addr from the socket to host name.

Example:

```bash
$ polkadot --rpc-rate-limit 10 --rpc-rate-limit-whitelisted-ips 127.0.0.1/8 --rpc-rate-limit-trust-proxy-headers
```

The ip addr is read from the HTTP proxy headers `Forwarded`,
`X-Forwarded-For` `X-Real-IP` if `--rpc-rate-limit-trust-proxy-headers`
is enabled if that is not enabled or the headers are not found then the
ip address is read from the socket.

//cc @BulatSaif can you test this and give some feedback on it?

This checks changed files against API surface changes against what the
prdoc says.

It will error if the detected semver change is greater than the one
listed in the prdoc. It will also error if any crates were touched but
not mentioned in the prdoc.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

silent, because it'll be deployed with the
https://github.com/paritytech/polkadot-sdk/pull/4102, where this code
has been introduced

I've planned originally to avoid doing that check in the runtime code,
because it **may be** checked offchain. But actually, the check is quite
cheap and we could do that onchain too.

Adds an additional extrinsic call to the `pallet-balances` to _burn_
tokens.
Depending on the `keep_alive` flag, the call might or might not reap the
account.

Required modification of the _fungible's_ `Mutate` trait, `burn_from`
function to allow the `Preservation` argument.

**TODO**
- [x] run benchmarks & update weights
- [x] make sure prdoc is required & properly formatted

Related issue: https://github.com/paritytech/polkadot-sdk/issues/3943

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

# Context

Estimating fees for XCM execution and sending has been an area with bad
UX.
The addition of the
[XcmPaymentApi](https://github.com/paritytech/polkadot-sdk/pull/3607)
exposed the necessary components to be able to estimate XCM fees
correctly, however, that was not the full story.
The `XcmPaymentApi` works for estimating fees only if you know the
specific XCM you want to execute or send.
This is necessary but most UIs want to estimate the fees for extrinsics,
they don't necessarily know the XCM program that's executed by them.

# Main addition

A new runtime API is introduced, the `XcmDryRunApi`, that given an
extrinsic, or an XCM program, returns its effects:
- Execution result
- Local XCM (in the case of an extrinsic)
- Forwarded XCMs
- List of events

This API can be used on its own for dry-running purposes, for
double-checking or testing, but it mainly shines when used in
conjunction with the `XcmPaymentApi`.
UIs can use these two APIs to estimate transfers.

# How it works

New tests are added to exemplify how to incorporate both APIs.
There's a mock test just to make sure everything works under
`xcm-fee-payment-runtime-api`.
There's a real-world test using Westend and AssetHubWestend under
`cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`.
Added both a test for a simple teleport between chains and a reserve
transfer asset between two parachains going through a reserve.

The steps to follow:
- Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and
forwarded messages
- For each forwarded message
- Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery
fees
- Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the
remote execution weight
- (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE
DESTINATION to know on which assets the execution fees can be paid
- Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to
convert weight to the actual remote execution fees
- Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new
message will be forwarded, if so, continue

# Dear reviewer

The changes in this PR are grouped as follows, and in order of
importance:
- Addition of new runtime API
- Definition, mock and simple tests:
polkadot/xcm/xcm-fee-payment-runtime-api/*
- Implemented on Westend, Asset Hub Westend and Penpal, will implement
on every runtime in a following PR
- Addition of a new config item to the XCM executor for recording xcms
about to be executed
  - Definition: polkadot/xcm/xcm-executor/*
  - Implementation: polkadot/xcm/pallet-xcm/*
- had to update all runtime xcm_config.rs files with `type XcmRecorder =
XcmPallet;`
- Addition of a new trait for inspecting the messages in queues
  - Definition: polkadot/xcm/xcm-builder/src/routing.rs
  - Implemented it on all routers:
    - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs
- ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on
implementation in cumulus/pallets/parachain-system/src/lib.rs)
    - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs
    - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs
- More complicated and useful tests:
-
cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs

## Next steps

With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have
the new API.
UIs can test on these runtimes to create better experiences around
cross-chain operations.

Next:
- Add XcmDryRunApi to all system parachains
- Integrate xcm fee estimation in all emulated tests
- Get this on the fellowship runtimes

---------

Co-authored-by: Adrian Catangiu <[email protected]>

Addressing comment:
https://github.com/paritytech/polkadot-sdk/pull/3455#issuecomment-2094829076

---------

Co-authored-by: command-bot <>