The block header is required to derive inherents for a relay chain next
block, this is useful in testing environments.

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

Closes: https://github.com/paritytech/polkadot-sdk/issues/4428

As per #3326, removes pallet::getter usage from the pallet
authority-discovery. The syntax `StorageItem::<T, I>::get()` should be
used instead.

cc @muraca

---------

Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

A part of #3326 

Removes all #[pallet::getter] usage from the contracts mock network
pallet. As the storage values were pub(super), read-only visibility was
lost external to the crate upon the removal of the macros. I have
implemented custom getters as a replacement, keeping the api the same.

If we care very much about consistency of the
storagevalue::<T>::get() syntax, the other option would be to set
the storage values to pub. Though I find preserving data authority
better myself.

@muraca

The fact that this takes two sessions to come into effect is not
obvious. Just added some docs to explain that.

Also tidied up uses of "broker chain" -> "coretime chain"

Closes #1961

Disable logging of rustls to get rid off the following log lines:
```
Sending fatal alert BadCertificate
```

Upstream also removed them: https://github.com/rustls/rustls/pull/1278


Closes: https://github.com/paritytech/polkadot-sdk/issues/3252

This PR adds two new CLI options to disable rate limiting for certain ip
addresses and whether to trust "proxy header".
After going back in forth I decided to use ip addr instead host because
we don't want rely on the host header which can be spoofed but another
solution is to resolve the ip addr from the socket to host name.

Example:

```bash
$ polkadot --rpc-rate-limit 10 --rpc-rate-limit-whitelisted-ips 127.0.0.1/8 --rpc-rate-limit-trust-proxy-headers
```

The ip addr is read from the HTTP proxy headers `Forwarded`,
`X-Forwarded-For` `X-Real-IP` if `--rpc-rate-limit-trust-proxy-headers`
is enabled if that is not enabled or the headers are not found then the
ip address is read from the socket.

//cc @BulatSaif can you test this and give some feedback on it?

This checks changed files against API surface changes against what the
prdoc says.

It will error if the detected semver change is greater than the one
listed in the prdoc. It will also error if any crates were touched but
not mentioned in the prdoc.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

silent, because it'll be deployed with the
https://github.com/paritytech/polkadot-sdk/pull/4102, where this code
has been introduced

I've planned originally to avoid doing that check in the runtime code,
because it **may be** checked offchain. But actually, the check is quite
cheap and we could do that onchain too.

Adds an additional extrinsic call to the `pallet-balances` to _burn_
tokens.
Depending on the `keep_alive` flag, the call might or might not reap the
account.

Required modification of the _fungible's_ `Mutate` trait, `burn_from`
function to allow the `Preservation` argument.

**TODO**
- [x] run benchmarks & update weights
- [x] make sure prdoc is required & properly formatted

Related issue: https://github.com/paritytech/polkadot-sdk/issues/3943

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

# Context

Estimating fees for XCM execution and sending has been an area with bad
UX.
The addition of the
[XcmPaymentApi](https://github.com/paritytech/polkadot-sdk/pull/3607)
exposed the necessary components to be able to estimate XCM fees
correctly, however, that was not the full story.
The `XcmPaymentApi` works for estimating fees only if you know the
specific XCM you want to execute or send.
This is necessary but most UIs want to estimate the fees for extrinsics,
they don't necessarily know the XCM program that's executed by them.

# Main addition

A new runtime API is introduced, the `XcmDryRunApi`, that given an
extrinsic, or an XCM program, returns its effects:
- Execution result
- Local XCM (in the case of an extrinsic)
- Forwarded XCMs
- List of events

This API can be used on its own for dry-running purposes, for
double-checking or testing, but it mainly shines when used in
conjunction with the `XcmPaymentApi`.
UIs can use these two APIs to estimate transfers.

# How it works

New tests are added to exemplify how to incorporate both APIs.
There's a mock test just to make sure everything works under
`xcm-fee-payment-runtime-api`.
There's a real-world test using Westend and AssetHubWestend under
`cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs`.
Added both a test for a simple teleport between chains and a reserve
transfer asset between two parachains going through a reserve.

The steps to follow:
- Use `XcmDryRunApi::dry_run_extrinsic` to get local XCM program and
forwarded messages
- For each forwarded message
- Use `XcmPaymentApi::query_delivery_fee` LOCALLY to get the delivery
fees
- Use `XcmPaymentApi::query_xcm_weight` ON THE DESTINATION to get the
remote execution weight
- (optional) Use `XcmPaymentApi::query_acceptable_payment_assets` ON THE
DESTINATION to know on which assets the execution fees can be paid
- Use `XcmPaymentApi::query_weight_to_asset_fee` ON THE DESTINATION to
convert weight to the actual remote execution fees
- Use `XcmDryRunApi::dry_run_xcm` ON THE DESTINATION to know if a new
message will be forwarded, if so, continue

# Dear reviewer

The changes in this PR are grouped as follows, and in order of
importance:
- Addition of new runtime API
- Definition, mock and simple tests:
polkadot/xcm/xcm-fee-payment-runtime-api/*
- Implemented on Westend, Asset Hub Westend and Penpal, will implement
on every runtime in a following PR
- Addition of a new config item to the XCM executor for recording xcms
about to be executed
  - Definition: polkadot/xcm/xcm-executor/*
  - Implementation: polkadot/xcm/pallet-xcm/*
- had to update all runtime xcm_config.rs files with `type XcmRecorder =
XcmPallet;`
- Addition of a new trait for inspecting the messages in queues
  - Definition: polkadot/xcm/xcm-builder/src/routing.rs
  - Implemented it on all routers:
    - ChildParachainRouter: polkadot/runtime/common/src/xcm_sender.rs
- ParentAsUmp: cumulus/primitives/utility/src/lib.rs (piggybacked on
implementation in cumulus/pallets/parachain-system/src/lib.rs)
    - XcmpQueue: cumulus/pallets/xcmp-queue/src/lib.rs
    - Bridge: bridges/modules/xcm-bridge-hub-router/src/lib.rs
- More complicated and useful tests:
-
cumulus/parachains/integration-tests/emulated/tests/assets/asset-hub-westend/src/tests/xcm_fee_estimation.rs

## Next steps

With this PR, Westend, AssetHubWestend, Rococo and AssetHubRococo have
the new API.
UIs can test on these runtimes to create better experiences around
cross-chain operations.

Next:
- Add XcmDryRunApi to all system parachains
- Integrate xcm fee estimation in all emulated tests
- Get this on the fellowship runtimes

---------

Co-authored-by: Adrian Catangiu <[email protected]>

Addressing comment:
https://github.com/paritytech/polkadot-sdk/pull/3455#issuecomment-2094829076

---------

Co-authored-by: command-bot <>

Mention that offenders are no longer chilled and suggest node operators
and nominators to monitor their nodes/nominees closely.

---------

Co-authored-by: Maciej <[email protected]>

Added manual jobs for code coverage (triggered via `codecov-start` job):
 - **codecov-start** - initialize Codecov report for commit/pr
- **test-linux-stable-codecov** - perform `nextest run` and upload
coverage data parts
- **codecov-finish** - finalize uploading of data parts and generate
Codecov report

Coverage requires code to be built with `-C instrument-coverage` which
causes build errors (e .g. ```error[E0275]: overflow evaluating the
requirement `<mock::Test as pallet::Config>::KeyOwnerProof == _\` ```,
seems like related to
[2641](https://github.com/paritytech/polkadot-sdk/issues/2641)) and
unstable tests behavior
([example](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6004731)).
This is where we'll nee the developers assistance

closing [[polkadot-sdk] Add code coverage
#902](https://github.com/paritytech/ci_cd/issues/902)

Generated with the script from
https://github.com/paritytech/polkadot-sdk/pull/2931

Edit: previously linked to a very similar PR

Closes: https://github.com/paritytech/polkadot-sdk/issues/4003 (please
see for the problem description)

## TODO
- [x] add more tests covering `WrapVersion` corner cases (e.g. para has
lower version, ...)
- [x] regenerate benchmarks `runtime_parachains::hrmp` (fix for Rococo
is here: https://github.com/paritytech/polkadot-sdk/pull/4332)

## Questions / possible improvements
- [ ] A `WrapVersion` implementation for `pallet_xcm` initiates version
discovery with
[note_unknown_version](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/pallet-xcm/src/lib.rs#L2527C5-L2527C25),
there is possibility to avoid this overhead in this HRMP case to create
new `WrapVersion` adapter for `pallet_xcm` which would not use
`note_unknown_version`. Is it worth to do it or not?
- [ ] There's a possibility to decouple XCM functionality from the HRMP
pallet, allowing any relay chain to generate its own notifications. This
approach wouldn't restrict notifications solely to the XCM. However,
it's uncertain whether it's worthwhile or desirable to do so? It means
making HRMP pallet more generic. E.g. hiding HRMP notifications behind
some trait:
	```
	trait HrmpNotifications {

		fn on_channel_open_request(
			sender: ParaId,
			proposed_max_capacity: u32,
			proposed_max_message_size: u32) -> primitives::DownwardMessage;

fn on_channel_accepted(recipient: ParaId) ->
primitives::DownwardMessage;

fn on_channel_closing(initiator: ParaId, sender: ParaId, recipient:
ParaId) -> primitives::DownwardMessage;
	}
	```
and then we could have whatever adapter, `impl HrmpNotifications for
VersionedXcmHrmpNotifications {...}`,
	```
	impl parachains_hrmp::Config for Runtime {
	..
		type HrmpNotifications = VersionedXcmHrmpNotifications;
	..
	}
	```

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <[email protected]>

Improve Create release draft workflow + templates for the free notes and docker images sections in the notes (#4371)

This PR has the following changes:

- New templates for the free notes and docker images sections in the
release notes. There is going to be a section for the manual additions
to the release notes + a section with the links to the docker images for
`polkadot` and `polkadot-parachain` binaries at the end of the release
draft.
- Fix for matrix section in the Create release draft flow (adds the
release environment variable)
- Reduction of the message which is posted to the announcement chats, as
the current one with the full release notes text is too big.

Split from #4374

This PR helps to reduce dependencies and align versions, which would
help to move them to workspace dep

Needed for: https://github.com/polkadot-fellows/runtimes/issues/248

This PR introduces a new type `TypeWithDefault<T, D: Get<T>>` to be able
to provide a custom default for any type. This can, then, be used to
provide the nonce type that returns the current block number as the
default, to avoid replay of immortal transactions.

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

When a staker unbonds and withdraws, it is possible that their stash
will contain less currency than the existential deposit. If that
happens, their stash is reaped. But if the existential deposit is zero,
the reap is not triggered. This PR adjusts `pallet_staking` to reap a
stash in the special case that the stash value is zero and the
existential deposit is zero.

This change is important for blockchains built on substrate that require
an existential deposit of zero, becuase it conserves valued storage
space.

There are two places in which ledgers are checked to determine if their
value is less than the existential deposit and they should be reaped: in
the methods `do_withdraw_unbonded` and `reap_stash`. When the check is
made, the condition `ledger_total == 0` is also checked. If
`ledger_total` is zero, then it must be below any existential deposit
greater than zero and equal to an existential deposit of 0.

I added a new test for each method to confirm the change behaves as
expected.

Closes https://github.com/paritytech/polkadot-sdk/issues/4340

---------

Co-authored-by: command-bot <>

Related to
https://github.com/paritytech/parity-bridges-common/issues/2962
Relay companion:
https://github.com/paritytech/parity-bridges-common/pull/2978

Example usage:
```
./target/release/substrate-relay relay-parachain-head rococo-to-bridge-hub-westend \
    --source-host localhost --source-port 9942 \
    --target-host localhost --target-port 8945 --target-signer //Alice \
    --at-relay-block 61
```

Due to recent bump of Rococo/Westend versions + the fact that
https://github.com/paritytech/parity-bridges-common/pull/2894 has
finally reached this repo, tests now fail, because we've started
checking all client versions (even source) unless we specify
`--source-version-mode Auto` in CLI arguments. This looks like an
overkill, but all those version checks will be fixed by
https://github.com/paritytech/polkadot-sdk/pull/4256, so now it makes
sense just to add this CLI option. We also need to propagate it to
running relays eventually.

cc https://github.com/paritytech/ci_cd/issues/939

Allow for 0 existential deposit in benchmarks for `pallet_staking`, `pallet_session`, and `pallet_balances` (#4346)

This PR ensures non-zero values are available in benchmarks for
`pallet_staking`, `pallet_session`, and `pallet_balances` where required
for them to run.

This small change makes it possible to run the benchmarks for
`pallet_staking`, `pallet_session`, and `pallet_balances` in a runtime
for which existential deposit is set to 0.

The benchmarks for `pallet_staking` and `pallet_session` will still fail
in runtimes that use `U128CurrencyToVote`, but that is easy to work
around by creating a new `CurrencyToVote` implementation for
benchmarking.

The changes are implemented by checking if existential deposit equals 0
and using 1 if so.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <[email protected]>

The native executor is deprecated and downstream users should stop using
it.

This PR:
- changes `xcm::v4` to `xcm::prelude` imports for coretime stuff
- changes `query_acceptable_payment_assets` /
`query_weight_to_asset_fee` implementations to be more resilient to the
XCM version change
- adds `xcm_fee_payment_runtime_api::XcmPaymentApi` to the
AssetHubRococo/Westend exposing a native token as acceptable payment
asset

Continuation of: https://github.com/paritytech/polkadot-sdk/pull/3607

Closes: https://github.com/paritytech/polkadot-sdk/issues/4297

## Possible follow-ups

- [ ] add all sufficient assets (`Assets`, `ForeignAssets`) as
acceptable payment assets ?

This crate uses this feature however it appears to still work without
this feature enabled. I believe this is due to feature unification of
the workspace. Some other crate enables this feature so it also ends up
enabled here. But when this crate is pushed to crates.io and compiled
individualy it fails to compile.

use `server-core` feature instead of `server` feature when defining the
rpc api

This adds a small guide on how to prepare your parachain for pov-reclaim
usage.

This PR updates the litep2p crate to the latest version.

This fixes the build for developers that want to perform `cargo update`
on all their dependencies:
https://github.com/paritytech/polkadot-sdk/pull/4343, by porting the
latest changes.

The peer records were introduced to litep2p to be able to distinguish
and update peers with outdated records.
It is going to be properly used in substrate via:
https://github.com/paritytech/polkadot-sdk/pull/3786, however that is
pending the commit to merge on litep2p master:
https://github.com/paritytech/litep2p/pull/96.

Closes: https://github.com/paritytech/polkadot-sdk/pull/4343

---------

Signed-off-by: Alexandru Vasile <[email protected]>

see https://github.com/paritytech/parity-bridges-common/issues/2974 : we
still need to support unupgraded chains (BHK and BHP) in relay

We may need to revert this change when all chains are upgraded

Closes #4272.

It turned out that not only the node part but also the runtime wasn't
async backing ready. Now, both of them are using proper APIs and
producing 6-second blocks out of the box.