A follow-up to https://github.com/paritytech/polkadot-sdk/pull/5447

Instead of having an option to select a PR template (like is the case
with issues), this change will make the one PR template we have the
default, it will show up for every new PR.

Also updated one link so it works properly in the PR body.

This can make the log cleaner, especially when you specify `--log
sync=debug`.

After seeing some cases of reported changes that did not happen by the
merge request proposer (like
https://github.com/paritytech/polkadot-sdk/pull/5339), it became clear
that [this](https://github.com/orgs/community/discussions/59677) is
probably the issue.
The base commit of the SemVer check CI is currently using the *latest*
master commit, instead of the master commit at the time when the MR was
created.

Trying to get the correct base commit now. For this to be debugged, i
have to wait until another MR is merged into master.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

As I was looking at the coupling between `SyncingEngine`,
`SyncingStrategy` and individual strategies I noticed a few things that
were unused, redundant or awkward.

The awkward change comes from
https://github.com/paritytech/substrate/pull/13700 where
`num_connected_peers` property was added to `SyncStatus` struct just so
it can be rendered in the informer. While convenient, the property
didn't really belong there and was annoyingly set to `0` in some
strategies and to `num_peers` in others. I have replaced that with a
property on `SyncingService` that already stored necessary information
internally.

Also `ExtendedPeerInfo` didn't have a working `Clone` implementation due
to lack of perfect derive in Rust and while I ended up not using it in
the refactoring, I included fixed implementation for it in this PR
anyway.

While these changes are not strictly necessary for
https://github.com/paritytech/polkadot-sdk/issues/5333, they do reduce
coupling of syncing engine with syncing strategy, which I thought is a
good thing.

Reviewing individual commits will be the easiest as usual.

---------

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

This PR adds possibility to set BUILD_OPTIONS to the "Srtool Build" step
in the release pipeline while building runtimes.

Colses: https://github.com/paritytech/release-engineering/issues/213

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: EgorPopelyaev <EgorPopelyaev@users.noreply.github.com>

Introduce `MaybeConsideration` extension trait for `Consideration`.

The trait allows for the management of tickets that may represent no
cost. While the `MaybeConsideration` still requires proper handling, it
introduces the ability to determine if a ticket represents no cost and
can be safely forgotten without any side effects.

The new trait is particularly useful when a consumer expects the cost to
be zero under certain conditions (e.g., when the proposal count is below
a threshold N) and does not want to store such consideration tickets in
storage. The extension approach allows us to avoid breaking changes to
the existing trait and to continue using it as a non-optional version
for migrating pallets that utilize the `Currency` and `fungible` traits
for `holds` and `freezes`, without requiring any storage migration.

# Description

Moves `create_inherent_data_provider` after checking if major sync is in
progress.

## Integration

Change is internal to sc-consensus-slots. It should be no-op unless
someone is using fork of this SDK.

## Review Notes

Motivation for this change is to avoid calling
`create_inherent_data_providers` if it's result is going to be discarded
anyway during major sync. This has potential to speed up node operations
during major sync by not calling possibly expensive
`create_inherent_data_provider`.

TODO: labels T0-node D0-simple
TODO: there is no tests for `Slots`, should I add one for this case?

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
* External contributors: ask maintainers to put the right label on your
PR.
* [ ] I have made corresponding changes to the documentation (if
applicable)
* [ ] I have added tests that prove my fix is effective or that my
feature works (if applicable)

Addresses
[this](https://github.com/paritytech/eng-automation/issues/10#issuecomment-2286391379).

> Another side note is the checkmarks in
https://github.com/paritytech/polkadot-sdk/community. It seems like even
though we have `CONTRIBUTING.md` in `/docs` it is not picked up in this
list. I am not even sure what is the benefit of following these
standards, but it is probably for the best to do it.

The `pallet-collator-selection` is not correctly using the weight for
the
[new_session](https://github.com/blockdeep/pallet-collator-staking/blob/main/src/benchmarking.rs#L350-L353)
function.

The first parameter is the removed candidates, and the second one the
original number of candidates before the removal, but both values are
swapped.

Needed it downstream

`validity_vote` function is also called from `import_statement` not only
from `import_candidate`. The proof still holds if we assume seconded
statements always come before valid statements, which should be the case
but is not actually enforced by this module at all. On top of this the
proof is not local.

Co-authored-by: eskimor <eskimor@no-such-url.com>

This PR untangles syncing metrics and makes them reactive, the way
metrics are supposed to be in general.

Syncing metrics were bundled in a way that caused coupling across
multiple layers: justifications metrics were defined and managed by
`ChainSync`, but only updated periodically on tick in `SyncingEngine`,
while actual values were queried from `ExtraRequests`. This convoluted
architecture was hard to follow when I was looking into
https://github.com/paritytech/polkadot-sdk/issues/5333.

Now metrics that correspond to each component are owned by that
component and updated as changes are made instead of on tick every
1100ms.

This does add some annoying boilerplate that is a bit harder to
maintain, but it separates metrics more nicely and if someone queries
them more frequently will give arbitrary resolution. Since metrics
updates are just atomic operations I do not expect any performance
impact of these changes.

Will add prdoc if changes look good otherwise.

P.S. I noticed that importing requests (and corresponding metrics) were
not cleared ever since corresponding code was introduced in
https://github.com/paritytech/polkadot-sdk/commit/dc41558b#r145518721
and I left it as is to not change the behavior, but it might be
something worth fixing.

cc @dmitry-markin



---------

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

(Please, do not merge until SA, reverted and restored of
https://github.com/paritytech/polkadot-sdk/pull/4944)

Original PR with more context:
https://github.com/paritytech/parity-bridges-common/pull/2211
Relates to:
https://github.com/paritytech/parity-bridges-common/issues/2210

## TODO

- [x] fresh weighs for `pallet_bridge_messages`
- [x] add `try_state` for `pallet_bridge_messages` which checks for
unpruned messages - relates to the
[comment](https://github.com/paritytech/parity-bridges-common/pull/2211#issuecomment-1643224831)
- [x] ~prepare migration, that prunes leftovers, which would be pruned
eventually from `on_idle` the
[comment](https://github.com/paritytech/parity-bridges-common/pull/2211#issuecomment-1643224831)~
can be done also by `set_storage` / `kill_storage` or with
`OnRuntimeUpgrade` implementatino when `do_try_state_for_outbound_lanes`
detects problem.

## Open question

- [ ] Do we really need `oldest_unpruned_nonce` afterwards?
- after the runtime upgrade and when `do_try_state_for_outbound_lanes`
pass, we won't need any migrations here
    - we won't even need `do_try_state_for_outbound_lanes`
- please check comments bellow:
https://github.com/paritytech/polkadot-sdk/pull/4944#discussion_r1666737961

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>

Following this:
https://github.com/paritytech/polkadot-sdk-parachain-template/pull/11

---------

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

## Context

Currently, many crates have no readme files, even though they are public
and available on crates.io.
Others have just a couple of words that does not look super presentable.

Even though probably nobody starts a journey with `polkadot-sdk` or its
documentation with a readme of a random low-level crate, I think it
would look more mature to have a little better readmes there.

So, in an attempt to improve [the
aesthetics](https://github.com/paritytech/eng-automation/issues/10) of
`polkadot-sdk`, I propose a set of consistent, branded, better-looking
readmes for all published crates.

## What's inside

- ~~New readme files for published crates.~~
- A python script to generate new readmes, for the crates that have
none.
  - It will skip crates that do have a readme, and private crates.
- Added a new image asset to the repo - logo with a background.
- The main readme of the repo uses a [nice
trick](https://github.com/paritytech/polkadot-sdk/blob/ce6938ae/README.md?plain=1#L4-L5)
to accompany both light and dark modes - but that doesn't work on
`crates.io` so a single logo with a background is needed.

## Example

### Current

![Screenshot 2024-08-04 at 16 13
36](https://github.com/user-attachments/assets/3ae0881d-0f40-4614-9d2c-3c95029f0820)

### Changed

![Screenshot 2024-08-04 at 16 12
28](https://github.com/user-attachments/assets/fa7eadc8-aec8-4f77-84d9-54d63ce189cd)

This is a heavily modified and stripped down version of
`pallet_contracts`. We decided to fork instead of extend the old pallet.
Reasons for that are:

- There is no benefit of supporting both on the same pallet as the
intended payload for the new pallet (recompiled YUL) will be using a
different ABI.
- It is much easier since it allows us to remove all the code that was
necessary to support Wasm and focus fully on running cross compiled YUL
contracts.

**The code is reviewable but can't be merged because it depends on an
unreleased version of PolkaVM via git.**

## Current state

All tests are passing and the code is not quick and dirty but written to
last. The work is not finished, though. It is included in the
`kitchensink-runtime` and a node can be built. However, we merge early
in order to be able to start testing other components as early as
possible.

Outstanding changes are tracked here and will be merged separately:
https://github.com/paritytech/polkadot-sdk/issues/5308

## Syscall Interface

The syscall interface is best explored by generating the docs of this
crate and looking at the `SyscallDoc` trait. Arguments are passed in
registers a0-a5 in the order they are listed. If there are more than 6
arguments (call, instantiate) a pointer to a packed struct of the
arguments is expected as the only argument. I plan to create variants of
those syscalls with less arguments specifically for YUL.

Functions are just referenced by their name as ASCII within the PolkaVM
container. Rather than by a syscall number as it was the case in the
last implementation.
 

## Changes vs. `pallet_contracts`

The changes are too numerous to list them all here. This is an
incomplete list:

- Use PolkaVM instead of wasmi to execute contracts
- Made Runtime generic over a new `Memory` trait as we can't map memory
directly on PolkaVM anymore
- No static verification on code upload. Everything is a determinstic
runtime failure
- Removed all migrations and reset the pallet version
- Removed the nonce storage item and instead use the deployers account
nonce to generate a unique trie
- We now bump the deployers account nonce on contract instantiation to
they are bumped even within a batch transaction
- Removed the instantiation nonce host function: We should add a new
`instantiate` variant as a replacement for thos
- ContractInfoOf of uses the indentity hasher now
- Remove the determinism feature: User of that feature should switch to
soft floats
- The `unstable` attribute has been replaced by a `api_version`
attribute to declare at which version an API became available
	- leaving out that attribute makes the API effectively unstable
- a new `api_version` field on the CodeInfo makes sure that old
contracts can't access new APIs (necessary due to lack of static
verification.
- Added a `behaviour_version` field to CodeInfo that can used if we need
to introduce breaking changes and keep the old behaviour for existing
contracts
- Unified storage vs. transient and fixed vs. variable sized keys all
into one set of multiplexing host functions
- Removed all contract observeable limits from the `Config` trait and
instead hardcode them
- Removed the Schedule
- Removed all deprecated host functions
- Simplify chain extension as preperation for making it a pre-compile

---------

Co-authored-by: command-bot <>

# Description
Updates `template.rs` to reflect the two OZ templates available and a
short description

# Checklist

* [x] My PR includes a detailed description as outlined in the
"Description" and its two subsections above.
* [x] My PR follows the [labeling requirements](CONTRIBUTING.md#Process)
of this project (at minimum one label for `T`
  required)
* External contributors: ask maintainers to put the right label on your
PR.
* [x] I have made corresponding changes to the documentation (if
applicable)

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

I'm not sure if this is exactly what
https://github.com/paritytech/polkadot-sdk/issues/3537 meant, but I
think it should be fine to wait for relay chain before initializing
parachain node fully, which removed the need for background task and
extra hacks throughout the stack just to know where warp sync should
start.

Previously there were both `WarpSyncParams` and `WarpSyncConfig`, but
there was no longer any point in having two data structures, so I
simplified it to just `WarpSyncConfig`.

Fixes https://github.com/paritytech/polkadot-sdk/issues/3537

There are numerous reasons for invalid imports, most of them would
likely be caused by bugs. On the other side, dispute distribution
handles all connections fairly, thus there is little harm in keeping a
problematic connection open.

---------

Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: ordian <write@reusable.software>

This PR backports regular version bumps and `prdoc` reorganisation from
the `stable2407` release branch to master

Add the Polkadot Coretime chain-spec to the directory with the other
system chain-specs.

This is the chain-spec used at genesis and for which the genesis head
data was generated.

It is also included in the assets for fellowship [release
v1.3.0](https://github.com/polkadot-fellows/runtimes/releases/tag/v1.3.0)

This PR aims to make the logging from the peer store a bit more clear.

In the past, we aggressively produced warning logs from the peer store
component, even in cases where the reputation change was not malicious.
This has led to an extensive number of logs, as well to node operator
confusion.

In this PR, we produce a warning message if:
- The peer crosses the banned threshold for the first time. This is the
actual reason of a ban
- The peer misbehaves again while being banned. This may happen during a
batch peer report

cc @paritytech/networking 

Part of: https://github.com/paritytech/polkadot-sdk/issues/5379.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>

Just fixing the beefy primitives link to the right one after moving
these primitives to `consensus`

Related to https://github.com/paritytech/polkadot-sdk/issues/5210

Migrated the following scripts to GHA
- test-doc
- test-rustdoc
- build-rustdoc
- build-implementers-guide
- publish-rustdoc (only runs when `master` is modified)

Resolves paritytech/ci_cd#1016

---

Some questions I have:
- Should I remove the equivalent scripts from the `gitlab-ci` files?

---------

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>

Fixes https://github.com/paritytech/polkadot-sdk/issues/4309

If a new block is generated between these 2 lines:

```
  const proof = await apis[nodeName].rpc.mmr.generateProof([1, 9, 20]);

  const root = await apis[nodeName].rpc.mmr.root()
```

we will try to verify a proof for the previous block with the mmr root
at the current block. Which will fail.

So we generate the proof and get the mmr root at block 21 for
consistency.

We preallocated the approvals field in the ApprovalEntry by up to a
factor of two in the worse conditions, since we can't have more than 6
approvals and candidates.len() will return 20 if you have just the 20th
bit set.
This adds to a lot of wasted memory because we have an ApprovalEntry for
each assignment we received

This was discovered while running rust jemalloc-profiling with the steps
from here: https://www.magiroux.com/rust-jemalloc-profiling/

Just with this optimisation approvals subsystem-benchmark memory usage
on the worst case scenario is reduced from 6.1GiB to 2.4 GiB, even cpu
usage of approval-distribution decreases by 4-5%.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

Backport fixes from
https://github.com/polkadot-fellows/runtimes/pull/426

This PR fixes the issue with the publishing flow of the
`chain-speck-builder` image
Closes: https://github.com/paritytech/release-engineering/issues/219

## Before

<img width="320px"
src="https://github.com/user-attachments/assets/27c4c50b-632b-4a3b-a154-0e828a1ac650"/>

## After

<img width="320px"
src="https://github.com/user-attachments/assets/314954fa-7f49-415d-82dd-a9a99983a283"/>

The CI isn't happy with the amount of output:
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7035621/raw

---------

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

There was no need for it to be `&mut self` since block import can happen
concurrently for different blocks and in many cases it was `&mut Arc<dyn
BlockImport>` anyway :man_shrugging:

Similar in nature to
https://github.com/paritytech/polkadot-sdk/pull/4844

Fix https://github.com/paritytech/polkadot-sdk/issues/5184

`owner` can set himself as a `freezer` and freeze the asset so
requirement is not really needed. And requirement is not implemented.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

This removes the workspace of the minimal template, which (I think) is
redundant. The other two templates do not have such a workspace.

The synchronized template created [it's own
workspace](https://github.com/paritytech/polkadot-sdk-minimal-template/blob/master/Cargo.toml)
anyway, and the new readme replaced the old docs contained in `lib.rs`.

Closes
https://github.com/paritytech/polkadot-sdk-minimal-template/issues/11

Silent because the crate was private.

After

https://github.com/paritytech/polkadot-sdk/commit/0636ffdc


approval-distribution did not terminate anymore if Conclude signal was
received.

This should have been caught by the subsystem tests, but it wasn't
because the subsystem is also exiting on error when the channels are
dropped so the test overseer was dropped which made the susbystem exit
and masked the problem.

This pr fixes both the test and the subsystem.

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

Add more logs in `is_potential_spam` revealing why a statement was
marked as a spam.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

After https://github.com/paritytech/polkadot-sdk/pull/4129, a zombienet
bridge test was broken.
This is because XCMv4 locations have an array in the `interior` field,
which also has to appear in PJS.

---------

Co-authored-by: Serban Iorga <serban@parity.io>

- Typos.
- Those telemetry links like https://telemetry.polkadot.io/#list/Kusama
didn't seem to properly point to a proper list (anymore?) - updated
them.
- Also looks like it was trying to use rust-style linking instead of
markdown linking, changed that.
- Relative links do not work on crates.io - updated to absolute,
similarly as some already existing links, such as contribution
guidelines.

..without needing to provide your own `newtype` around it.  

This will allow `type Reason = u8` to be used as `FreezeReason` and
`HoldReason`, which I think is a nice simplification if one doens't want
to deal with the complications.

At the same time, it is a bit of an anti-pattern.

Putting it out there to check people's vibes.