Glutton currently is useful mostly for stress testing relay chain
validators. It is unusable for testing the collator networking and block
announcement and import scenarios. This PR resolves that by improving
glutton pallet to also buff up the blocks, up to the runtime configured
`BlockLength`.

### How it works
Includes an additional inherent in each parachain block. The `garbage`
argument passed to the inherent is filled with trash data. It's size is
computed by applying the newly introduced `block_length` percentage to
the maximum block length for mandatory dispatch class. After
https://github.com/paritytech/polkadot-sdk/pull/4765 is merged, the
length of inherent extrinsic will be added to the total block proof
size.

The remaining weight is burnt in `on_idle` as configured by the
`storage` percentage parameter.


TODO:
- [x] PRDoc
- [x] Readme update
- [x] Add tests

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Update parachain template pallet based on polkadot sdk docs

@Kianenigma



---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

this pr is a part of #3326

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

The crate `sc-cli` otherwise enables the `rocksdb` feature.

* fix rotko's pcollectives bootnode
* Update people-kusama.json
* Add Dwellir People Kusama bootnode
* add Gatotech bootnodes to `people-kusama`
* Add Dwellir People Kusama bootnode
* Update Amforc bootnodes for Kusama and Polkadot (#4668)

---------

Co-authored-by: RadiumBlock <info@radiumblock.com>
Co-authored-by: Jonathan Udd <jonathan@dwellir.com>
Co-authored-by: Milos Kriz <milos_kriz@hotmail.com>
Co-authored-by: tugy <33746108+tugytur@users.noreply.github.com>
Co-authored-by: Kutsal Kaan Bilgin <kutsalbilgin@gmail.com>
Co-authored-by: Petr Mensik <petr.mensik1@gmail.com>
Co-authored-by: Tommi <tommi@romeblockchain>

remove unnecessary words in comments.

Closes: https://github.com/paritytech/polkadot-sdk/issues/4801

@andreclaro I hope this makes it more clear from the docs directly.

Clobbers the `stable` branch with the `audited` tag as described in the
[RELEASE.md](https://github.com/paritytech/polkadot-sdk/blob/master/docs/RELEASE.md#clobbering).

Example of the `staging_stable` branch now after force-pushing the
`staging_audited` tag for a few times. The `staging_` prefix has now
been removed and should be ready for normal use.
The only trigger is currently manual, but can easily be set to three
months.
![Screenshot 2024-06-13 at 00 47
43](https://github.com/paritytech/polkadot-sdk/assets/10380170/97e070ad-ce2d-4504-83a0-ad6717b6e73e)

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Fix https://github.com/paritytech/polkadot-sdk/issues/4743 which allows
us to remove the defensive limit on pov size in Cumulus after relay
chain gets upgraded with these changes. Also add unit test to ensure
`CheckWeight` - `StorageWeightReclaim` integration works.

TODO:
- [x] PRDoc
- [x] Add a len to all the other tests in storage weight reclaim and
call `CheckWeight::pre_dispatch`

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>

Contains mainly changes/nits/refactors related to the relayer code
(`client-substrate` and `lib-substrate-relay`) migrated from the Bridges
V2 [branch](https://github.com/paritytech/polkadot-sdk/pull/4427).

Relates to:
https://github.com/paritytech/parity-bridges-common/issues/2976
Companion: https://github.com/paritytech/parity-bridges-common/pull/2988


## TODO
- [x] fix comments

## Questions
- [x] Do we need more testing for client V2 stuff? If so, how/what is
the ultimate test? @svyatonik


- [x] check
[comment](https://github.com/paritytech/polkadot-sdk/pull/4494#issuecomment-2117181144)
for more testing

---------

Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Serban Iorga <serban@parity.io>

Related to: https://github.com/paritytech/polkadot-sdk/issues/5

A couple of cosmetics and improvements related to
`polkadot-parachain-bin`:

- Adding some convenience traits in order to avoid declaring long
duplicate bounds
- Specifically check if the runtime exposes `AuraApi` when executing
`start_lookahead_aura_consensus()`
- Some fixes for the `RelayChainCli`. Details in the commits description

This PR backports regular version bumps and prdocs reordering from the
release branch back to master

## TODO

- [x] test/confirm that the release script is ok

---------

Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
Co-authored-by: Egor_P <egor@parity.io>

This PR aligns the settings for `MaxFreezes`, `RuntimeFreezeReason`, and
`FreezeIdentifier`.

#### Future work and improvements
https://github.com/paritytech/polkadot-sdk/issues/2997 (remove
`MaxFreezes` and `FreezeIdentifier`)

After using this tutorial in PBA, there was a few areas to improve it.
Moreover, I have:

- Improve `your_first_pallet`, link it in README, improve the parent
`guide` section.
- Updated the templates page, in light of recent efforts related to in
https://github.com/paritytech/polkadot-sdk/issues/3155
- Added small ref docs about metadata, completed the one about native
runtime, added one about host functions.
- Remove a lot of unfinished stuff from sdk-docs
- update diagram for `Hooks`

Addressing comment:
https://github.com/paritytech/polkadot-sdk/pull/4102/files#r1635502496

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

This branch propose to avoid clones in append by storing offset and size
in previous overlay depth.
That way on rollback we can just truncate and change size of existing
value.
To avoid copy it also means that :

- append on new overlay layer if there is an existing value: create a
new Append entry with previous offsets, and take memory of previous
overlay value.
- rollback on append: restore value by applying offsets and put it back
in previous overlay value
- commit on append: appended value overwrite previous value (is an empty
vec as the memory was taken). offsets of commited layer are dropped, if
there is offset in previous overlay layer they are maintained.
- set value (or remove) when append offsets are present: current
appended value is moved back to previous overlay value with offset
applied and current empty entry is overwrite (no offsets kept).

The modify mechanism is not needed anymore.
This branch lacks testing and break some existing genericity (bit of
duplicated code), but good to have to check direction.

Generally I am not sure if it is worth or we just should favor
differents directions (transients blob storage for instance), as the
current append mechanism is a bit tricky (having a variable length in
first position means we sometime need to insert in front of a vector).

Fix #30.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: EgorPopelyaev <egor@parity.io>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>

## Issue

Currently, syncing parachains from scratch can lead to a very long
finalization time once they reach the tip of the chain. The problem is
that we try to finalize everything from 0 to the tip, which can be
thousands or even millions of blocks.

We finalize sequentially and try to compute displaced branches during
finalization. So for every block on the way, we compute an expensive
tree route.

## Proposed Improvements

In this PR, I propose improvements that solve this situation:

- **Skip tree route calculation if `leaves().len() == 1`:** This should
be enough for 90% of cases where there is only one leaf after sync.
- **Optimize finalization for long distances:** It can happen that the
parachain has imported some leaf and then receives a relay chain
notification with the finalized block. In that case, the previous
optimization will not trigger. A second mechanism should ensure that we
do not need to compute the full tree route. If the finalization distance
is long, we check the lowest common ancestor of all the leaves. If it is
above the to-be-finalized block, we know that there are no displaced
leaves. This is fast because forks are short and close to the tip, so we
can leverage the header cache.

## Alternative Approach

- The problem was introduced in #3962. Reverting that PR is another
possible strategy.
- We could store for every fork where it begins, however sounds a bit
more involved to me.


fixes #4614

... this is superfluous because set_reserved_peers implementation
already calls this method here:

https://github.com/paritytech/polkadot-sdk/blob/cdb297b1

/substrate/client/network/src/protocol_controller.rs#L571,
so the call just ends producing this warnings whenever we manipulate the
peers set.

```
Trying to remove unknown reserved node 12D3KooWRCePWvHoBbz9PSkw4aogtdVqkVDhiwpcHZCqh4hdPTXC from SetId(3)
peerset warnings (from different peers)
```

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

- unit tests for pov-recovery
- elastic scaling support (recovering multiple candidates in a single
relay chain block)
- also some small cleanups
- also switches to candidates_pending_availability in
`handle_empty_block_announce_data`

Fixes https://github.com/paritytech/polkadot-sdk/issues/3577

After https://github.com/paritytech/polkadot-sdk/pull/4097 is merged, we
should also add a zombienet test, similar to the existing
`0002-pov_recovery.toml` but which has a single collator using elastic
scaling on multiple cores.

Add some debug logs to be able to identify the validators and parachains
that have most no-shows, this metric is valuable because it will help us
identify validators and parachains that regularly have this problem.

From the validator_index we can then query the on-chain information and
identify the exact validator that is causing the no-shows.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

- Addresses
[this](https://github.com/paritytech/polkadot-sdk/issues/3155#issuecomment-2126934939).
- Revamps the Readme, very similar to [the minimal
template](https://github.com/paritytech/polkadot-sdk/pull/4649).
- Changed `polkadot-launch` to `zombienet`, with instructions how to run
it.
- See the [rendered
version](https://github.com/paritytech/polkadot-sdk/blob/rzadp/parachain-template-readme/templates/parachain/README.md).

Update README.md to move the PSVM link under a "Tooling" section under the "Releases" section (#4734)

This update implements the suggestion from @Kianenigma mentioned in
https://github.com/paritytech/polkadot-sdk/pull/4718#issuecomment-2153777367

Replaces the "Other useful resources and tooling" section at the bottom
with a new (nicer) "Tooling" section just under the "Releases" section.

Small refactoring PR to improve the readability of the proc macros.
- small improvement in docs
- use new `let Some(..) else` expression
- removed extra indentations by early returns

Discovered during metadata v16 poc, extracted from:
https://github.com/paritytech/polkadot-sdk/pull/4358

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Co-authored-by: gupnik <mail.guptanikhil@gmail.com>

take over #2941 
[Weights
compare](https://weights.tasty.limo/compare?unit=weight&ignore_errors=true&threshold=10&method=asymptotic&repo=polkadot-sdk&old=master&new=pg%2Fwasmi-to-v0.32.0-beta.7&path_pattern=substrate%2Fframe%2F**%2Fsrc%2Fweights.rs%2Cpolkadot%2Fruntime%2F*%2Fsrc%2Fweights%2F**%2F*.rs%2Cpolkadot%2Fbridges%2Fmodules%2F*%2Fsrc%2Fweights.rs%2Ccumulus%2F**%2Fweights%2F*.rs%2Ccumulus%2F**%2Fweights%2Fxcm%2F*.rs%2Ccumulus%2F**%2Fsrc%2Fweights.rs)

---------

Co-authored-by: command-bot <>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Closes: https://github.com/paritytech/parity-bridges-common/issues/2734

Thanks for @xlc for the original seed info, I've just fixed it up a bit
and added example links.

I've moved the comparison between eth-rpc-api and frontier outside, as
it is opinionation. I think the content there was good but should live
in the README of the corresponding repos. No strong opinion, happy
either way.

---------

Co-authored-by: Bryan Chen <xlchen1291@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: command-bot <>

<img width="484" alt="Screenshot 2024-05-30 at 10 30 41"
src="https://github.com/paritytech/polkadot-sdk/assets/5588131/92ece1d0-cda3-46cf-9f46-4026c565b52d">

---------

Co-authored-by: Dónal Murray <donal.murray@parity.io>

Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

... when backing group is of size 1.

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

Adds a link to the [Polkadot SDK Version
Manager](https://github.com/paritytech/psvm) since this tool is not well
known, but very useful for developers using the Polkadot SDK.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Part of https://github.com/paritytech/polkadot-sdk/issues/4324
We don't change but extend the existing cleanup strategy. 
- We still don't touch artifacts being stale less than 24h
- First time we attempt pruning only when we hit cache limit (10 GB)
- If somehow happened that after we hit 10 GB and least used artifact is
stale less than 24h we don't remove it.

---------

Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>

The link to
[releases](https://github.com/paritytech/polkadot-sdk/releases) usually
takes you to a list with a bunch of drafts at the top so you have to
scroll.

[This link](https://github.com/paritytech/polkadot-sdk/releases/latest)
takes you directly to the latest release.

Nobody seems to maintain this and the job is disabled since months. I
think unless the Security team wants to pick this up we delete it for
now.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

`secp256k1 v0.28.0` and `secp256k1-sys v0.9.0` were yanked because
building them fails for `aarch64-apple-darwin` targets.

Use the `secp256k1 v0.28.2` and `secp256k1-sys v0.9.2` patched versions
that build fine on ARM chipset MacOS.

Inherited workspace dependencies cannot be renamed by the crate using
them (see [1](https://github.com/rust-lang/cargo/issues/12546),
[2](https://stackoverflow.com/questions/76792343/can-inherited-dependencies-in-rust-be-aliased-in-the-cargo-toml-file)).
Since we want to use inherited workspace dependencies everywhere, we
first need to unify all aliases that we use for a dependency throughout
the workspace.
The umbrella crate is currently excluded from this procedure, since it
should be able to export the crates by their original name without much
hassle.

For example: one crate may alias `parity-scale-codec` to `codec`, while
another crate does not alias it at all. After this change, all crates
have to use `codec` as name. The problematic combinations were:
- conflicting aliases: most crates aliases as `A` but some use `B`.
- missing alias: most of the crates alias a dep but some dont.
- superfluous alias: most crates dont alias a dep but some do.

The script that i used first determines whether most crates opted to
alias a dependency or not. From that info it decides whether to use an
alias or not. If it decided to use an alias, the most common one is used
everywhere.

To reproduce, i used
[this](https://github.com/ggwpez/substrate-scripts/blob/master/uniform-crate-alias.py)
python script in combination with
[this](https://github.com/ggwpez/zepter/blob/38ad10585fe98a5a86c1d2369738bc763a77057b/renames.json)
error output from Zepter.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

In preparation for launching re-enabling
(https://github.com/paritytech/polkadot-sdk/issues/2418), we need to
adjust the disabling strategy of statement-distribution to use the relay
parent's state instead of the latest state (union of active leaves).
This will also ensure no raciness of getting the latest state vs
accepting statements from disabling validators at the cost of being more
lenient/potentially accepting more statements from disabled validators.

- [x] PRDoc

Follow up on #4414 to clean up the old storage.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>