Just the upper-stream for
https://github.com/Snowfork/polkadot-sdk/pull/137

 and more context
there.

---------

Co-authored-by: Clara van Staden <[email protected]>
Co-authored-by: Adrian Catangiu <[email protected]>

Closes https://github.com/paritytech/polkadot-sdk/issues/1966,
https://github.com/paritytech/polkadot-sdk/issues/1963 and
https://github.com/paritytech/polkadot-sdk/issues/1962.

Disabling strategy specification
[here](https://github.com/paritytech/polkadot-sdk/pull/2955). (Updated
13/02/2024)

Implements:
* validator disabling for a whole era instead of just a session
* no more than 1/3 of the validators in the active set are disabled
Removes:
* `DisableStrategy` enum - now each validator committing an offence is
disabled.
* New era is not forced if too many validators are disabled.

Before this PR not all offenders were disabled. A decision was made
based on [`enum
DisableStrategy`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/primitives/staking/src/offence.rs#L54).
Some offenders were disabled for a whole era, some just for a session,
some were not disabled at all.

This PR changes the disabling behaviour. Now a validator committing an
offense is disabled immediately till the end of the current era.

Some implementation notes:
* `OffendingValidators` in pallet session keeps all offenders (this is
not changed). However its type is changed from `Vec<(u32, bool)>` to
`Vec<u32>`. The reason is simple - each offender is getting disabled so
the bool doesn't make sense anymore.
* When a validator is disabled it is first added to
`OffendingValidators` and then to `DisabledValidators`. This is done in
[`add_offending_validator`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/slashing.rs#L325)
from staking pallet.
* In
[`rotate_session`](https://github.com/paritytech/polkadot-sdk/blob/bdbe98297032e21a553bf191c530690b1d591405/substrate/frame/session/src/lib.rs#L623)
the `end_session` also calls
[`end_era`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/pallet/impls.rs#L490)
when an era ends. In this case `OffendingValidators` are cleared
**(1)**.
* Then in
[`rotate_session`](https://github.com/paritytech/polkadot-sdk/blob/bdbe98297032e21a553bf191c530690b1d591405/substrate/frame/session/src/lib.rs#L623)
`DisabledValidators` are cleared **(2)**
* And finally (still in `rotate_session`) a call to
[`start_session`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/pallet/impls.rs#L430

)
repopulates the disabled validators **(3)**.
* The reason for this complication is that session pallet knows nothing
abut eras. To overcome this on each new session the disabled list is
repopulated (points 2 and 3). Staking pallet knows when a new era starts
so with point 1 it ensures that the offenders list is cleared.

---------

Co-authored-by: ordian <[email protected]>
Co-authored-by: ordian <[email protected]>
Co-authored-by: Maciej <[email protected]>
Co-authored-by: Gonçalo Pestana <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Ankan <[email protected]>

Currently we discard errors content
We should at least log it.

Code now is more similar to what is written in try_on_runtime_upgrade.

label should be R0

---------

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Javier Bullrich <[email protected]>

related to
https://github.com/paritytech/parity-bridges-common/issues/2962


silent, because the actual code for subcommand is added in the
`parity-bridges-common` repo, where binary lives

---------

Co-authored-by: Adrian Catangiu <[email protected]>

This PR includes two changes:
- added `workflow_dispatch` to review bot
- reverted #4271

### Added `workflow_dispatch` to review bot
This allows us, in the case that review-bot fails for some fork reasons,
to trigger it manually ensuring that we can overcame the problem with
the multiple actions while we look for a solution.

<img width="342" alt="image"
src="https://github.com/paritytech/polkadot-sdk/assets/8524599/f432f91b-829a-4da4-b4ca-54cc4fe280c8">

### Reverted #4271
Unfortunately, the changes added in #4271 do not work in forks.

Here is a lengthy discussion of many individuals facing the same problem
as me:
- [GitHub Action `pull_request` attribute empty in `workflow_run` event
object for PR from forked repo
#25220](https://github.com/orgs/community/discussions/25220)

So I had to revert it (but I updated the dependencies to latest).

#### Miscellaneous changes

I added a debug log at the end of review bot in case it fails so we can
easily debug it without having to make a lot of boilerplate and forks to
duplicate the environment.

TODOs:
- [x] wait and see if test `1` works;
- [x] ~think of whether we need remaining tests.~ I think we should keep
it - will try to revive and update it

There are some accounts that do not have a consumer ref while having a
reserve.
This adds a fail-safe mechanism to trigger in the case that
`does_consume` is true, but the assumption of `consumer>0` is not.

This should prevent those accounts from loosing balance and the TI from
getting messed up even more, but is not an "ideal" fix. TBH an ideal fix
is not possible, since on-chain data is in an invalid state.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

The solochain template links to parachain template instead of solochain.

Should help us to understand more what's happening between individual
runs and possibly adjust the number of runs

Makes https://github.com/paritytech/polkadot-sdk/pull/4035 easier to
review

Currently, PVFs are re-prepared if any execution environment parameter
changes. As we've recently seen on Kusama and Polkadot, that may lead to
a severe finality lag because every validator has to re-prepare every
PVF. That cannot be avoided altogether; however, we could cease
re-preparing PVFs when a change in the execution environment can't lead
to a change in the artifact itself. For example, it's clear that
changing the execution timeout cannot affect the artifact.

In this PR, I'm introducing a separate hash for the subset of execution
environment parameters that changes only if a preparation-related
parameter changes. It introduces some minor code duplication, but
without that, the scope of changes would be much bigger.

TODO:
- [x] Add a test to ensure the artifact is not re-prepared if
non-preparation-related parameter is changed
- [x] Add a test to ensure the artifact is re-prepared if a
preparation-related parameter is changed
- [x] Add comments, warnings, and, possibly, a test to ensure a new
parameter ever added to the executor environment parameters will be
evaluated by the author of changes with respect to its artifact
preparation impact and added to the new hash preimage if needed.

Closes #4132

... a few sessions too late :(, this already happened on polkadot, so as
of now there are no known relay-chains without async backing enabled in
runtime, but let's fix it in case someone else wants to repeat our
steps.

Fixes: https://github.com/paritytech/polkadot-sdk/issues/4226



---------

Signed-off-by: Alexandru Gheorghe <[email protected]>

Changes:
- Add new error variant `ProcessMessageError::StackLimitReached` and
treat XCM error `ExceedsStackLimit` as such.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>

See 
https://github.com/paritytech/ink/pull/1912
https://github.com/paritytech/ink-docs/pull/338

Original PR:
https://github.com/paritytech/parity-bridges-common/pull/2884

. Since
chain-specific code lives in the `parity-bridges-common` repo, some
parts of original PR will require another PR

---------

Co-authored-by: Adrian Catangiu <[email protected]>

supersedes https://github.com/paritytech/parity-bridges-common/pull/2873

Draft because of couple of TODOs:
- [x] fix remaining TODOs;
- [x] double check that all changes from
https://github.com/paritytech/parity-bridges-common/pull/2873 are
correctly ported;
- [x] create a separate PR (on top of that one or a follow up?) for
https://github.com/paritytech/polkadot-sdk/tree/sv-try-new-bridge-fees;


- [x] fix compilation issues (haven't checked, but there should be
many).

---------

Co-authored-by: Adrian Catangiu <[email protected]>

Revert "pallet-xcm: Deprecate `execute` and `send` in favor of
`execute_blob` and `send_blob` (#3749)"

This reverts commit feee773d

.

---------

Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: Javier Bullrich <[email protected]>

It seems that `review-trigger` is not uploading the artifact that is
used by `review-bot`, so I changed the PR-Number to be obtained by the
previous event that triggered this action.

I also took the liberty to replace `tibdex/github-app-token` for
`actions/create-github-app-token` which is GitHub's official app.

This PR:
- introduces `AllowHrmpNotificationsFromRelayChain` barrier for allowing
HRMP notifications just from the relay chain (to fulfill safety
assumptions -
[see](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/src/v4/mod.rs#L532))
- sets it up for all testnet SP parachains

Continuation of: https://github.com/paritytech/polkadot-sdk/pull/3696

Trying to split parts of the
https://github.com/paritytech/polkadot-sdk/pull/1903 into smaller PRs

For https://github.com/paritytech/polkadot-sdk/pull/1903 it would help
if `verify_with_validator_set()` returned the list of valid
authority-signatures pairs, since after the verification we need to send
them in the equivocation proof.

Seems like Externalities already [return a
vector](https://github.com/paritytech/polkadot-sdk/blob/ffbce2a817ec2e7c8b7ce49f7ed6794584f19667/substrate/primitives/externalities/src/lib.rs#L86

),
so calling `to_vec` on a vector just results in an unneeded copying.

Co-authored-by: Liam Aharon <[email protected]>

Change `transfer_assets_using_type()` to not assume `DepositAssets` as
the intended use of the assets on the destination.

Instead provides the caller with the ability to specify custom XCM that
be executed on `dest` chain as the last step of the transfer, thus
allowing custom usecases for the transferred assets. E.g. some are
used/swapped/etc there, while some are sent further to yet another
chain.

Note: this is a follow-up on
https://github.com/paritytech/polkadot-sdk/pull/3695, bringing in an API
change for `transfer_assets_using_type()`. This is ok as the previous
version has not been yet released. Thus, its first release will include
the new API proposed by this PR.

This allows usecases such as:
https://forum.polkadot.network/t/managing-sas-on-multiple-reserve-chains-for-same-asset/7538/4

BTW: all this pallet-xcm asset transfers code will be massively reduced
once we have https://github.com/paritytech/xcm-format/pull/54



---------

Signed-off-by: Adrian Catangiu <[email protected]>

Part of https://github.com/paritytech/polkadot-sdk/issues/4126

 we want
to safely increase the execute_workers_max_num gradually from chain to
chain and assess if there are any negative impacts.

This PR performs the necessary plumbing to be able to increase it based
on the chain id, it increase the number of execution workers from 2 to 4
on test network but lives kusama and polkadot unchanged until we gather
more data.

---------

Signed-off-by: Alexandru Gheorghe <[email protected]>

This PR updates the `validate_unsigned` hook for `frame_system` to allow
valid tasks to be submitted as unsigned transactions. It also updates
the task example to be able to submit such transactions via an off-chain
worker.

---------

Co-authored-by: Bastian Köcher <[email protected]>

Fixes https://github.com/paritytech/polkadot-sdk/issues/4206



In #1340 one of the storage types was changed from `Candidates` to
`CandidateList`. Since the actual key includes the hash of this value,
all of the candidates stored here are (a) "missing" and (b) unable to
unreserve their candidacy bond.

This migration kills the storage values and refunds the deposit held for
each candidate.

---------

Signed-off-by: georgepisaltu <[email protected]>
Co-authored-by: georgepisaltu <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: georgepisaltu <[email protected]>

Ensure outbound XCMs are decodable with limits + add `EnsureDecodableXcm` router (for testing purposes) (#4186)

This PR:
- adds `EnsureDecodableXcm` (testing) router that attempts to *encode*
and *decode* passed XCM `message` to ensure that the receiving side will
be able to decode, at least with the same XCM version.
- fixes `pallet_xcm` / `pallet_xcm_benchmarks` assets data generation

Relates to investigation of
https://substrate.stackexchange.com/questions/11288 and missing fix
https://github.com/paritytech/polkadot-sdk/pull/2129 which did not get
into the fellows 1.1.X release.

## Questions/TODOs

- [x] fix XCM benchmarks, which produces undecodable data - new router
catched at least two cases
  - `BoundedVec exceeds its limit`
  - `Fungible asset of zero amount is not allowed`  
- [x] do we need to add `sort` to the `prepend_with` as we did for
reanchor [here](https://github.com/paritytech/polkadot-sdk/pull/2129)?
@serban300 (**created separate/follow-up PR**:
https://github.com/paritytech/polkadot-sdk/pull/4235)
- [x] We added decoding check to `XcmpQueue` -> `validate_xcm_nesting`,
why not to added to the `ParentAsUmp` or `ChildParachainRouter`?
@franciscoaguirre (**created separate/follow-up PR**:
https://github.com/paritytech/polkadot-sdk/pull/4236

)
- [ ] `SendController::send_blob` replace `VersionedXcm::<()>::decode(`
with `VersionedXcm::<()>::decode_with_depth_limit(MAX_XCM_DECODE_DEPTH,
data)` ?

---------

Co-authored-by: Adrian Catangiu <[email protected]>

This is for adding onfinality polkadot bootnode. Please correct me if
this is not the right place for adding a new bootnode

This should be configured on the runtime level and not somewhere inside
the pallet.

---------

Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>

This PR:
- moves `validate_xcm_nesting` from `XcmpQueue` into the `VersionedXcm`
- adds `validate_xcm_nesting` to the `ParentAsUmp`
- adds `validate_xcm_nesting` to the `ChildParachainRouter`


Based on discussion
[here](https://github.com/paritytech/polkadot-sdk/pull/4186#discussion_r1571344270)
and/or
[here](https://github.com/paritytech/polkadot-sdk/pull/4186#discussion_r1572076666)
and/or [here]()

## Question/TODO

- [x] To the
[comment](https://github.com/paritytech/polkadot-sdk/pull/4186#discussion_r1572072295)
- Why was `validate_xcm_nesting` added just to the `XcmpQueue` router
and nowhere else? What kind of problem `MAX_XCM_DECODE_DEPTH` is
solving? (see
[comment](https://github.com/paritytech/polkadot-sdk/pull/4236#discussion_r1574605191))

Add a metric to be able to understand the time jobs are waiting in the
execution queue waiting for an available worker.
https://github.com/paritytech/polkadot-sdk/issues/4126



Signed-off-by: Alexandru Gheorghe <[email protected]>

Weights::v3 also accounts for PoV weight so we no longer need the
SafeCallFilter. All calls are allowed as long as they "fit in the
block".

Basically combines all the recommended calls into one
`build_using_defaults()` call or `init_with_defaults()` when there are
some custom changes required.