Relates to: https://github.com/paritytech/polkadot-sdk/issues/3214

## TODO

- [ ] backport to the `1.7.0` release

Fixes: https://github.com/paritytech/polkadot-sdk/issues/3165

---------


(cherry picked from commit 5ba89217

)

Signed-off-by: Alexandru Gheorghe <[email protected]>

Fixup for https://github.com/paritytech/polkadot-sdk/pull/2587 to make
the `core-fellowship` crate work with swapped members.

Adds a `MemberSwappedHandler` to the `ranked-collective` pallet that are
implemented by `core-fellowship+salary`.
There is are exhaustive tests
[here](https://github.com/paritytech/polkadot-sdk/blob/72aa7ac17a0e5b16faab5d2992aa2db2e01b05d0/substrate/frame/core-fellowship/src/tests/integration.rs#L338)
and
[here](https://github.com/paritytech/polkadot-sdk/blob/ab3cdb05a5ebc1ff841f8dda67edef0ea40bbba5/substrate/frame/salary/src/tests/integration.rs#L224

)
to check that adding member `1` is equivalent to adding member `0` and
then swapping.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

This PR addresses two issues:
- It modifies `DepositAsset`'s asset filter from `All` to
`AllCounted(1)` to prevent potentially charging excessive weight/fees.
This adjustment avoids situations where fees could be calculated based
on the count of assets, as illustrated
[here](https://github.com/paritytech/polkadot-sdk/blob/master/cumulus/parachains/runtimes/bridge-hubs/bridge-hub-rococo/src/weights/xcm/mod.rs#L38-L46).
- It encapsulates `DepositAsset` with `SetAppendix` to ensure that
`fees` are not trapped in any case. For instance, this prevents issues
when `ExportXcm::validate` encounters an error during the processing of
`ExportMessage`.

Bumps [bounded-collections](https://github.com/paritytech/parity-common)
from 0.1.9 to 0.2.0.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/paritytech/parity-common/commits/impl-rlp-v0.2.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=bounded-collections&package-manager=cargo&previous-version=0.1.9&new-version=0.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores

)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Liam Aharon <[email protected]>

Signed-off-by: Alexandru Gheorghe <[email protected]>

This fix aims to solve an issue in Kusama that resulted in failed
reserve asset transfers.

During multi-hop XCMs, like reserve asset transfers where the reserve is
not the sender nor the destination, but a third remote chain, the origin
is not available to pay for delivery fees out of their account directly,
so delivery fees should be paid out of transferred assets.

This commit also adds an xcm-emulator regression test that validates
this scenario is now working.

Signed-off-by: Adrian Catangiu <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>

Usual chore after bumping Rust toolchain version.

Found it when trying to run: 
```
bot bench-all pallet --pallet=pallet_balances
```
https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5081585

## TODO
- [x] check/fix command bot for `people-westend-dev` /
`people-rococo-dev`
https://github.com/paritytech/command-bot-scripts/pull/67

[frame] `#[pallet::composite_enum]` improved variant count handling + removed `pallet_balances`'s `MaxHolds` config (#2657)

I started this investigation/issue based on @liamaharon question
[here](https://github.com/paritytech/polkadot-sdk/pull/1801#discussion_r1410452499).

## Problem

The `pallet_balances` integrity test should correctly detect that the
runtime has correct distinct `HoldReasons` variant count. I assume the
same situation exists for RuntimeFreezeReason.

It is not a critical problem, if we set `MaxHolds` with a sufficiently
large value, everything should be ok. However, in this case, the
integrity_test check becomes less useful.

**Situation for "any" runtime:**
- `HoldReason` enums from different pallets:
```rust
        /// from pallet_nis
        #[pallet::composite_enum]
	pub enum HoldReason {
		NftReceipt,
	}

        /// from pallet_preimage
        #[pallet::composite_enum]
	pub enum HoldReason {
		Preimage,
	}

        // from pallet_state-trie-migration
        #[pallet::composite_enum]
	pub enum HoldReason {
		SlashForContinueMigrate,
		SlashForMigrateCustomTop,
		SlashForMigrateCustomChild,
	}
```

- generated `RuntimeHoldReason` enum looks like:
```rust
pub enum RuntimeHoldReason {

    #[codec(index = 32u8)]
    Preimage(pallet_preimage::HoldReason),

    #[codec(index = 38u8)]
    Nis(pallet_nis::HoldReason),

    #[codec(index = 42u8)]
    StateTrieMigration(pallet_state_trie_migration::HoldReason),
}
```

- composite enum `RuntimeHoldReason` variant count is detected as `3`
- we set `type MaxHolds = ConstU32<3>`
- `pallet_balances::integrity_test` is ok with `3`(at least 3)

However, the real problem can occur in a live runtime where some
functionality might stop working. This is due to a total of 5 distinct
hold reasons (for pallets with multi-instance support, it is even more),
and not all of them can be used because of an incorrect `MaxHolds`,
which is deemed acceptable according to the `integrity_test`:
  ```
  // pseudo-code - if we try to call all of these:

T::Currency::hold(&pallet_nis::HoldReason::NftReceipt.into(),
&nft_owner, deposit)?;
T::Currency::hold(&pallet_preimage::HoldReason::Preimage.into(),
&nft_owner, deposit)?;

T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForContinueMigrate.into(),
&nft_owner, deposit)?;

  // With `type MaxHolds = ConstU32<3>` these two will fail

T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForMigrateCustomTop.into(),
&nft_owner, deposit)?;

T::Currency::hold(&pallet_state_trie_migration::HoldReason::SlashForMigrateCustomChild.into(),
&nft_owner, deposit)?;
  ```  


## Solutions

A macro `#[pallet::*]` expansion is extended of `VariantCount`
implementation for the `#[pallet::composite_enum]` enum type. This
expansion generates the `VariantCount` implementation for pallets'
`HoldReason`, `FreezeReason`, `LockId`, and `SlashReason`. Enum variants
must be plain enum values without fields to ensure a deterministic
count.

The composite runtime enum, `RuntimeHoldReason` and
`RuntimeFreezeReason`, now sets `VariantCount::VARIANT_COUNT` as the sum
of pallets' enum `VariantCount::VARIANT_COUNT`:
```rust
#[frame_support::pallet(dev_mode)]
mod module_single_instance {

	#[pallet::composite_enum]
	pub enum HoldReason {
		ModuleSingleInstanceReason1,
		ModuleSingleInstanceReason2,
	}
...
}

#[frame_support::pallet(dev_mode)]
mod module_multi_instance {

	#[pallet::composite_enum]
	pub enum HoldReason<I: 'static = ()> {
		ModuleMultiInstanceReason1,
		ModuleMultiInstanceReason2,
		ModuleMultiInstanceReason3,
	}
...
}


impl self::sp_api_hidden_includes_construct_runtime::hidden_include::traits::VariantCount
    for RuntimeHoldReason
{
    const VARIANT_COUNT: u32 = 0
        + module_single_instance::HoldReason::VARIANT_COUNT
        + module_multi_instance::HoldReason::<module_multi_instance::Instance1>::VARIANT_COUNT
        + module_multi_instance::HoldReason::<module_multi_instance::Instance2>::VARIANT_COUNT
        + module_multi_instance::HoldReason::<module_multi_instance::Instance3>::VARIANT_COUNT;
}
```

In addition, `MaxHolds` is removed (as suggested
[here](https://github.com/paritytech/polkadot-sdk/pull/2657#discussion_r1443324573))
from `pallet_balances`, and its `Holds` are now bounded to
`RuntimeHoldReason::VARIANT_COUNT`. Therefore, there is no need to let
the runtime specify `MaxHolds`.


## For reviewers

Relevant changes can be found here:
- `substrate/frame/support/procedural/src/lib.rs` 
-  `substrate/frame/support/procedural/src/pallet/parse/composite.rs`
-  `substrate/frame/support/procedural/src/pallet/expand/composite.rs`
-
`substrate/frame/support/procedural/src/construct_runtime/expand/composite_helper.rs`
-
`substrate/frame/support/procedural/src/construct_runtime/expand/hold_reason.rs`
-
`substrate/frame/support/procedural/src/construct_runtime/expand/freeze_reason.rs`
- `substrate/frame/support/src/traits/misc.rs`

And the rest of the files is just about removed `MaxHolds` from
`pallet_balances`

## Next steps

Do the same for `MaxFreezes`
https://github.com/paritytech/polkadot-sdk/issues/2997

.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Dónal Murray <[email protected]>
Co-authored-by: gupnik <[email protected]>

Some more work regarding XCMv4. Two limits from v3 were not transferred
over, those are:
- The instructions limit
- The number of assets limit
Both of these are now in v4.

For some reason `AssetInstance` increased in size, don't know why CI
didn't catch that before.

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: command-bot <>

The `parachains-common` contains a lots of constants and type
definitions which are used for `polkadot-sdk`'s testnet runtimes and
also for `polkadot-fellows`'s production [SP
runtimes](https://github.com/polkadot-fellows/runtimes/tree/main/system-parachains/constants).
This PR cleans `parachains-common` module to contain only common and
generic functionality.

Testnet-specific constants have been moved to the separate module
dedicated just for testnets:
`polkadot-sdk/cumulus/parachains/runtimes/constants/`


Part of: https://github.com/paritytech/polkadot-sdk/issues/3054



---------

Co-authored-by: georgepisaltu <[email protected]>

cc https://github.com/paritytech/ci_cd/issues/926



---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Dónal Murray <[email protected]>
Co-authored-by: Vladimir Istyufeev <[email protected]>

Add `Balances::force_adjust_total_issuance` as preparation for fixing
https://github.com/polkadot-fellows/runtimes/issues/147

.
Important changes in `substrate/frame/balances/src/lib.rs`.

TODO:
- [x] Update weights

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <[email protected]>

fixes #675

---------

Signed-off-by: Andrei Sandu <[email protected]>

closes https://github.com/polkadot-fellows/help-center/issues/1



---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

Moves `pallet_mmr` back behind `pallet_session` to address
polkadot-fellows/runtimes#160.

Opening draft for CI - should be merged or closed depending on outcome
of w3f/polkadot-spec#718.

---------

Co-authored-by: Adrian Catangiu <[email protected]>

This fixes a bug in nomination pools that msitakenly claimed the
rewards, upon pool destruction, into the caller of `unbond` and not the
actual member.

More description and a PA coming soon. Opening this for now to expediate
backport.

---------

Co-authored-by: Gonçalo Pestana <[email protected]>

To deploy https://github.com/paritytech/polkadot-sdk/pull/3029

Co-authored-by: claravanstaden <Cats 4 life!>

Reverts paritytech/polkadot-sdk#2302. 🤦‍♂️ should have checked the
migration CI first.

We either need to reduce the `max_message_size` for the open HRMP
channels on the failing chains or increase the `PageSize` of the XCMP
queue.
Both would be fine on a test-net, but i assume this will also fail
before the next SP runtime upgrade so first need to think what best to
do.
AFAIK its not possible currently to change the `max_message_size` of an
open HRMP channel.

Closes: https://github.com/paritytech/polkadot-sdk/issues/3123

In CI jobs I see calls to `forklift clean`, which apparently does not
exist.
From CI logs:
```
1m$ forklift clean[0;m
Cargo cache management utility

Usage:
  forklift [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  serve       Run forklift coordinator server for current location
  start       Start detached forklift coordinator server for current location
  stop        Stop forklift coordinator server for current location

Flags:
  -c, --compression string     Compression algorithm to use
                               Available: none, xz (default "zstd")
  -h, --help                   help for forklift
  -p, --param stringToString   map of additional parameters
                                ex: -p S3_BUCKET_NAME=my_bucket (default [])
  -s, --storage string         Storage driver
                               Available: s3, fs (default "s3")
  -v, --verbose string         Available: panic, fatal, error, warn, warning, info, debug, trace (default "info")
      --version                version for forklift

Use "forklift [command] --help" for more information about a command.
```

Candidate validation has a lot of operations that are cpu bound on its
main loop things, like:
```
validation_code.hash()
sp_maybe_compressed_blob::decompress(
		&validation_code.0,
		VALIDATION_CODE_BOMB_LIMIT,
)

sp_maybe_compressed_blob::decompress(&pov.block_data.0, POV_BOMB_LIMIT) 
let code_hash = sp_crypto_hashing::blake2_256(&code).into();
```

When you add all that you for large POV and CODE it is going to take in
the order of 10s of ms and because these are cpu bound operation it is
going to hog the executor thread and negatively affect other subsystems
around it, so it is better to just move the subsystem on the blocking
pool to make sure such unexpected behaviour is avoided.

Note! In practice this subsystem does not have a high number of work to
be done, so probably the impact of it is really low, but better safe
than sorry.

Signed-off-by: Alexandru Gheorghe <[email protected]>

- Prepares for the Deneb hardfork on Sepolia testnet on 31 January
(needs to be deployed to Rococo before then)
- Removes `beacon-minimal-spec` flag for simpler config
- Adds test comments

---------

Co-authored-by: Ron <[email protected]>
Co-authored-by: claravanstaden <Cats 4 life!>
Co-authored-by: Alistair Singh <[email protected]>

Remove `without_storage_info` from the XCMP queue pallet. Part of
https://github.com/paritytech/polkadot-sdk/issues/323



Changes:
- Limit the number of channels that can be suspended at the same time.
- Limit the number of channels that can have messages or signals pending
at the same time.

A No-OP migration is put in place to ensure that all `BoundedVec`s still
decode and not truncate after upgrade. The storage version is thereby
bumped to 4 to have our tooling remind us to deploy that migration.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>

Pulls: https://github.com/paritytech/orchestra/pull/71



---------

Signed-off-by: Alexandru Gheorghe <[email protected]>

#2970 accidentally added a crate twice to the workspace. Now extending
the workspace check to explicitly error then.

I think the check should also be required now.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

While including files cross crate works locally. When pushed to
crates.io each crate is seperate so the file path no longer exists.

Instead change it to a symlink, which cargo will change to a read
directory when published.

extracted useful code from #2982 

This PR:
- adds test 2 for Rococo <> Westend bridge: checks that relayer doesn't
submit any extra headers while there are no any messages;
- adds test 3 for Rococo <> Westend bridge: checks that relayer doesn't
submit any extra headers when there are messages;
- fixes most of comments from #2439 (like: log names, ability to run
specify test number when calling `run-tests.sh`).

Right now of all our tests, only test 2 is working (until BHs will be
upgraded to use async backing), so you can test it with
`./bridges/zombienet/run-tests.sh --test 2` locally.

After some conversations around the topic of launching a community
staging network ( [forum post
1](https://forum.polkadot.network/t/a-new-test-network-for-polkadot/4325),
[forum post
2](https://forum.polkadot.network/t/the-new-polkadot-community-testnet/4956)
). And Erin having introduced the community testnet in the [last episode
of AAG](https://www.youtube.com/watch?v=uzJEGVG78_E&t=1612s

).

I would like to include the `paseo` options as one of the chains
available for launching using the polkadot node.

Note that the chain spec implementation is not yet included at this
stage.

UPDATE:

Merged master with rebase enabled and introduced quite some noise in
this PR.
Relevant changes are in:
- `polkadot/node/service/chain-specs/paseo.json`
- `polkadot/node/service/src/chain_spec.rs`
- `polkadot/cli/src/command.rs`

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Jegor Sidorenko <[email protected]>
Co-authored-by: Alexander Theißen <[email protected]>
Co-authored-by: Sergej Sakac <[email protected]>
Co-authored-by: Dónal Murray <[email protected]>
Co-authored-by: Adel Arja <[email protected]>
Co-authored-by: Gonçalo Pestana <[email protected]>
Co-authored-by: Ross Bulat <[email protected]>
Co-authored-by: Francisco Gamundi <[email protected]>
Co-authored-by: Adrian Catangiu <[email protected]>

Currently, collators and their alongside nodes spin up a full-scale
overseer running a bunch of subsystems that are not needed if the node
is not a validator. That was considered to be harmless; however, we've
got problems with unused subsystems getting stalled for a reason not
currently known, resulting in the overseer exiting and bringing down the
whole node.

This PR aims to only run needed subsystems on such nodes, replacing the
rest with `DummySubsystem`.

It also enables collator-optimized availability recovery subsystem
implementation.

Partially solves #1730.

Topology is coming only at the beginning of each session, so we might
lose it if prospective parachains was not enabled at the begining of the
session, so cache it for later use.

Fixes: https://github.com/paritytech/polkadot-sdk/issues/3058



---------

Signed-off-by: Alexandru Gheorghe <[email protected]>