cumulus: add asset-hub-rococo runtime based on asset-hub-kusama and add asset-bridging support to it (#1215)

This commit adds Rococo Asset Hub dedicated runtime so we can test new
features here, before merging them in Kusama Asset Hub.
Also adds one such feature: asset transfer over bridge (Rococo AssetHub
<> Wococo AssetHub)

- clone `asset-hub-kusama-runtime` -> `asset-hub-rococo-runtime`
- make it use Rococo primitives, names, assets, constants, etc
- add asset-transfer-over-bridge support to Rococo AssetHub <> Wococo
AssetHub

Fixes #1128

---------

Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>

Remove `publish = false` to publish the crate

Adds descriptions needed for publishing bridges crates to crates.io.

see https://forum.parity.io/t/crates-need-descriptions/2115

Signed-off-by: Adrian Catangiu <[email protected]>

Closes https://github.com/paritytech/polkadot-sdk/issues/1916

Changes:
- Trivially wrap the migration into a version migration to enforce
idempotency.
- Opinionated logging nits

@liamaharon maybe we can add a check to the `try-runtime-cli` that
migrations are idempotent? It should be possible to check that the
storage root is identical after executing a second time (and that it
does not panic like it did here 😆

).

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>

Partially addresses #1638

Still need a better solution to allow devs to have better control of
this.

---------

Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: joe petrowski <[email protected]>

Co-authored-by: Mira Ressel <[email protected]>

These two jobs are now required by github branch protection settings.

Slightly less verbose use of fungible(s).

---------

Co-authored-by: Liam Aharon <[email protected]>

## Summary

This PR introduces several enhancements.

The current implementation of `NetworkExportTable` lacks remote location
filtering support beyond `NetworkId` lookup. To provide more control and
granularity, it's essential to allow configuration for bridging to
different consensus `NetworkId` while restricting access e.g. to
particular remote parachains.

Additionally, the `StartsWith` and `Equals` and
`StartsWithExplicitGlobalConsensus` helper functions, which are in
active use, are moved to the `xcm-builder` and `frame_support` modules
for better code organization.

Adds a new `LocationWithAssetFilters` filter to enable location-based
and asset-related filtering. This filter is useful for configuring the
`pallet_xcm` filter for
[XcmTeleportFilter](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/pallet-xcm/src/lib.rs#L212)
and
[XcmReserveTransferFilter](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm/pallet-xcm/src/lib.rs#L216)
to restrict specific assets.

Furthermore, the `BridgeMessage` fields are not accessible outside of
`xcm-builder`, limiting the ability to create custom logic dependent on
it.

---------

Co-authored-by: Francisco Aguirre <[email protected]>

Just making a few hidden imports cleaner and hidden in docs.

---------

Co-authored-by: Keith Yeung <[email protected]>

Allow Locks/Holds/Reserves/Freezes by default when using `pallet_balances` `TestDefaultConfig` (#1880)

Allow Locks/Holds/Reserves/Freezes by default when using
`pallet_balances` `TestDefaultConfig`.

Changes:
- Add missing crate to the workspace
- Remove versions from local dependency links

Maybe it is finally worth it to add this scrip to the CI to find these
things earlier:
[check-deps.py](https://github.com/ggwpez/substrate-scripts/blob/master/import-runtime-repos/check-deps.py).

@paritytech/ci what would be the best location for that check?  
It takes only a second to run, so maybe we can squeeze it into one of
the existing checks?
Otherwise creating a new GH workflow feels a bit wasteful... maybe i can
group it with https://github.com/paritytech/polkadot-sdk/pull/1831

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Make System Parachain trusted Teleporters of each other.

Migration of https://github.com/paritytech/cumulus/pull/2842

---------

Co-authored-by: command-bot <>
Co-authored-by: joe petrowski <[email protected]>

This PR adds a missing env to allow publishing the Docker image.

Introduces an `extract` amount method for `fungible/s` `Imbalance`.

Fix situation where BEEFY initial validator set could not be determined.

If state is unavailable at BEEFY genesis block to get initial validator
set, get the info from header digests. For this, we need to walk back
the chain starting from BEEFY genesis looking for the BEEFY digest
announcing the active validator set for that respective session.

This commit fixes a silly bug where walking back the chain was stopped
when reaching BEEFY genesis block, which is incorrect when BEEFY genesis
is not session boundary block. When BEEFY genesis is set to some random
block within a session, we need to walk back to the start of the session
to see the validator set announcement.

Added regression test for this fix.

Fixes https://github.com/paritytech/polkadot-sdk/issues/1885

Signed-off-by: Adrian Catangiu <[email protected]>

Co-authored-by: command-bot <>

Adding link checker to the CI (closes
https://github.com/paritytech/polkadot-sdk/issues/993). It would be nice
to have the docs people own this and extend accordingly.
Currently all known-bad links are excluded, but we should one-by-one
include those as well until all are fixed.

This check now ensures that 1) no new broken links are introduced into
`.rs` files and 2) that no old links break unnoticed.

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

# Description

Follow up for https://github.com/paritytech/polkadot-sdk/pull/1489.
Closes #611 

Before we calculated the channel size during alert expression but in
#1489 a new metric was introduced that reports channel size.
## Changes:
1. updated alert rule to use new metric.

Remove `publish = false` to publish the crate

Following
[polkadot#7314](https://github.com/paritytech/polkadot/pull/7314) and
after merging https://github.com/paritytech/polkadot-sdk/pull/1177 this
PR solves https://github.com/paritytech/polkadot-sdk/issues/1618

The following is a summary of the outcome of the migration.

| Module | Total Accounts | Total stake to unlock | Total deposit to
unreserve |
| ------- | --------------- | --------------------- |
-------------------------- |
| Elections Phragmen | 27 | 1,132.821063320441 ROC | 1.465386531600 ROC
|
| Democracy | 69 | 2733.923509345613 ROC | 0.166666665000 ROC |
| Tips | 4 | N/A | 0.015099999849 ROC |

The migrations will also remove the following amount of keys

103 Democracy keys 🧹
5 Council keys 🧹
1 TechnicalCommittee keys 🧹
25 PhragmenElection keys 🧹
1 TechnicalMembership keys 🧹
9 Tips keys 🧹

- Removal of Arkworks unit tests. These tests were just testing the
arkworks upstream implementation which should be assumed correct. This
is not the place to test well known dependencies.
- Removal of some over-engineering. We just store the calls to Arkworks
in one file. Per-curve sources are not required.
- Docs formatting

---

I also took the opportunity to bump the `bandersnatch-vrfs` crate
revision internally providing some new shiny stuff.

The runtime code of a parachain can be replaced on the relay-chain via:

[cumulus]:
[enact_authorized_upgrade](https://github.com/paritytech/polkadot-sdk/blob/1a38d6d6/cumulus/pallets/parachain-system/src/lib.rs#L661);
this is used for a runtime upgrade when a parachain is not bricked.

[polkadot] (these are used when a parachain is bricked):
-
[force_set_current_code](https://github.com/paritytech/polkadot-sdk/blob/1a38d6d6/polkadot/runtime/parachains/src/paras/mod.rs#L823):
immediately changes the runtime code of a given para without a pvf check
(root).
-
[force_schedule_code_upgrade](https://github.com/paritytech/polkadot-sdk/blob/1a38d6d6/polkadot/runtime/parachains/src/paras/mod.rs#L864):
schedules a change to the runtime code of a given para including a pvf
check of the new code (root).
-
[schedule_code_upgrade](https://github.com/paritytech/polkadot-sdk/blob/1a38d6d6/polkadot/runtime/common/src/paras_registrar.rs#L395):
schedules a change to the runtime code of a given para including a pvf
check of the new code. Besides root, the parachain or parachain manager
can call this extrinsic given that the parachain is unlocked.

Polkadot signals a parachain to be ready for a runtime upgrade through
the
[GoAhead](https://github.com/paritytech/polkadot-sdk/blob/e4949344

/polkadot/primitives/src/v5/mod.rs#L1229)
signal.

When in cumulus `enact_authorized_upgrade` is executed, the same
underlying helper function of `force_schedule_code_upgrade` &
`schedule_code_upgrade`:
[schedule_code_upgrade](https://github.com/paritytech/polkadot/blob/09b61286da11921a3dda0a8e4015ceb9ef9cffca/runtime/parachains/src/paras/mod.rs#L1778),
is called on the relay-chain, which sets the `GoAhead` signal (if the
pvf is accepted).

If Cumulus receives the `GoAhead` signal from polkadot without having
the `PendingValidationCode` ready, it will panic
([ref](https://github.com/paritytech/polkadot/pull/7412)). For
`enact_authorized_upgrade` we know for sure the `PendingValidationCode`
is set. On the contrary, for `force_schedule_code_upgrade` &
`schedule_code_upgrade` this is not the case.

This PR includes a flag such that the `GoAhead` signal will only be set
when a runtime upgrade is enacted by the parachain
(`enact_authorized_upgrade`).

additional info: https://github.com/paritytech/polkadot/pull/7412

Closes #641

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

This PR refactors the staking ledger logic to encapsulate all reads and
mutations of `Ledger`, `Bonded`, `Payee` and stake locks within the
`StakingLedger` struct implementation.

With these changes, all the reads and mutations to the `Ledger`, `Payee`
and `Bonded` storage map should be done through the methods exposed by
StakingLedger to ensure the data and lock consistency of the operations.
The new introduced methods that mutate and read Ledger are:

- `ledger.update()`: inserts/updates a staking ledger in storage;
updates staking locks accordingly (and ledger.bond(), which is synthatic
sugar for ledger.update())
- `ledger.kill()`: removes all Bonded and StakingLedger related data for
a given ledger; updates staking locks accordingly;
`StakingLedger::get(account)`: queries both the `Bonded` and `Ledger`
storages and returns a `Option<StakingLedger>`. The pallet impl exposes
fn ledger(account) as synthatic sugar for `StakingLedger::get(account)`.

Retrieving a ledger with `StakingLedger::get()` can be done by providing
either a stash or controller account. The input must be wrapped in a
`StakingAccount` variant (Stash or Controller) which is treated
accordingly. This simplifies the caller API but will eventually be
deprecated once we completely get rid of the controller account in
staking. However, this refactor will help with the work necessary when
completely removing the controller.

Other goals:

- No logical changes have been introduced in this PR;
- No breaking changes or updates in wallets required;
- No new storage items or need to perform storage migrations;
- Centralise the changes to bonds and ledger updates to simplify the
OnStakingUpdate updates to the target list (related to
https://github.com/paritytech/polkadot-sdk/issues/443)

Note: it would be great to prevent or at least raise a warning if
`Ledger<T>`, `Payee<T>` and `Bonded<T>` storage types are accessed
outside the `StakingLedger` implementation. This PR should not get
blocked by that feature, but there's a tracking issue here
https://github.com/paritytech/polkadot-sdk/issues/149

Related and step towards
https://github.com/paritytech/polkadot-sdk/issues/443

BEEFY needs two cryptographic keys at the same time. Validators should
sign BEEFY payload using both ECDSA and BLS key. The network will gossip
a payload which contains a valid ECDSA key. The prover nodes aggregate
the BLS keys if aggregation fails to verifies the validator which
provided a valid ECDSA signature but an invalid BLS signature is subject
to slashing.

As such BEEFY session should be initiated with both key. Currently there
is no straight forward way of doing so, beside having a session with
RuntimeApp corresponding to a crypto scheme contains both keys.

This pull request implement a generic paired_crypto scheme as well as
implementing it for (ECDSA, BLS) pair.

---------

Co-authored-by: Davide Galassi <[email protected]>
Co-authored-by: Robert Hambrock <[email protected]>

closes #695

Could potentially be helpful to preserving caches when applicable, as
discussed in #685

kusama address: FvpsvV1GQAAbwqX6oyRjemgdKV11QU5bXsMg9xsonD1FLGK

# Description
- What does this PR do?
  - link added
- Why are these changes needed?
  - improve docs
- How were these changes implemented and what do they affect?
  - only concerns docs

closes #622 

Pros:
* simpler interface, just functions:
`create_runtime_from_artifact_bytes()` and `execute_artifact()`

Cons:
* extra overhead of constructing executor semantics each time

I could make it a combination of
* `create_runtime_config(params)` (such that we could clone the
constructed semantics)
* `create_runtime(blob, config)`
* `execute_artifact(blob, config, params)`

Not sure if it's worth it though.

---------

Co-authored-by: Bastian Köcher <[email protected]>

The lint `clippy::clone_double_ref` was renamed to
`suspicious_double_ref_op` in [this
commit](https://github.com/rust-lang/rust/commit/5c99175a9efcaa3d65712c119f361add22e3a859)
(thanks @liamaharon) and now generates a lot of noise in the terminal
when run both in CI and locally with 1.73.

This renames the lint in line with this, but does not change
functionality.

May cause issues for people running earlier versions locally - @altaua
requesting review to get your opinion on this.

- This PR refactors `alliance/src/benchmarkings.rs` to use benchmarking
v2. These changes are needed to improve the readability and
maintainability of the benchmarking code.

- No known issue to backlink.

## Local Testing 
1. `cargo build --features runtime-benchmarks` 
2. `cargo run --locked --release -p node-cli --bin substrate-node
--features runtime-benchmarks -- benchmark pallet --execution wasm
--wasm-execution compiled --chain dev --pallet "*" --extrinsic "*"
--steps 2 --repeat 1`

Co-authored-by: Marcin S <[email protected]>

Part of #171

- Remove cached messages used for deduplication in `GossipValidator`
since they're already deduplicated in upper layer `NetworkGossip`.
- Add cache for "justified rounds" to quickly discard any further (even
if potentially different) justifications at the gossip level, once a
valid one (for a respective round) is submitted to the worker.
- Add short-circuit in worker `finalize()` method to not attempt to
finalize same block multiple times (for example when we get
justifications for same block from multiple components like
block-import, gossip or on-demand).
- Change a test which had A LOT of latency in syncing blocks for some
weird reason and would only run after ~150seconds. It now runs
instantly.

Fixes https://github.com/paritytech/polkadot-sdk/issues/1728

Fixes https://github.com/paritytech/polkadot-sdk/issues/1839

Currently, `composite_enum`s do not support pallet instances. This PR
allows the following:
```rust
	#[pallet::composite_enum]
	pub enum HoldReason<I: 'static = ()> {
		SomeHoldReason
	}
```

### Todo

- [x]  UI Test

# Description

In a couple of cases, there were links pointing to the w3f github pages
domain. In other instances, there were links pointing to the old
polkadot repo's github pages. Both of these are now pointing to the
relevant links in
https://paritytech.github.io/polkadot-sdk/book/index.html.

These changes were made specifically because the w3f github pages
returns a 404, and while fixing the links, the old polkadot repo links
were touched up as well even if they do redirect properly.

This shouldn't affect anything as these are documentation link changes
only.

Exposes disabled validators list via a runtime API.

---------

Co-authored-by: ordian <[email protected]>
Co-authored-by: ordian <[email protected]>

This PR adds a `try_state` hook for the `Treasury` pallet.
Part of #239.

This pr resolves https://github.com/paritytech/polkadot-sdk/issues/1428.

*Added only to Kusama for now*

I did raise it
[here](https://github.com/polkadot-fellows/runtimes/pull/19) and we
discussed creating a chopsticks test to run an end-to-end test

however, to do that I will need a build agent/custom runner that is
powerful enough to run the build

I will be doing that separately as I still think having chopsticks test
your runtime with each commit will be very powerful and extremely useful
for the ecosystem

For now I have used XCM simulator and replicated what the other reserve
tests do

---------

Co-authored-by: Gavin Wood <[email protected]>
Co-authored-by: Francisco Aguirre <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>