Co-authored-by: Branislav Kontur <[email protected]>

Since the hash rules of this part of the `pallet_prefix/storage_prefix`
are always fixed, we can put the runtime calculation into compile time.

---
polkadot address: 15ouFh2SHpGbHtDPsJ6cXQfes9Cx1gEFnJJsJVqPGzBSTudr

---------

Co-authored-by: Juan <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

re:
https://forum.polkadot.network/t/blocknumber-vs-timestamps-should-we-abandon-blocktimes-altogether/4077

This exposes the `LastRelayChainBlockNumber` storage member of
`cumulus-pallet-parachain-system` with a getter and alters the behavior
of this storage item to only be updated in `on_finalize` to ensure a
consistent value throughout `on_initialize` and within transactions.

Parachains, especially with features such as asynchronous backing and
agile coretime, should not use the parachain block number as a clock.
Any feature of Polkadot intended to optimize core utilization and
parachain coretime consumption is likely to worsen this clock as it is
practically applied.

Makes SPs first class citizens along with the relay chains in the
context of our CI runtime upgrade checks.

## Code changes

- Sets missing current storage version in `uniques` pallet
- Adds multisig V1 migration to run where it was missing
- Removes executed migration whos pre/post hooks were failing from
collectives runtime
- Initializes storage versions for SP pallets added after genesis
- Originally I was going to wait for
https://github.com/paritytech/polkadot-sdk/pull/1297 to be merged so
this wouldn't need to be done manually, but it doesn't seem like it'll
be merged any time soon so I've decided to set them manually to unblock
this

## CI changes

- Removed dependency of `westend` runtime upgrades being complete prior
to other ones running. I assume it is supposed to cache the
`try-runtime` build for a performance benefit, but it seems it wasn't
working. Maybe someone from the CI team can look into this or explain
why it needs to be there?

- Adds check-runtime-migration jobs for Parity asset-hubs, bridge-hubs
and contract chains

- Updated VARIABLES to accomodate the `kusama-runtime` package being
renamed to `staging-kusama-runtime` in
https://github.com/paritytech/polkadot-sdk/pull/1241

- Added `EXTRA_ARGS` variable to `check-runtime-migration`, and set
`--no-weight-warnings` to the relay chain runtime upgrade checks (relay
chains don't have weight restrictions).

Closes: https://github.com/paritytech/polkadot-sdk/issues/1718

Moving a few pallets to the latest and greatest `derive_impl` to give it
a try.

Part of #171

---------

Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>

What does this PR do?
- Introduced the TotalValueLocked storage for nomination-pools. 
- introduced a slashing api in mock.rs 
- additional test for tracking a slashing event towards a pool without
sub-pools
- migration for the nomination-pools (V6 to V7) with
`VersionedMigration`

Why are these changes needed?
this is the continuation of the work by @Kianenigma in this
[PR](https://github.com/paritytech/substrate/pull/13319)

How were these changes implemented and what do they affect?
- It's an extra StorageValue that's modified whenever funds flow in or
out of staking for any of the `bonded_account` of `BondedPools`
- The `PoolSlashed`event is now emitted even when no `SubPools` are
found

Closes https://github.com/paritytech/polkadot-sdk/issues/155


KSM: HHEEgVzcqL3kCXgsxSfJMbsTy8dxoTctuXtpY94n4s8F4pS

---------

Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Ankan <[email protected]>
Co-authored-by: Ankan <[email protected]>
Co-authored-by: command-bot <>

Co-authored-by: Bastian Köcher <[email protected]>

Derive `RuntimeDebug\Eq\PartialEq` but do not bound any generics.

This achieved by using their equivalent no bound versions:
`EqNoBound\PartialEqNoBound\RuntimeDebugNoBound`.

Deriving with `Debug`, `Eq`, and `PartialEq` for the `Debt` and `Credit`
type aliases of `Imbalance` is not feasible due to the `OnDrop` and
`OppositeOnDrop` generic types lacking implementations of the same
traits.

This absence posed challenges in testing and any scenarios that demanded
the traits implementations for the type.

closes https://github.com/paritytech/polkadot-sdk/issues/158.
partially addresses
https://github.com/paritytech/polkadot-sdk/issues/226.

Instead of fragile calculation of current balance by looking at `free
balance - ED`, Nomination Pool now freezes ED in the pool reward account
to restrict an account from going below minimum balance. This also has a
nice side effect that if ED changes, we know how much is the imbalance
in ED frozen in the pool and the current required ED. A pool operator
can diligently top up the pool with the deficit in ED or vice versa,
withdraw the excess they transferred to the pool.

## Notable changes
- New call `adjust_pool_deposit`: Allows to top up the deficit or
withdraw the excess deposited funds to the pool.
- Uses Fungible trait (instead of Currency trait). Since NP was not
doing any locking/reserving previously, no migration is needed for this.
- One time migration of freezing ED from each of the existing pools (not
very PoV friendly but fine for relay chain).

This PR is part of [Sync
2.0](https://github.com/paritytech/polkadot-sdk/issues/534) refactoring
aimed at making `ChainSync` a pure state machine.

Resolves https://github.com/paritytech/polkadot-sdk/issues/501.

# Description
This PR introduces two changes:
- the previous `Tracing` trait has been modified to accept contract
address instead of code hash (seems to be way more convenient)
- a new trait `CallInterceptor` that allows intercepting contract calls;
in particular the default implementation for `()` will just proceed in a
standard way (after compilation optimizations, there will be no
footprint of that); however, implementing type might decide to mock
invocation and return `ExecResult` instead

Note: one might try merging `before_call` and `intercept_call`. However,
IMHO this would be bad, since it would mix two completely different
abstractions - tracing without any effects and actual intervention into
execution process.

This will unblock working on mocking contracts utility in drink and
similar tools (https://github.com/Cardinal-Cryptography/drink/issues/33)

# Checklist

- [x] My PR includes a detailed description as outlined in the
"Description" section above
- [ ] My PR follows the [labeling
requirements](https://github.com/paritytech/polkadot-sdk/blob/master/docs/CONTRIBUTING.md#process)
of this project (at minimum one label for `T` required)
- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works (if applicable)

This reverts commit b749ff22 (#1745).

This pull request is removing the Kusama and Polkadot runtime crates. As
still some crates dependent on the runtime crates, this pull request is
doing some more changes.

- It removes the `hostperfcheck` CLI command. This CLI command could
compare the current node against the standard hardware by doing some
checks. Later we added the hardware benchmark feature to Substrate. This
hardware benchmark is running on every node startup and prints a warning
if the current node is too slow. This makes this CLI command a duplicate
that was also depending on the kusama runtime.

- The pull request is removing the emulated integration tests that were
requiring the Kusama or Polkadot runtime crates.

Closes #1671 

Adds a `type_id` function to the `Extension` trait, allowing to properly store an retrieve
boxed `Extensions`.

---------

Co-authored-by: Bastian Köcher <[email protected]>

This PR adjusts the serialized format of the the returned RuntimeVersion
in the rpc-spec-v2 methods. This is done to match the format defined
here:
https://paritytech.github.io/json-rpc-interface-spec/api/chainHead_unstable_follow.html#about-the-runtime

- ##### `apis` field as object
`apis` field of `RuntimeVersion` is now returned as an object, e.g. 
```
"apis": {
      "0xdf6acb689907609b": 3,
      "0x37e397fc7c91f5e4": 1,
}
```
instead of 
```
"apis": [
      ["0xdf6acb689907609b", 3],
      ["0x37e397fc7c91f5e4", 1],
]
```
- ##### removed `stateVersion` and `authoringVersion`
`stateVersion` and `authoringVersion` are no longer returned in the
`RuntimeVersion` JSON Object.

- ##### block index in chain head events as integer

### Related Issues

Closes: #1507
Closes: #1146

### Testing Done
Adjusted existing tests to make sure data is returned in the correct
format.

This pull request is to fix the output text misalignment of `subkey
inspect` command:

(the `Network ID` line has an extra space at the end, and the `Secret
seed` line lacks a leading space)

```
[btwiuse@railway ~]$ subkey inspect '' | cat -A
Secret Key URI `` is account:$
  Network ID:        substrate $
 Secret seed:       0xfac7959dbfe72f052e5a0c3c8d6530f202b02fd8f9f5ca3580ec8deb7797479e$
  Public key (hex):  0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a$
  Account ID:        0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a$
  Public key (SS58): 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV$
  SS58 Address:      5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV$
```

The output should be in the same YAML-like format as `subkey generate`:

```
[btwiuse@railway ~]$ subkey generate  | cat -A
Secret phrase:       awkward eagle survey resemble novel resist modify memory pistol shed flower run$
  Network ID:        substrate$
  Secret seed:       0x20502f79366325b7dc7620664e8844ae69d441baf6e5b571a57d3b3ff28e9586$
  Public key (hex):  0x468874b9e5b6b77333fa702b9201b924d6834bf956e33e2bbe37d131134ca830$
  Account ID:        0x468874b9e5b6b77333fa702b9201b924d6834bf956e33e2bbe37d131134ca830$
  Public key (SS58): 5DfBkAMg5xQmsePFr3BWLZm99smiNyy9axWSgembvFgDKh9v$
  SS58 Address:      5DfBkAMg5xQmsePFr3BWLZm99smiNyy9axWSgembvFgDKh9v$
```

This change will fix `subkey` as well as all binaries that embed it as
the `key` subcommand, for example: `substrate key`, `polkadot key`, etc.

---------

Co-authored-by: Liam Aharon <[email protected]>

This PR implements:
- `archive_unstable_finalized_height`: Get the height of the most recent
finalized block
- `archive_unstable_hash_by_height`: Get the hashes (possible empty) of
blocks from the given height
- `archive_unstable_call`: Call into the runtime of a block

Builds on top of: https://github.com/paritytech/polkadot-sdk/pull/1560

### Testing Done
- unit tests for the methods with custom block tree for different
heights / forks

Closes: https://github.com/paritytech/polkadot-sdk/issues/1510
Closes: https://github.com/paritytech/polkadot-sdk/issues/1513
Closes: https://github.com/paritytech/polkadot-sdk/issues/1511



@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: Sebastian Kunert <[email protected]>

The PR exposes the `rpc_methods` name in the `rpc_methods` response.
This feature is useful for servers that only forward requests to methods
that are reported by the `rpc_methods`.

Jsonrpsee exposes the available methods registered so far. Registering
the `rpc_methods` requires a closure that already stores the result. As
such, the `rpc_methods` method name is manually added to the available
methods.

Closes: https://github.com/paritytech/polkadot-sdk/issues/1627

### Testing Done

```
$> curl -H "Content-Type: application/json" -d '{"id":1, "jsonrpc":"2.0", "method": "rpc_methods", "params":[]}' http://localhost:9944



{"jsonrpc":"2.0","result":{"methods":["account_nextIndex",... "rpc_methods",..."unsubscribe_newHead"]},"id":1}⏎
```

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>

[`RococoGenesisExt`](https://github.com/paritytech/polkadot-sdk/blob/a414ea7515c9cdc81f1d12410e646afc148250e8/polkadot/node/service/src/chain_spec.rs#L152-L171

)
is removed. It was the hack to allow overwriting
`EpochDurationInBlocks`. Removal of `RococGenesisExt` prevents from
manipulating the state to change the runtime constants.

Changes:
- Environment variable which controls the `time::EpochDurationInBlocks`
value was added: `ROCOCO_EPOCH_DURATION` (epoch duration will be set to
the value of env),
- `10,100,600` versions of rococo-runtime are built in CI and put into `polkadot-debug` docker image.

`rococo-runtime` building examples:
- to build runtime for `versi_staging_testnet` which had
EpochDurationInBlocks set to 100:
  ```
ROCOCO_EPOCH_DURATION=100 cargo build --features=fast-runtime -p
rococo-runtime
  ```
- to build runtime for `wococo_development`
  ```
ROCOCO_EPOCH_DURATION=10 cargo build --features=fast-runtime -p
rococo-runtime
  ```
- to build `versi-staging` chain spec:
  ```
ROCOCO_EPOCH_DURATION=100 cargo run -p polkadot --features=fast-runtime
-- build-spec --chain versi-staging --raw
  ```
- to build `wococo-dev` chain spec:
  ```
ROCOCO_EPOCH_DURATION=10 cargo run -p polkadot --features=fast-runtime
-- build-spec --chain wococo-dev --raw
  ```

It is also possible to change the epoch duration by replacing the `code` field in the chain spec with the hex dump of pre-built runtime wasm blob (because the epoch duration is hard-coded into wasm blob).

---------

Co-authored-by: Bastian Köcher <[email protected]>

Last week we experienced a governance attack.
Surprisingly, there was no upper limit on the tip amount.

Due to the mechanism of pallet-fragment-election, the council members
will be refreshed immediately. Attacker is easy to control the council
and give a large tip amount.

Add missing events for pallet-bounties

Created a Github Action that uses the [Review-Bot
app](https://github.com/paritytech/review-bot) to require more fine
tuned requirements to review pull requests before allowing the PR to be
merged.

This uses
[`pull_request_target`](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request_target)
for the event, not `pull_request`. This is a security measure so that an
attacker doesn’t have access to the secrets.

All the rules have been copied from the original
`.github/pr-custom-review.yml` file.

I want to clarify, this particular commit is **not intended to replace
PRCR yet**.

# Advantages it brings over `PRCR`

Most of the features available in `PRCR` have been duplicated and
enhanced. For a complete detailed write up, please see:
- paritytech/pr-custom-review#114 -> Proposal for the rewrite
- [Review Bot
Documentation](https://github.com/paritytech/review-bot/blob/main/README.md)

The most important features are:
- `include` and `exclude` fields now accept an array, making it easier
to read the regular expressions.
- Ability to skip a rule
- We can set that PRs coming from a particular user or team will cause
the rule to be skipped.
- This is used in the `Audit rule`, which was requested by
@the-right-joyce.
  - This resolves paritytech/pr-custom-review#136
- Ability to request fellows instead of teams
- As requested in polkadot-fellows/runtimes#7, this bot has the ability
to request fellows by rank instead of users.
- We currently have polkadot-fellows/runtimes#31 which is using that
feature.

Aside from all the rules available in `PRCR` I have added a particular
rule to lock the review-bot files and require a review from the
`locks-review` team, the @paritytech/ci team and the
@paritytech/opstooling team to ensure that the file has been written
correctly.

## Next steps

The next steps will consist on paritytech/review-bot#53, once this issue
has been resolved, and `review-bot` has worked without any issues on
this repository for a while, we will upgrade it to be able to fully
replace `PRCR`.

Update the HRMP pallet to use field names for Event variants to improve
metadata for a better client experience.
Event variants are now structs instead of unnamed tuples.

Partially implements Substrate issue
[9903](https://github.com/paritytech/substrate/issues/9903) which
doesn't appear to have been moved to the monorepo.

Follow-up to #1543.

Move request-response handling from `ChainSync` to `SyncingEngine` as
part of [Sync
2.0](https://github.com/paritytech/polkadot-sdk/issues/534) refactoring
aimed at making `ChainSync` a pure state machine.

Resolves https://github.com/paritytech/polkadot-sdk/issues/502

.

---------

Co-authored-by: Aaro Altonen <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

Expand `StorageNoopGuard` to be able to add extra context through a
custom error message. When the guard is triggered it panics with an
error message which can be defaulted, set on construction, or set after
it has been constructed.

Turn `StorageNoopGuard` into struct with `storage_root` and
`error_message` and added `from_error_message` constructor and
`set_error_message` setter.

Also added `new()` aliased to `default()`.

Closes #375

---------

Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>

This PR removes some unnecessary `r#`...`#` around a string and the
corresponding comment that it was done because rustfmt wasn't working
for "some reason". It seems to work fine now and clippy prefers it this
way.

---------

Co-authored-by: Joshy Orndorff <[email protected]>

Migrating [PR from the archived polkadot
repo](https://github.com/paritytech/polkadot/pull/7272)

As per
https://github.com/paritytech/polkadot/pull/7272#issuecomment-1559240466,
the changes in this MR include the following pallets into [x] Rococo and
[x] Westend runtimes:

    pallet_conviction_voting
    pallet_referenda
    pallet_ranked_collective
    pallet_custom_origins
    pallet_whitelist

And only for westend-runtime:

    pallet_treasury

Following [Kusama runtime
config](https://github.com/paritytech/polkadot/tree/dbae30efe080a1d41fe54ef4da8af47614c9ca93/runtime/kusama/src)
as a baseline.

Benchmarking of the following pallets done for both Rococo and Westend:

    pallet_conviction_voting
    pallet_referenda
    pallet_ranked_collective (only on Rococo)
    pallet_whitelist

And only for Westend:

    pallet_treasury

Removed Gov1 from Rococo as in
https://github.com/paritytech/polkadot/pull/6701

Rococo Gov1 storage will be cleaned in a different PR - [issue ](https://github.com/paritytech/polkadot-sdk/issues/1618

)

---------

Co-authored-by: Giles Cope <[email protected]>

I hope it's enough to fix #1701 
the only solution I found to make it happen is to put an associated type
to the `Bounded` enum as well.
@liamaharon @Kianenigma @bkchr

 

Polkadot address: 12poSUQPtcF1HUPQGY3zZu2P8emuW9YnsPduA4XG3oCEfJVp

---------

Signed-off-by: muraca <[email protected]>
Co-authored-by: Liam Aharon <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>

- Async-backing related primitives are stable `primitives::v6`
- Async-backing API is now part of `api_version(7)`
- It's enabled on Rococo and Westend runtimes

---------

Signed-off-by: Andrei Sandu <[email protected]>
Co-authored-by: Andrei Sandu <[email protected]>

This PR implements [`GenesisBuilder`
API](https://github.com/paritytech/polkadot-sdk/blob/a414ea7515c9cdc81f1d12410e646afc148250e8/substrate/primitives/genesis-builder/src/lib.rs#L38

)
for all the runtimes in polkadot repo.

Step towards: paritytech/polkadot-sdk#25

---------

Co-authored-by: ordian <[email protected]>

It changes following APIs:
- trait `ChainApi`
-- `validate_transaction`

- trait `TransactionPool` 
--`submit_at`
--`submit_one`
--`submit_and_watch`

and some implementation details, in particular:
- impl `Pool` 
--`submit_at`
--`resubmit_at`
--`submit_one`
--`submit_and_watch`
--`prune_known`
--`prune`
--`prune_tags`
--`resolve_block_number`
--`verify`
--`verify_one`

- revalidation queue

All tests are also adjusted.

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <[email protected]>

Bumps [directories](https://github.com/soc/directories-rs) from 4.0.1 to
5.0.1.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/soc/directories-rs/commits">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=directories&package-manager=cargo&previous-version=4.0.1&new-version=5.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores

)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Add debug_assertions to WASM builds in the short-benchmarks CI job.

Disallow warnings, show a full backtrace and remove the WASM output
colour to improve information from this CI step in the event of a failure.

This is motivated by a
[case](https://github.com/paritytech/polkadot-sdk/pull/1676/commits/23d64918be3ca68413050383c5d8a73f63d451dd)
where a benchmark was misconfigured in master and failing to send
notifications, but passing CI. We don't want this to fail if the problem
happens in production, but ideally we'd know from the CI if it was
misconfigured and the messages weren't getting sent, as that could
(would?) affect the weights that the benchmark generates.

Enabling debug-assertions in WASM builds for the short-benchmarks would
catch "soft" failures like this in future.

This PR ensures that all storage keys under a prefix are returned by the
`chainHead_storage` method.

Before this PR, the `storage_keys` was used with just the `start_key`.
Before the pagination event was generated, the last reported key
`last_key` was saved internally.
When the pagination is resumed, the `last_key` will serve as the next
`start_key` to the `storage_keys` API.

However, this behavior does not function properly for non-prefixed
storage keys.

Entry keys `a`, `ab`, `abc` share a common prefix and therefore the `ab`
key leads to `abc`.
However, for `a`, `ab`, `aB` and `abc`, the `aB` key does not
immediately lead to `abc`.

To mitigate this, the PR saves the start key of the query, together with
the next pagination key.
Improve testing to ensure we have a key entry that doesn't share the
prefix with the descendant key.

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>

I had quite a few conversions to/from `u64` because these were lacking.

Let me know if others are desirable as well for symmetry.