Ref https://github.com/paritytech/ci_cd/issues/1107

We mainly need that so that we can finally compile the `pallet_revive`
fixtures on stable. I did my best to keep the commits focused on one
thing to make review easier.

All the changes are needed because rustc introduced more warnings or is
more strict about existing ones. Most of the stuff could just be fixed
and the commits should be pretty self explanatory. However, there are a
few this that are notable:

## `non_local_definitions `

A lot of runtimes to write `impl` blocks inside functions. This makes
sense to reduce the amount of conditional compilation. I guess I could
have moved them into a module instead. But I think allowing it here
makes sense to avoid the code churn.

## `unexpected_cfgs`

The FRAME macros emit code that references various features like `std`,
`runtime-benchmarks` or `try-runtime`. If a create that uses those
macros does not have those features we get this warning. Those were
mostly when defining a `mock` runtime. I opted for silencing the warning
in this case rather than adding not needed features.

For the benchmarking ui tests I opted for adding the `runtime-benchmark`
feature to the `Cargo.toml`.

## Failing UI test

I am bumping the `trybuild` version and regenerating the ui tests. The
old version seems to be incompatible. This requires us to pass
`deny_warnings` in `CARGO_ENCODED_RUSTFLAGS` as `RUSTFLAGS` is ignored
in the new version.

## Removing toolchain file from the pallet revive fixtures

This is no longer needed since the latest stable will compile them fine
using the `RUSTC_BOOTSTRAP=1`.

---------

Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

cc https://github.com/paritytech/ci_cd/issues/1088

PR removes short-benchmarks from GitLab, adds condition for
cargo-check-rutimes

cc https://github.com/paritytech/ci_cd/issues/1035

cc https://github.com/paritytech/ci_cd/issues/1023

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Maksym H <1177472+mordamax@users.noreply.github.com>
Co-authored-by: gui <gui.thiolliere@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: ggwpez <ggwpez@users.noreply.github.com>

PR removes cargo check job that runs on osx runner from gitlab and moves
`cargo-check-each-crate-macos` job to be required

cc https://github.com/paritytech/ci_cd/issues/1037

- [return macos jobs to
gitlab](https://github.com/paritytech/polkadot-sdk/commit/dcd44b1d)
- [add benches to merge
queue](https://github.com/paritytech/polkadot-sdk/commit/494eb21b)
- [require test-deterministic-wasm and run it
earlier](https://github.com/paritytech/polkadot-sdk/commit/ab9ae5ca)

Closes: https://github.com/paritytech/ci_cd/issues/1018


![image](https://github.com/user-attachments/assets/b434d809-2c38-47e9-8a62-448f32cf24cb)

Added DAG similar to how it was on Gitlab

---------

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>

PR moves jobs `test-linux-stable`, `test-linux-stable-int` and
`test-linux-stable-runtime-benchmarks` to github actions

cc https://github.com/paritytech/ci_cd/issues/1006

PR moves rococo and wococo check-runtime-migration jobs to GHA

cc https://github.com/paritytech/ci_cd/issues/1006

PR migrates jobs `quick-benchmarks`, `cargo-clippy`, `check-try-runtime`
and `check-core-crypto-features` from Gitlab to GitHub

cc https://github.com/paritytech/ci_cd/issues/1006

---------

Co-authored-by: Maksym H <1177472+mordamax@users.noreply.github.com>

PR changes forklift authentication to gcs

cc https://github.com/paritytech/ci_cd/issues/987

PR adds forklift settings and forklift to test-github-actions

cc https://github.com/paritytech/ci_cd/issues/939

close https://github.com/paritytech/ci_cd/issues/974

cc https://github.com/paritytech/ci_cd/issues/974

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <info@kchr.de>

Added check if forklift already exists in ci image as forklift binary is
now bundled with the ci-unified.
This is a temporary check for the transition period

Use of `- >` instead of `- |` workarounds GitLab quirk when it crops
collapsed multiline `script:` section commands in its CI job logs.
This PR also fixes `- |` based `script:` steps to behave properly after
`- >` conversion.

Resolves https://github.com/paritytech/ci_cd/issues/972.

PR adds CI jobs that collect subsystem-benchmarks results and publishes
them to gh-pages.

cc https://github.com/paritytech/ci_cd/issues/934
cc @AndreiEres

Add [forklift
caching](https://gitlab.parity.io/parity/infrastructure/ci_cd/forklift/forklift)
to remainig jobs

by .sh and .py scripts:
- cargo-check-each-crate x6 (`.gitlab/check-each-crate.py`)
- build-linux-stable (`polkadot/scripts/build-only-wasm.sh`)

by before_script:
- build-linux-substrate
- build-subkey-linux (with `.build-subkey` job)
- cargo-check-benches x2

**To disable feature set FORKLIFT_BYPASS variable to true in [project
settings in
gitlab](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/settings/ci_cd)**
(forklift now handles FORKLIFT_BYPASS by itself)

Usual chore after bumping Rust toolchain version.

cc https://github.com/paritytech/ci_cd/issues/926

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>

In CI jobs I see calls to `forklift clean`, which apparently does not
exist.
From CI logs:
```
1m$ forklift clean[0;m
Cargo cache management utility

Usage:
  forklift [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  serve       Run forklift coordinator server for current location
  start       Start detached forklift coordinator server for current location
  stop        Stop forklift coordinator server for current location

Flags:
  -c, --compression string     Compression algorithm to use
                               Available: none, xz (default "zstd")
  -h, --help                   help for forklift
  -p, --param stringToString   map of additional parameters
                                ex: -p S3_BUCKET_NAME=my_bucket (default [])
  -s, --storage string         Storage driver
                               Available: s3, fs (default "s3")
  -v, --verbose string         Available: panic, fatal, error, warn, warning, info, debug, trace (default "info")
      --version                version for forklift

Use "forklift [command] --help" for more information about a command.
```

<del>PR custom review is deprecated. Now `review-bot` performs these
functions.</del>
PR removes unused ci jobs and adjusts zombienet jobs for merge queues

cc @Bullrich

bump polkavm-x deps to 0.4.0

close https://github.com/paritytech/ci_cd/issues/900

cc https://github.com/paritytech/ci_cd/issues/900

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: command-bot <>

This PR adds cargo caching feature with custom ['forklift'
tool](https://gitlab.parity.io/parity/infrastructure/ci_cd/forklift/forklift)
Forklift acts as RUSTC_WRAPPER, intercepts rustc calls and stores
produced artifacts in S3 bucket (see [forklift
readme](https://gitlab.parity.io/parity/infrastructure/ci_cd/forklift/forklift/-/blob/main/README.MD?ref_type=heads)
for detailed description)

All settings are made in [`.forklift` job's
before_script](https://github.com/paritytech/polkadot-sdk/blob/es/forklift-test/.gitlab-ci.yml#L119)
and affect all jobs that extend `.docker-env` job

To disable feature set `FORKLIFT_BYPASS` variable to true in [project
settings in
gitlab](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/settings/ci_cd)

Docker images from paritypr are also used in testnets. PR reverts docker
tags naming to a more convenient.

cc @PierreBesson

Includes:
 - bump `pjs` modules versions
- re-enable test disable in
https://github.com/paritytech/polkadot-sdk/pull/2294

---------

Co-authored-by: Bastian Köcher <info@kchr.de>

New release includes logic to move all jobs to `spot instances`.
Thx!

PR prepares CI to the GitHub Merge Queues. All github actions that were
running in PR adjusted so they can run in the merge queues. Zombienet
jobs will do nothing during PRs but they will run during merge queues.

Jobs that will be skipped during PR:
 - all zombienet jobs
 - all publish docker jobs

Jobs that will be skipped during merge queue:
 - check-labels
 - check-prdoc
 - pr-custom-review
 - review trigger

cc https://github.com/paritytech/ci_cd/issues/862

This PR prepares chains specs for _native-runtime-free_  world.

This PR has following changes:
- `substrate`:
  - adds support for:
- JSON based `GenesisConfig` to `ChainSpec` allowing interaction with
runtime `GenesisBuilder` API.
- interacting with arbitrary runtime wasm blob to[
`chain-spec-builder`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/bin/utils/chain-spec-builder/src/lib.rs#L46)
command line util,
- removes
[`code`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/frame/system/src/lib.rs#L660)
from `system_pallet`
  - adds `code` to the `ChainSpec`
- deprecates
[`ChainSpec::from_genesis`](https://github.com/paritytech/substrate/blob/3ef576eaeb3f42610e85daecc464961cf1295570/client/chain-spec/src/chain_spec.rs#L263),
but also changes the signature of this method extending it with `code`
argument.
[`ChainSpec::builder()`](https://github.com/paritytech/substrate/blob/20bee680ed098be7239cf7a6b804cd4de267983e/client/chain-spec/src/chain_spec.rs#L507)
should be used instead.
- `polkadot`:
- all references to `RuntimeGenesisConfig` in `node/service` are
removed,
- all
`(kusama|polkadot|versi|rococo|wococo)_(staging|dev)_genesis_config`
functions now return the JSON patch for default runtime `GenesisConfig`,
  - `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,

- `cumulus`:
  - `ChainSpecBuilder` is used, `ChainSpec::from_genesis` is removed,
- _JSON_ patch configuration used instead of `RuntimeGenesisConfig
struct` in all chain specs.
  
---------

Co-authored-by: command-bot <>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Kevin Krone <kevin@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

CI image has been updated in the shared snippet, reverting the variable
back.

Run CI using new image with nightly 2023-11-01

cc https://github.com/paritytech/polkadot-sdk/issues/2113
cc https://github.com/paritytech/ci_cd/issues/896

This version includes:
- Move `spot` usage in CI to 50%
- Fix `PodMonitor`, metrics will be relayed to grafana

Includes:
-  PoC for new `spot instance` infra for jobs.
- Fixes in test-runner/cli

Co-authored-by: command-bot <>

Bump zombiente version. This version includes the fixes needed for
`mixnet`.
Thx!