Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • parity/mirrors/polkadot-sdk
1 result
Show changes
Showing
with 1587 additions and 629 deletions
name: Check the getting-started.sh script
# This workflow aims to make sure that the `getting-started.sh` script
# is functional and allows to build the templates
# on different operating systems.
#
# There are two jobs inside.
# One for systems that can run in a docker container, and one for macOS.
#
# Each job consists of:
# 1. Some necessary prerequisites for the workflow itself.
# 2. A first pass of the script, which will install dependencies and clone a template.
# 3. A second pass of the script, to make sure the behaviour is as expected.
# 4. Building the template - making sure it's buildable and runnable.
#
# The script is interacted with using the `expect` tool, which is available on all relevant systems.
# The steps are not re-used between macOS and other systems,
# because they are very similar but a little different.
# Additionally, macOS does NOT start from scratch here - for example, we have homebrew already installed.
#
# There are many combinations of systems, shells and templates.
# We test a selected handful of combinations here.
on:
pull_request:
paths:
- ".github/workflows/check-getting-started.yml"
- "scripts/getting-started.sh"
types: [opened, synchronize, reopened, ready_for_review]
schedule:
- cron: "0 5 * * *"
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
isdraft:
uses: ./.github/workflows/reusable-isdraft.yml
check-getting-started:
needs: isdraft
strategy:
fail-fast: true
matrix:
include:
- name: ubuntu
container: ubuntu
template: minimal
shell: bash
- name: debian
container: debian
template: parachain
shell: sh
- name: arch
container: archlinux
template: solochain
shell: sh
- name: fedora
container: fedora
template: parachain
shell: sh
- name: opensuse
container: opensuse/tumbleweed
template: solochain
shell: sh
runs-on: parity-large
container: ${{ matrix.container }}:latest
steps:
# A minimal amount of prerequisites required before we can run the actual getting-started script,
# which will install the rest of requirements.
- name: Install ubuntu/debian prerequisites
run: apt update && apt install -y expect sudo git
if: contains(matrix.name, 'ubuntu') || contains(matrix.name, 'debian')
- name: Install arch prerequisites
run: pacman -Syu --needed --noconfirm expect sudo git
if: contains(matrix.name, 'arch')
- name: Install fedora prerequisites
run: dnf --assumeyes install expect sudo git
if: contains(matrix.name, 'fedora')
- name: Install opensuse prerequisites
run: zypper install --no-confirm expect sudo git
if: contains(matrix.name, 'opensuse')
- name: Checkout
uses: actions/checkout@v4
- name: Set additional expect flags if necessary
run: |
# Add a debug flag to expect, if github is re-run with debug logging enabled.
[ "${{ runner.debug }}" = "1" ] && EXPECT_FLAGS="-d" || EXPECT_FLAGS=""
echo "EXPECT_FLAGS=${EXPECT_FLAGS}" >> $GITHUB_ENV
- name: Check the first run of the script
run: |
expect $EXPECT_FLAGS -c '
set timeout 240
spawn ${{ matrix.shell }} scripts/getting-started.sh
expect_after {
timeout { puts stderr "Timed out on an expect"; exit 1 }
eof { puts stderr "EOF received on an expect"; exit 1 }
}
expect -nocase "Detected ${{ matrix.name }}"
expect "Rust is not installed. Install it?" {
send "y\r"
expect "Proceed with standard installation (default - just press enter)" {
send "\r"
expect "Rust is installed now"
}
}
expect "Setup the Rust environment" {
send "y\r"
}
expect "start with one of the templates" {
send "y\r"
}
expect -re "(.)\\) ${{ matrix.template }} template" {
send "$expect_out(1,string)\r"
}
expect "compile the node?" {
send "n\r"
}
expect eof
'
timeout-minutes: 15
- name: Check the second run of the script
run: |
expect $EXPECT_FLAGS -c '
set timeout 120
spawn ${{ matrix.shell }} scripts/getting-started.sh
expect_after {
timeout { puts stderr "Timed out on an expect"; exit 1 }
eof { puts stderr "EOF received on an expect"; exit 1 }
}
expect "Rust already installed" {}
expect "Setup the Rust environment" {
send "n\r"
}
expect "start with one of the templates" {
send "y\r"
}
expect -re "(.)\\) ${{ matrix.template }} template" {
send "$expect_out(1,string)\r"
expect "directory already exists" {}
}
expect "compile the node?" {
send "n\r"
}
expect eof
'
timeout-minutes: 15
- name: Compile the node outside of the script
run: |
. "$HOME/.cargo/env"
cd ${{ matrix.template }}-template
cargo build --release
timeout-minutes: 120
- name: Check that the binary is executable
run: |
. "$HOME/.cargo/env"
cd ${{ matrix.template }}-template
cargo run --release -- --help
timeout-minutes: 5
check-getting-started-macos:
needs: isdraft
strategy:
fail-fast: true
matrix:
include:
- template: parachain
shell: sh
- template: solochain
shell: bash
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set additional expect flags if necessary
run: |
# Add a debug flag to expect, if github is re-run with debug logging enabled.
[ "${{ runner.debug }}" = "1" ] && EXPECT_FLAGS="-d" || EXPECT_FLAGS=""
echo "EXPECT_FLAGS=${EXPECT_FLAGS}" >> $GITHUB_ENV
- name: Check the first run of the script
run: |
expect $EXPECT_FLAGS -c '
set timeout 120
spawn ${{ matrix.shell }} scripts/getting-started.sh
expect_after {
timeout { puts stderr "Timed out on an expect"; exit 1 }
eof { puts stderr "EOF received on an expect"; exit 1 }
}
expect -nocase "Detected macOS"
expect "Homebrew already installed"
expect "Install cmake" {
send "y\r"
}
expect "Rust already installed" {}
expect "Setup the Rust environment" {
send "y\r"
}
expect "start with one of the templates" {
send "y\r"
}
expect -re "(.)\\) ${{ matrix.template }} template" {
send "$expect_out(1,string)\r"
}
expect "compile the node?" {
send "n\r"
}
expect eof
'
timeout-minutes: 15
- name: Check the second run of the script
run: |
expect $EXPECT_FLAGS -c '
set timeout 120
spawn ${{ matrix.shell }} scripts/getting-started.sh
expect_after {
timeout { puts stderr "Timed out on an expect"; exit 1 }
eof { puts stderr "EOF received on an expect"; exit 1 }
}
expect "Homebrew already installed"
expect "Install cmake" {
send "y\r"
}
expect "Rust already installed" {}
expect "Setup the Rust environment" {
send "n\r"
}
expect "start with one of the templates" {
send "y\r"
}
expect -re "(.)\\) ${{ matrix.template }} template" {
send "$expect_out(1,string)\r"
expect "directory already exists" {}
}
expect "compile the node?" {
send "n\r"
}
expect eof
'
timeout-minutes: 15
- name: Compile the node outside of the script
run: |
. "$HOME/.cargo/env"
cd ${{ matrix.template }}-template
cargo build --release
timeout-minutes: 120
- name: Check that the binary is executable
run: |
. "$HOME/.cargo/env"
cd ${{ matrix.template }}-template
cargo run --release -- --help
timeout-minutes: 5
name: Check labels
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
on:
pull_request:
types: [labeled, opened, synchronize, unlabeled]
......@@ -8,6 +12,7 @@ on:
jobs:
check-labels:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Check labels
env:
......
......@@ -2,22 +2,30 @@ name: Check licenses
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
merge_group:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
packages: read
jobs:
isdraft:
uses: ./.github/workflows/reusable-isdraft.yml
check-licenses:
runs-on: ubuntu-latest
needs: isdraft
timeout-minutes: 10
env:
LICENSES: "'Apache-2.0' 'GPL-3.0-only' 'GPL-3.0-or-later WITH Classpath-exception-2.0'"
LICENSES: "'Apache-2.0' 'GPL-3.0-only' 'GPL-3.0-or-later WITH Classpath-exception-2.0' 'MIT-0' 'Unlicense'"
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout sources
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/setup-node@v4.0.3
uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- uses: actions/setup-node@v4.1.0
with:
node-version: "18.x"
registry-url: "https://npm.pkg.github.com"
......@@ -28,19 +36,59 @@ jobs:
shopt -s globstar
npx @paritytech/license-scanner scan \
--ensure-licenses ${{ env.LICENSES }} \
-- ./polkadot/**/*.rs
--file-extensions '.rs' \
-- ./polkadot
- name: Check the licenses in Cumulus
run: |
shopt -s globstar
npx @paritytech/license-scanner scan \
--ensure-licenses ${{ env.LICENSES }} \
--exclude ./cumulus/parachain-template \
-- ./cumulus/**/*.rs
--ensure-licenses ${{ env.LICENSES }} 'Unlicense' \
--file-extensions '.rs' \
-- ./cumulus
- name: Check the licenses in Substrate
run: |
shopt -s globstar
npx @paritytech/license-scanner scan \
--ensure-licenses ${{ env.LICENSES }} \
-- ./substrate/**/*.rs
--file-extensions '.rs' \
-- ./substrate
check-product-references:
runs-on: ubuntu-latest
timeout-minutes: 10
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout sources
uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- uses: actions/setup-node@v4.1.0
with:
node-version: "18.x"
registry-url: "https://npm.pkg.github.com"
scope: "@paritytech"
- name: Check the product references in Polkadot
run: |
shopt -s globstar
npx @paritytech/license-scanner scan \
--ensure-product 'Polkadot' \
--file-extensions '.rs' \
-- ./polkadot
- name: Check the product references in Cumulus
run: |
shopt -s globstar
npx @paritytech/license-scanner scan \
--ensure-product 'Cumulus' \
--file-extensions '.rs' \
-- ./cumulus
- name: Check the product references in Substrate
run: |
shopt -s globstar
npx @paritytech/license-scanner scan \
--ensure-product 'Substrate' \
--file-extensions '.rs' \
-- ./substrate
......@@ -10,12 +10,20 @@ on:
types: [opened, synchronize, reopened, ready_for_review]
merge_group:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
packages: read
jobs:
isdraft:
uses: ./.github/workflows/reusable-isdraft.yml
link-checker:
runs-on: ubuntu-latest
needs: isdraft
timeout-minutes: 10
steps:
- name: Restore lychee cache
uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v3.3.2 (7. Sep 2023)
......@@ -25,10 +33,10 @@ jobs:
# This should restore from the most recent one:
restore-keys: cache-lychee-
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.0 (22. Sep 2023)
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.0 (22. Sep 2023)
- name: Lychee link checker
uses: lycheeverse/lychee-action@2b973e86fc7b1f6b36a93795fe2c9c6ae1118621 # for v1.9.1 (10. Jan 2024)
uses: lycheeverse/lychee-action@f613c4a64e50d792e0b31ec34bbcbba12263c6a6 # for v1.9.1 (10. Jan 2024)
with:
args: >-
--config .config/lychee.toml
......
name: Check PRdoc
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
on:
pull_request:
types: [labeled, opened, synchronize, unlabeled]
merge_group:
env:
IMAGE: docker.io/paritytech/prdoc:v0.0.8
IMAGE: docker.io/paritytech/prdoc:v0.1.1
API_BASE: https://api.github.com/repos
REPO: ${{ github.repository }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
......@@ -17,49 +21,34 @@ env:
jobs:
check-prdoc:
runs-on: ubuntu-latest
if: github.event.pull_request.number != ''
timeout-minutes: 10
steps:
- name: Checkout repo
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 #v4.1.7
# we cannot show the version in this step (ie before checking out the repo)
# due to https://github.com/paritytech/prdoc/issues/15
- name: Check if PRdoc is required
id: get-labels
uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc #v4.1.7
- name: Check prdoc format
run: |
echo "Pulling $IMAGE"
$ENGINE pull $IMAGE
# Fetch the labels for the PR under test
echo "Fetch the labels for $API_BASE/${REPO}/pulls/${GITHUB_PR}"
labels=$( curl -H "Authorization: token ${GITHUB_TOKEN}" -s "$API_BASE/${REPO}/pulls/${GITHUB_PR}" | jq '.labels | .[] | .name' | tr "\n" ",")
echo "Labels: ${labels}"
echo "labels=${labels}" >> "$GITHUB_OUTPUT"
echo "Checking PRdoc version"
$ENGINE run --rm -v $PWD:/repo $IMAGE --version
- name: Early exit if PR is silent
if: ${{ contains(steps.get-labels.outputs.labels, 'R0') }}
run: |
hits=$(find prdoc -name "pr_$GITHUB_PR*.prdoc" | wc -l)
if (( hits > 0 )); then
echo "PR detected as silent, but a PRDoc was found, checking it as information"
$ENGINE run --rm -v $PWD:/repo $IMAGE check -n ${GITHUB_PR} || echo "Ignoring failure"
else
echo "PR detected as silent, no PRDoc found, exiting..."
fi
echo "If you want to add a PRDoc, please refer to $PRDOC_DOC"
exit 0
echo "Check prdoc format"
echo "For PRDoc format, please refer to $PRDOC_DOC"
$ENGINE run --rm -v $PWD:/repo -e RUST_LOG=info $IMAGE check
- name: PRdoc check for PR#${{ github.event.pull_request.number }}
if: ${{ !contains(steps.get-labels.outputs.labels, 'R0') }}
- name: Check if PRdoc is required
if: github.event.pull_request.number != ''
id: get-labels
run: |
echo "Checking for PR#${GITHUB_PR}"
echo "You can find more information about PRDoc at $PRDOC_DOC"
$ENGINE run --rm -v $PWD:/repo -e RUST_LOG=info $IMAGE check -n ${GITHUB_PR}
# Fetch the labels for the PR under test
echo "Fetch the labels for $API_BASE/${REPO}/pulls/${GITHUB_PR}"
labels=$( curl -H "Authorization: token ${GITHUB_TOKEN}" -s "$API_BASE/${REPO}/pulls/${GITHUB_PR}" | jq '.labels | .[] | .name' | tr "\n" ",")
echo "Labels: ${labels}"
echo "labels=${labels}" >> "$GITHUB_OUTPUT"
- name: Validate prdoc for PR#${{ github.event.pull_request.number }}
if: ${{ !contains(steps.get-labels.outputs.labels, 'R0') }}
if: ${{ github.event.pull_request.number != '' && !contains(steps.get-labels.outputs.labels, 'R0') }}
run: |
echo "Validating PR#${GITHUB_PR}"
python3 --version
......
......@@ -6,6 +6,9 @@ on:
- master
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
# Take a snapshot at 5am when most SDK devs are not working.
schedule:
- cron: "0 5 * * *"
merge_group:
workflow_dispatch:
......@@ -13,95 +16,72 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions: {}
jobs:
set-image:
# GitHub Actions allows using 'env' in a container context.
# However, env variables don't work for forks: https://github.com/orgs/community/discussions/44322
# This workaround sets the container image for each job using 'set-image' job output.
runs-on: ubuntu-latest
outputs:
IMAGE: ${{ steps.set_image.outputs.IMAGE }}
steps:
- name: Checkout
uses: actions/checkout@v4
- id: set_image
run: cat .github/env >> $GITHUB_OUTPUT
# rococo and westend are disabled for now (no access to parity-chains.parity.io)
isdraft:
uses: ./.github/workflows/reusable-isdraft.yml
preflight:
needs: isdraft
uses: ./.github/workflows/reusable-preflight.yml
# More info can be found here: https://github.com/paritytech/polkadot/pull/5865
check-runtime-migration:
runs-on: arc-runners-polkadot-sdk-beefy
runs-on: ${{ needs.preflight.outputs.RUNNER }}
if: ${{ needs.preflight.outputs.changes_rust }}
# We need to set this to rather long to allow the snapshot to be created, but the average time
# should be much lower.
timeout-minutes: 60
needs: [set-image]
needs: [preflight]
container:
image: ${{ needs.set-image.outputs.IMAGE }}
image: ${{ needs.preflight.outputs.IMAGE }}
strategy:
fail-fast: false
matrix:
network: [
# westend,
# rococo,
network:
[
westend,
asset-hub-westend,
asset-hub-rococo,
bridge-hub-westend,
bridge-hub-rococo,
contracts-rococo,
collectives-westend,
coretime-rococo,
coretime-westend,
]
include:
# - network: westend
# package: westend-runtime
# wasm: westend_runtime.compact.compressed.wasm
# uri: "wss://westend-try-runtime-node.parity-chains.parity.io:443"
# subcommand_extra_args: "--no-weight-warnings"
# command_extra_args: ""
# - network: rococo
# package: rococo-runtime
# wasm: rococo_runtime.compact.compressed.wasm
# uri: "wss://rococo-try-runtime-node.parity-chains.parity.io:443"
# subcommand_extra_args: "--no-weight-warnings"
# command_extra_args: ""
- network: westend
package: westend-runtime
wasm: westend_runtime.compact.compressed.wasm
uri: "wss://try-runtime-westend.polkadot.io:443"
subcommand_extra_args: "--no-weight-warnings --blocktime 6000"
command_extra_args: ""
- network: asset-hub-westend
package: asset-hub-westend-runtime
wasm: asset_hub_westend_runtime.compact.compressed.wasm
uri: "wss://westend-asset-hub-rpc.polkadot.io:443"
subcommand_extra_args: ""
command_extra_args: ""
- network: "asset-hub-rococo"
package: "asset-hub-rococo-runtime"
wasm: "asset_hub_rococo_runtime.compact.compressed.wasm"
uri: "wss://rococo-asset-hub-rpc.polkadot.io:443"
subcommand_extra_args: ""
subcommand_extra_args: " --blocktime 6000"
command_extra_args: ""
- network: "bridge-hub-westend"
package: "bridge-hub-westend-runtime"
wasm: "bridge_hub_westend_runtime.compact.compressed.wasm"
- network: bridge-hub-westend
package: bridge-hub-westend-runtime
wasm: bridge_hub_westend_runtime.compact.compressed.wasm
uri: "wss://westend-bridge-hub-rpc.polkadot.io:443"
- network: "bridge-hub-rococo"
package: "bridge-hub-rococo-runtime"
wasm: "bridge_hub_rococo_runtime.compact.compressed.wasm"
uri: "wss://rococo-bridge-hub-rpc.polkadot.io:443"
- network: "contracts-rococo"
package: "contracts-rococo-runtime"
wasm: "contracts_rococo_runtime.compact.compressed.wasm"
uri: "wss://rococo-contracts-rpc.polkadot.io:443"
- network: "collectives-westend"
package: "collectives-westend-runtime"
wasm: "collectives_westend_runtime.compact.compressed.wasm"
subcommand_extra_args: " --blocktime 6000"
- network: collectives-westend
package: collectives-westend-runtime
wasm: collectives_westend_runtime.compact.compressed.wasm
uri: "wss://westend-collectives-rpc.polkadot.io:443"
command_extra_args: "--disable-spec-name-check"
- network: "coretime-rococo"
package: "coretime-rococo-runtime"
wasm: "coretime_rococo_runtime.compact.compressed.wasm"
uri: "wss://rococo-coretime-rpc.polkadot.io:443"
subcommand_extra_args: " --blocktime 6000"
- network: coretime-westend
package: coretime-westend-runtime
wasm: coretime_westend_runtime.compact.compressed.wasm
uri: "wss://westend-coretime-rpc.polkadot.io:443"
subcommand_extra_args: " --blocktime 6000"
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Download CLI
run: |
curl -sL https://github.com/paritytech/try-runtime-cli/releases/download/v0.7.0/try-runtime-x86_64-unknown-linux-musl -o try-runtime
curl -sL https://github.com/paritytech/try-runtime-cli/releases/download/v0.8.0/try-runtime-x86_64-unknown-linux-musl -o try-runtime
chmod +x ./try-runtime
echo "Using try-runtime-cli version:"
./try-runtime --version
......@@ -126,17 +106,44 @@ jobs:
./try-runtime create-snapshot --uri ${{ matrix.uri }} snapshot.raw
- name: Build Runtime
id: required1
run: |
echo "---------- Building ${{ matrix.package }} runtime ----------"
time forklift cargo build --release --locked -p ${{ matrix.package }} --features try-runtime -q
forklift cargo build --release --locked -p ${{ matrix.package }} --features try-runtime -q
- name: Run Check
id: required2
run: |
echo "Running ${{ matrix.network }} runtime migration check"
export RUST_LOG=remote-ext=debug,runtime=debug
echo "---------- Executing on-runtime-upgrade for ${{ matrix.network }} ----------"
time ./try-runtime ${{ matrix.command_extra_args }} \
./try-runtime ${{ matrix.command_extra_args }} \
--runtime ./target/release/wbuild/${{ matrix.package }}/${{ matrix.wasm }} \
on-runtime-upgrade --disable-spec-version-check --checks=all ${{ matrix.subcommand_extra_args }} snap -p snapshot.raw
sleep 5
- name: Stop all workflows if failed
if: ${{ failure() && (steps.required1.conclusion == 'failure' || steps.required2.conclusion == 'failure') }}
uses: ./.github/actions/workflow-stopper
with:
app-id: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_ID }}
app-key: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_KEY }}
# name of this job must be unique across all workflows
# otherwise GitHub will mark all these jobs as required
confirm-required-checks-passed:
runs-on: ubuntu-latest
name: All runtime migrations passed
# If any new job gets added, be sure to add it to this array
needs: [check-runtime-migration]
if: always() && !cancelled()
steps:
- run: |
tee resultfile <<< '${{ toJSON(needs) }}'
FAILURES=$(cat resultfile | grep '"result": "failure"' | wc -l)
if [ $FAILURES -gt 0 ]; then
echo "### At least one required job failed ❌" >> $GITHUB_STEP_SUMMARY
exit 1
else
echo '### Good job! All the required jobs passed 🚀' >> $GITHUB_STEP_SUMMARY
fi
......@@ -2,32 +2,39 @@ name: Check semver
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]
workflow_dispatch:
merge_group:
concurrency:
group: check-semver-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env:
TOOLCHAIN: nightly-2024-06-01
TOOLCHAIN: nightly-2024-11-19
jobs:
isdraft:
uses: ./.github/workflows/reusable-isdraft.yml
preflight:
needs: isdraft
uses: ./.github/workflows/reusable-preflight.yml
check-semver:
runs-on: ubuntu-latest
timeout-minutes: 90
needs: [preflight]
container:
image: docker.io/paritytech/ci-unified:bullseye-1.77.0-2024-04-10-v20240408
image: ${{ needs.preflight.outputs.IMAGE }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
with:
fetch-depth: 2
- name: extra git setup
env:
BASE: ${{ github.event.pull_request.base.sha }}
run: |
git config --global --add safe.directory '*'
git fetch --no-tags --no-recurse-submodules --depth=1 origin $BASE
git branch old $BASE
git branch old HEAD^1
- name: Comment If Backport
if: ${{ startsWith(github.event.pull_request.base.ref, 'stable') }}
......@@ -37,16 +44,12 @@ jobs:
run: |
echo "This is a backport into stable."
wget -q https://github.com/cli/cli/releases/download/v2.51.0/gh_2.51.0_linux_amd64.tar.gz -O gh.tar.gz && \
tar -xzf gh.tar.gz && mv gh_2.51.0_linux_amd64/bin/gh /usr/local/bin/gh && rm gh.tar.gz
chmod +x /usr/local/bin/gh
cat > msg.txt <<EOF
This pull request is amending an existing release. Please proceed with extreme caution,
as to not impact downstream teams that rely on the stability of it. Some things to consider:
- Backports are only for 'patch' or 'minor' changes. No 'major' or other breaking change.
- Should be a legit *fix* for some bug, not adding tons of new features.
- Must either be already audited or trivial (not sure audit).
- Must either be already audited or not need an audit.
<details><summary><i>Emergency Bypass</i></summary>
<p>
......@@ -62,22 +65,36 @@ jobs:
echo "PRDOC_EXTRA_ARGS=--max-bump minor" >> $GITHUB_ENV
- name: Echo Skip
if: ${{ contains(github.event.pull_request.labels.*.name, 'R0-silent') }}
run: echo "Skipping this PR because it is labeled as R0-silent."
- name: Rust Cache
uses: Swatinem/rust-cache@23bce251a8cd2ffc3c1075eaa2367cf899916d84 # v2.7.3
if: ${{ !contains(github.event.pull_request.labels.*.name, 'R0-silent') }}
uses: Swatinem/rust-cache@f0deed1e0edfc6a9be95417288c0e1099b1eeec3 # v2.7.7
with:
cache-on-failure: true
- name: Rust compilation prerequisites
if: ${{ !contains(github.event.pull_request.labels.*.name, 'R0-silent') }}
run: |
rustup default $TOOLCHAIN
rustup target add wasm32-unknown-unknown --toolchain $TOOLCHAIN
rustup component add rust-src --toolchain $TOOLCHAIN
- name: install parity-publish
if: ${{ !contains(github.event.pull_request.labels.*.name, 'R0-silent') }}
# Set the target dir to cache the build.
run: CARGO_TARGET_DIR=./target/ cargo install parity-publish@0.8.0 -q
run: CARGO_TARGET_DIR=./target/ cargo install parity-publish@0.10.4 --locked -q
- name: check semver
if: ${{ !contains(github.event.pull_request.labels.*.name, 'R0-silent') }}
run: |
if [ -z "$PR" ]; then
echo "Skipping master/merge queue"
exit 0
fi
export CARGO_TARGET_DIR=target
export RUSTFLAGS='-A warnings -A missing_docs'
export SKIP_WASM_BUILD=1
......
......@@ -15,34 +15,35 @@ concurrency:
permissions: {}
jobs:
set-image:
# GitHub Actions allows using 'env' in a container context.
# However, env variables don't work for forks: https://github.com/orgs/community/discussions/44322
# This workaround sets the container image for each job using 'set-image' job output.
runs-on: ubuntu-latest
timeout-minutes: 20
outputs:
IMAGE: ${{ steps.set_image.outputs.IMAGE }}
steps:
- name: Checkout
uses: actions/checkout@v4
- id: set_image
run: cat .github/env >> $GITHUB_OUTPUT
isdraft:
uses: ./.github/workflows/reusable-isdraft.yml
preflight:
needs: isdraft
uses: ./.github/workflows/reusable-preflight.yml
fmt:
runs-on: ubuntu-latest
timeout-minutes: 20
needs: [set-image]
needs: [preflight]
container:
image: ${{ needs.set-image.outputs.IMAGE }}
image: ${{ needs.preflight.outputs.IMAGE }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: Cargo fmt
id: required
run: cargo +nightly fmt --all -- --check
- name: Stop all workflows if failed
if: ${{ failure() && steps.required.conclusion == 'failure' && !github.event.pull_request.head.repo.fork }}
uses: ./.github/actions/workflow-stopper
with:
app-id: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_ID }}
app-key: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_KEY }}
check-dependency-rules:
runs-on: ubuntu-latest
needs: isdraft
timeout-minutes: 20
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: check dependency rules
run: |
cd substrate/
......@@ -50,11 +51,11 @@ jobs:
check-rust-feature-propagation:
runs-on: ubuntu-latest
timeout-minutes: 20
needs: [set-image]
needs: [preflight]
container:
image: ${{ needs.set-image.outputs.IMAGE }}
image: ${{ needs.preflight.outputs.IMAGE }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: fetch deps
run: |
# Pull all dependencies eagerly:
......@@ -66,30 +67,31 @@ jobs:
test-rust-features:
runs-on: ubuntu-latest
timeout-minutes: 20
needs: [set-image]
needs: [preflight]
container:
image: ${{ needs.set-image.outputs.IMAGE }}
image: ${{ needs.preflight.outputs.IMAGE }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: run rust features
run: bash .gitlab/rust-features.sh .
check-toml-format:
runs-on: ubuntu-latest
timeout-minutes: 20
needs: [set-image]
needs: [preflight]
container:
image: ${{ needs.set-image.outputs.IMAGE }}
image: ${{ needs.preflight.outputs.IMAGE }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: check toml format
run: |
taplo format --check --config .config/taplo.toml
echo "Please run `taplo format --config .config/taplo.toml` to fix any toml formatting issues"
check-workspace:
runs-on: ubuntu-latest
needs: isdraft
timeout-minutes: 20
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.0 (22. Sep 2023)
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.0 (22. Sep 2023)
- name: install python deps
run: |
sudo apt-get update && sudo apt-get install -y python3-pip python3
......@@ -104,12 +106,13 @@ jobs:
run: python3 .github/scripts/deny-git-deps.py .
check-markdown:
runs-on: ubuntu-latest
needs: isdraft
timeout-minutes: 20
steps:
- name: Checkout sources
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: Setup Node.js
uses: actions/setup-node@v4.0.3
uses: actions/setup-node@v4.1.0
with:
node-version: "18.x"
registry-url: "https://npm.pkg.github.com"
......@@ -128,11 +131,11 @@ jobs:
check-umbrella:
runs-on: ubuntu-latest
timeout-minutes: 20
needs: [set-image]
needs: [preflight]
container:
image: ${{ needs.set-image.outputs.IMAGE }}
image: ${{ needs.preflight.outputs.IMAGE }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.0 (22. Sep 2023)
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.0 (22. Sep 2023)
- name: install python deps
run: pip3 install "cargo-workspace>=1.2.4" toml
- name: check umbrella correctness
......@@ -140,7 +143,7 @@ jobs:
# Fixes "detected dubious ownership" error in the ci
git config --global --add safe.directory '*'
python3 scripts/generate-umbrella.py --sdk . --version 0.1.0
cargo +nightly fmt --all
cargo +nightly fmt -p polkadot-sdk
if [ -n "$(git status --porcelain)" ]; then
cat <<EOF
......@@ -154,3 +157,83 @@ jobs:
git diff
exit 1
fi
check-fail-ci:
runs-on: ubuntu-latest
needs: isdraft
container:
# there's no "rg" in ci-unified, and tools is a smaller image anyway
image: "paritytech/tools:latest"
# paritytech/tools uses "nonroot" user by default, which doesn't have enough
# permissions to create GHA context
options: --user root
steps:
- name: Fetch latest code
uses: actions/checkout@v4
- name: Check
run: |
set +e
rg --line-number --hidden --type rust --glob '!{.git,target}' "$ASSERT_REGEX" .; exit_status=$?
if [ $exit_status -eq 0 ]; then
echo "$ASSERT_REGEX was found, exiting with 1";
exit 1;
else
echo "No $ASSERT_REGEX was found, exiting with 0";
exit 0;
fi
env:
ASSERT_REGEX: "FAIL-CI"
GIT_DEPTH: 1
check-readme:
runs-on: ubuntu-latest
needs: isdraft
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: Install prerequisites
run: |
sudo apt-get update
sudo apt-get install -y protobuf-compiler
- name: Set rust version from env file
run: |
RUST_VERSION=$(cat .github/env | sed -E 's/.*ci-unified:([^-]+)-([^-]+).*/\2/')
echo $RUST_VERSION
echo "RUST_VERSION=${RUST_VERSION}" >> $GITHUB_ENV
- name: Install Rust
uses: actions-rust-lang/setup-rust-toolchain@11df97af8e8102fd60b60a77dfbf58d40cd843b8 # v1.10.1
with:
cache: false
toolchain: ${{ env.RUST_VERSION }}
components: cargo, clippy, rust-docs, rust-src, rustfmt, rustc, rust-std
- name: Find README.docify.md files and check generated READMEs
run: .github/scripts/check-missing-readme-generation.sh
confirm-required-checks-quick-jobs-passed:
runs-on: ubuntu-latest
name: All quick checks passed
# If any new job gets added, be sure to add it to this array
needs:
- fmt
- check-dependency-rules
- check-rust-feature-propagation
- test-rust-features
- check-toml-format
- check-workspace
- check-markdown
- check-umbrella
- check-fail-ci
- check-readme
if: always() && !cancelled()
steps:
- run: |
tee resultfile <<< '${{ toJSON(needs) }}'
FAILURES=$(cat resultfile | grep '"result": "failure"' | wc -l)
if [ $FAILURES -gt 0 ]; then
echo "### At least one required job failed ❌" >> $GITHUB_STEP_SUMMARY
exit 1
else
echo '### Good job! All the required jobs passed 🚀' >> $GITHUB_STEP_SUMMARY
fi
name: checks
name: Checks
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened, ready_for_review, labeled]
types: [opened, synchronize, reopened, ready_for_review]
merge_group:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
......@@ -14,70 +15,74 @@ concurrency:
permissions: {}
jobs:
changes:
# TODO: remove once migration is complete or this workflow is fully stable
if: contains(github.event.label.name, 'GHA-migration')
permissions:
pull-requests: read
uses: ./.github/workflows/reusable-check-changed-files.yml
set-image:
# GitHub Actions allows using 'env' in a container context.
# However, env variables don't work for forks: https://github.com/orgs/community/discussions/44322
# This workaround sets the container image for each job using 'set-image' job output.
runs-on: ubuntu-latest
timeout-minutes: 20
outputs:
IMAGE: ${{ steps.set_image.outputs.IMAGE }}
steps:
- name: Checkout
uses: actions/checkout@v4
- id: set_image
run: cat .github/env >> $GITHUB_OUTPUT
isdraft:
uses: ./.github/workflows/reusable-isdraft.yml
preflight:
needs: isdraft
uses: ./.github/workflows/reusable-preflight.yml
cargo-clippy:
runs-on: arc-runners-polkadot-sdk-beefy
needs: [set-image, changes] # , build-frame-omni-bencher ]
if: ${{ needs.changes.outputs.rust }}
runs-on: ${{ needs.preflight.outputs.RUNNER }}
needs: [preflight]
if: ${{ needs.preflight.outputs.changes_rust }}
timeout-minutes: 40
container:
image: ${{ needs.set-image.outputs.IMAGE }}
image: ${{ needs.preflight.outputs.IMAGE }}
env:
RUSTFLAGS: "-D warnings"
SKIP_WASM_BUILD: 1
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: script
id: required
run: |
forklift cargo clippy --all-targets --locked --workspace
forklift cargo clippy --all-targets --all-features --locked --workspace
cargo clippy --all-targets --locked --workspace --quiet
cargo clippy --all-targets --all-features --locked --workspace --quiet
- name: Stop all workflows if failed
if: ${{ failure() && steps.required.conclusion == 'failure' && !github.event.pull_request.head.repo.fork }}
uses: ./.github/actions/workflow-stopper
with:
app-id: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_ID }}
app-key: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_KEY }}
check-try-runtime:
runs-on: arc-runners-polkadot-sdk-beefy
needs: [set-image, changes] # , build-frame-omni-bencher ]
if: ${{ needs.changes.outputs.rust }}
runs-on: ${{ needs.preflight.outputs.RUNNER }}
needs: [preflight]
if: ${{ needs.preflight.outputs.changes_rust }}
timeout-minutes: 40
container:
image: ${{ needs.set-image.outputs.IMAGE }}
image: ${{ needs.preflight.outputs.IMAGE }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: script
id: required
run: |
forklift cargo check --locked --all --features try-runtime
forklift cargo check --locked --all --features try-runtime --quiet
# this is taken from cumulus
# Check that parachain-template will compile with `try-runtime` feature flag.
forklift cargo check --locked -p parachain-template-node --features try-runtime
# add after https://github.com/paritytech/substrate/pull/14502 is merged
# experimental code may rely on try-runtime and vice-versa
forklift cargo check --locked --all --features try-runtime,experimental
forklift cargo check --locked --all --features try-runtime,experimental --quiet
- name: Stop all workflows if failed
if: ${{ failure() && steps.required.conclusion == 'failure' && !github.event.pull_request.head.repo.fork }}
uses: ./.github/actions/workflow-stopper
with:
app-id: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_ID }}
app-key: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_KEY }}
# check-core-crypto-features works fast without forklift
check-core-crypto-features:
runs-on: arc-runners-polkadot-sdk-beefy
needs: [set-image, changes] # , build-frame-omni-bencher ]
if: ${{ needs.changes.outputs.rust }}
runs-on: ${{ needs.preflight.outputs.RUNNER }}
needs: [preflight]
if: ${{ needs.preflight.outputs.changes_rust }}
timeout-minutes: 30
container:
image: ${{ needs.set-image.outputs.IMAGE }}
image: ${{ needs.preflight.outputs.IMAGE }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@6d193bf28034eafb982f37bd894289fe649468fc # v4.1.7
- name: script
id: required
run: |
cd substrate/primitives/core
./check-features-variants.sh
......@@ -88,3 +93,27 @@ jobs:
cd substrate/primitives/keyring
./check-features-variants.sh
cd -
- name: Stop all workflows if failed
if: ${{ failure() && steps.required.conclusion == 'failure' && !github.event.pull_request.head.repo.fork }}
uses: ./.github/actions/workflow-stopper
with:
app-id: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_ID }}
app-key: ${{ secrets.WORKFLOW_STOPPER_RUNNER_APP_KEY }}
# name of this job must be unique across all workflows
# otherwise GitHub will mark all these jobs as required
confirm-required-checks-passed:
runs-on: ubuntu-latest
name: All checks passed
# If any new job gets added, be sure to add it to this array
needs: [cargo-clippy, check-try-runtime, check-core-crypto-features]
if: always() && !cancelled()
steps:
- run: |
tee resultfile <<< '${{ toJSON(needs) }}'
FAILURES=$(cat resultfile | grep '"result": "failure"' | wc -l)
if [ $FAILURES -gt 0 ]; then
echo "### At least one required job failed ❌" >> $GITHUB_STEP_SUMMARY
exit 1
else
echo '### Good job! All the required jobs passed 🚀' >> $GITHUB_STEP_SUMMARY
fi
This diff is collapsed.
name: Command Bot Tests
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
isdraft:
uses: ./.github/workflows/reusable-isdraft.yml
test-cmd-bot:
runs-on: ubuntu-latest
needs: [isdraft]
steps:
- uses: actions/checkout@v4
- run: python3 .github/scripts/cmd/test_cmd.py
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
......@@ -2,20 +2,21 @@ name: Inform of new command action
on:
issue_comment:
types: [created]
types: [ created ]
jobs:
comment:
runs-on: ubuntu-latest
# Temporary disable the bot until the new command bot works properly
if: github.event.issue.pull_request && startsWith(github.event.comment.body, 'bot ')
steps:
- name: Inform that the new command exist
uses: actions/github-script@v7
with:
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: 'We are migrating the command bot to be a GitHub Action<br/><br/>Please, see the <a href="https://github.com/paritytech/polkadot-sdk/blob/master/.github/commands-readme.md">documentation on how to use it</a>'
})
- name: Inform that the new command exist
uses: actions/github-script@v7
with:
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: 'We have migrated the command bot to GHA<br/><br/>Please, see the new usage instructions <a href="https://github.com/paritytech/polkadot-sdk/blob/master/docs/contributor/commands-readme.md">here</a> or <a href="https://forum.parity.io/t/streamlining-weight-generation-and-more-the-new-cmd-bot/2411">here</a>. Soon the old commands will be disabled.'
})
This diff is collapsed.