* test (ci): prepopulate caches with debug

* test (ci): error in path

* optimization (ci): CARGO_HOME cache prepopulation

* add post-build-test stage

* check status

* fix busybox shell

* renaming for consistency

* Enable `wasmtime` only for `x86_64` in substrate node

When using the new feature of cargo `cargo build -Z features=itarget`,
the `wasmtime` feature will not bleed into the build for other
architectures.

* Use new cargo feature in CI

* primitives: add missing cargo metadata to sp-arithmetic-fuzzer

* ci: add sp-arithmetic-fuzzer to unleash skip list

* add cargo-deny job

* feat(ci): add cargo-deny CI job, checks registries, licenses, dependencies dupes and advisories

* check_polkadot: move to external script

* check_polkadot: check for companion pr

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Add `runtime-benchmarks` compile

* Use cargo check and no wasm build

* Make sure frame examples compile for wasm

This makes sure that `frame-example` and `frame-example-offchain-worker`
compile for wasm.

This also fixes compilation for these crates. The offchain worker
example doesn't use serde-json anymore as that is too heavy and breaks
`no_std` compilation.

* Apply suggestions from code review

Co-Authored-By: Nikolay Volf <[email protected]>

Co-authored-by: Nikolay Volf <[email protected]>

* Adding missing metadata for benchmark-pallet

* Missed to bump a few

* release on roling tag

* Wasm executors were missed

* test run warnings test

* fix dependency

* fail on warnings is now necessary

* Bump to alpha.3

* update gitlab-ci

* do not check unleash on every PR, only master and tags

* move scripts folder

* add signed-tag check to CI

* remove publish-to-crates-io dependencies

Co-authored-by: s3krit <[email protected]>

* adding unleash to ci

* fixing formatting

* with a dot please

* alpha.3 now

* do not publish testing helpers

* remove old test-helpers cruft

* fix cargo.lock

* with alpha 4

* do not publish runtime-interface-test either

* disable more test crates from publishing

* switch to alpha.5

* replace tempdir with tempfile

* update lru

* switch to bytes 0.5

* release script fixes

* switch on and to latest alpha

* BUT THE SPACES

Idea is much the same as we currently do on polkadot - When a new tag is pushed (that fits our pattern for tags, e.g. v1.2.3), find a list of labelled changes and generate a changelog based on that. Create a draft release on Github and post about it on Matrix.

* build for pre-tags

* shallow clone rustdocs

These are a few changes I missed during the refactoring.

1. Initialization issue and boilerplate

    Most importantly: part of the `Configuration` initialization was done in `sc_cli::init`. This means the user can not benefit from this initialization boilerplate if they have multiple `Configuration` since `sc_cli::init` can only be called once.

2. Boilerplate for `VersionInfo` and `Configuration`

    I'm also answering to the critic of @bkchr on the initialization using version: https://github.com/paritytech/substrate/pull/4692/files/bea809d4c14a2ede953227ac885e3b3f9771c548#r372047238 This will allow initializing a `Configuration` and provide the version by default.

3. Loading the `chain_spec` explicitly

    In the past it was done automatically but in some cases we want to delay this. I moved the code to `Configuration.load_spec()` so it can be called later on. `chain_spec` can also be written directly to the `Configuration` without using this `load_spec` helper.

4. [deleted]

5. Fixing issue that prevents the user to override the port

    In the refactoring I introduced a bug by mistake that could potentially prevent the CLI user to override the ports if defaults where provided for these ports (only on cumulus).

6. Change task_executor from Box to Arc

    This is useful for cumulus where we have 2 nodes with 2 separate Configuration that need to spawn tasks to the same runtime.

7. Renamed TasksExecutorRequired to TaskExecutor

    For consistency.

This is related to https://github.com/paritytech/cumulus/issues/24

This is the continuation (and hopefully the end of) #4692 

* fixes trigger-contracts-ci job to actually depend on build-linux-substrate

* ci: increase retention for logs of tests to 144 hours

* change to days

* triggers srml-contracts-waterfall ci

* only nightly and merges

* substrate does not use tager cache so far

* build node job is separated from build substrate; less dupes, cleanup

* it's not effective yet to split these jobs

* ci: increase git cloning depth to 100

* check_runtime: verify shallow git clones to contain origin/master

* check_runtime: fetch master branch

* rename crate: sp-transaction-pool-api -> sp-transaction-pool

* move primitives/core/derive-debug -> primitives/derive-debug; primitives/core/storage -> primitives/storage

* rename crate sp-core-storage -> sp-storage

* rename and move: test/utils/transaction-factory -> client/transaction-factory

* move transaction-factory -> node/transaction-factory

* fix missing rename

* Move chain-spec-builder into bin/utils

* move subkey into bin/utils

* Update new subkey location

* Update docs to reflect new location for utils

* fixing import name

Follows #4317

* Fix `check_polkadot` CI check

* Another try

* Use path overrides

* Last fix

* Update .gitlab-ci.yml

* Fix wasm builder

* Update wasm_project.rs

* Adding script for rename, could be applicable for nodes on top of it, too

* add stderr and gitlab ci features

* apply script

* fix now minor details in expected stderr

* Update the Cargo.lock

* fix name: sc-transaction -> sc-tracing

* fix rename in script, too

* Move transaction pool to primitives

* move backend, errors into primitives

* remove unused client depencies

* Move rpc-api into primitives

* Move peerset back to client

* Move rpc/api back to client, move palette/support/rpc into utils

* move support-rpc into subfolder

* move system-rpc into utils

* move transaction-pool  and -graph back into client

* fix broken imports

* Clean up test primitives

* Make support test utils independent of frame

* remove unnecessary node dependencies from service

* Reactivate dependency script:
 - only enforce the now achieved status quo will remain
 - allow for primitives to depend on /client for now without failing
 - more discriptive error message so people understand, what it wants
 - minor fix to differentiative between ../client and /client (which may be a subfolder)
 - don't allow this to fail anylonger.

* fix doc comment

* 'Should not' rather than 'must not'.

* Revert unwanted dependency changes

* fix faulty import

* fixup derive_more version

* fix wrong import path

* palette -> frame

* PALETTE, Palette -> FRAME

* Move folder pallete -> frame

* Update docs/Structure.adoc

Co-Authored-By: Benjamin Kampmann <[email protected]>

* Update docs/README.adoc

Co-Authored-By: Benjamin Kampmann <[email protected]>

* Update README.adoc

Upgrade from futures-preview to futures 0.3.1, and remove futures 0.1 where currently possible (#4083)

* Migrate node and node-template

* Migrate srml

* Simple changes

* Add async-std for interval

* Fix test-runtime warning

* Small changes

* move futures01 in core/rpc to dev-deps

* Change wasm CI builds

* Switch to async-std 1.0.1

* Remove async-std dep of network

* Add modified lockfile

* Fix node cli browser build

* Remove authority-discovery async-std dep

* Add Send + Sync to interval dyn stream

* fix runtime check script

* fix subkey deploy

* fix rustdoc-header path

* Remove vulnerable rust-yaml by directly parsing clap in subkey

* Make network and password optional as the test requires

* /paint to /palette

* rename paint to palette

* rename the modules in palette to be pallets

* update Structure.adoc

* bump impl

* fix CI directory

* Update docs/Structure.adoc

Co-Authored-By: Benjamin Kampmann <[email protected]>

* Adding first rough ouline of the repository structure

* Remove old CI stuff

* add title

* formatting fixes

* move node-exits job's script to scripts dir

* Move docs into subdir

* move to bin

* move maintainence scripts, configs and helpers into its own dir

* add .local to ignore

* move core->client

* start up 'test' area

* move test client

* move test runtime

* make test move compile

* Add dependencies rule enforcement.

* Fix indexing.

* Update docs to reflect latest changes

* Moving /srml->/paint

* update docs

* move client/sr-* -> primitives/

* clean old readme

* remove old broken code in rhd

* update lock

* Step 1.

* starting to untangle client

* Fix after merge.

* start splitting out client interfaces

* move children and blockchain interfaces

* Move trie and state-machine to primitives.

* Fix WASM builds.

* fixing broken imports

* more interface moves

* move backend and light to interfaces

* move CallExecutor

* move cli off client

* moving around more interfaces

* re-add consensus crates into the mix

* fix subkey path

* relieve client from executor

* starting to pull out client from grandpa

* move is_decendent_of out of client

* grandpa still depends on client directly

* lemme tests pass

* rename srml->paint

* Make it compile.

* rename interfaces->client-api

* Move keyring to primitives.

* fixup libp2p dep

* fix broken use

* allow dependency enforcement to fail

* move fork-tree

* Moving wasm-builder

* make env

* move build-script-utils

* fixup broken crate depdencies and names

* fix imports for authority discovery

* fix typo

* update cargo.lock

* fixing imports

* Fix paths and add missing crates

* re-add missing crates

* Fix a check for polkadot to work on forked repos.

* .gitlab-ci.yml: Add `git diff` for CI debugging

* More future proof and less redundant.

* .gitlab-ci.yml: Improve comments and refactor sed command

* .gitlab-ci.yml: Remove two commas in a row

* .gitlab-ci.yml: Ensure to match branch statements at the end

* .gitlab-ci.yml: Reference concrete commit

When one does not specify the concrete commit, cargo-update tries to
checkout 'master' in the Substrate repository.

* .gitlab-ci.yml: Remove 'git diff' debug line

* Check polkadot compilation.

* Fix git grep.

* Make sure to update properly. Use commit hash instead.