Recently we added the wasm binaries to the `rerun-if-changed` list. The problem with that is that
they have a later mtime than the `invoked.timestamp` file and this file's mtime is used to determine
if the `build.rs` script needs to be re-run. The solution to this is that we copy the mtime of this
`invoked.timestamp` file and add it to the wasm binaries. Then cargo/rustc doesn't constantly wants
to rerun the `build.rs` script.

* basic lint

* lint ordering

* Apply some clippy hints

* Revert clippy ci changes

* Update client/cli/src/commands/generate.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/cli/src/commands/inspect_key.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/transactions.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/network/src/protocol.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Revert due to missing `or_default` function.

* Fix compilation and simplify code

* Undo change that corrupts benchmark.

* fix clippy

* Update client/service/test/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/state-db/src/noncanonical.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update client/state-db/src/noncanonical.rs

remove leftovers!

* Update client/tracing/src/logging/directives.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update utils/fork-tree/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* added needed ref

* Update frame/referenda/src/benchmarking.rs

* Simplify byte-vec creation

* let's just not overlap the ranges

* Correction

* cargo fmt

* Update utils/frame/benchmarking-cli/src/shared/stats.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Giles Cope <[email protected]>

Bumps [cargo_metadata](https://github.com/oli-obk/cargo_metadata) from 0.14.1 to 0.14.2.
- [Release notes](https://github.com/oli-obk/cargo_metadata/releases)
- [Commits](https://github.com/oli-obk/cargo_metadata/compare/0.14.1...0.14.2

)

---
updated-dependencies:
- dependency-name: cargo_metadata
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* wasm-builder: Support latest nightly

With latest nightly, aka rust version 1.60+ namespaced features are added. This changes the handling
of optional dependencies. We currently have features that enable optional dependencies when `std` is
enabled. This was before no problem, but now the wasm-builder detects them as enabled. To support
the transition period until 1.60 is released as stable, this pr adds an heuristic to not enable these
optional crates in the wasm build when they are enabled in the `std` feature. This heuristic fails
when someones enables these optional dependencies from the outside as well as via the `std` feature,
however we hope that no one is doing this at the moment. When namespaced features are enabled, these
dependencies needs to be enabled using `dep:dependency-name` to solve this properly.

https://doc.rust-lang.org/cargo/reference/unstable.html#namespaced-features

* Remove accidentally added features

* Add production profile to wasm builder

* Fix profile detection

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Replace panic! by println! + exit

* Default to `release` for wasm on debug builds

* Replaced unwrap by expect

* Update all weights

Rerun on the bm2 server.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Parity Bot <[email protected]>

This is required for projects like Cumulus that have dependencies on `westend-runtime`, but this
runtime is only added when the `runtime-benchmarks` feature is enabled. By having all features
"enabled" in `cargo metadata` we ensure that all crates can be found.

When building a wasm binary from a different repo inside a local workspace, we did not used the
correct `Cargo.toml` to find the correct patches and features. The solution to this is to just walk
up from the target directory until we find the workspace we are currently compiling. If this
heuristic isn't working, we print a warning and let the user set an env variable
`WASM_BUILD_WORKSPACE_HINT` to tell the `wasm-builder` where the actual workspace is.

* Set current dir at check_wasm_toolchain_installed

* Add comments

Bumps [cargo_metadata](https://github.com/oli-obk/cargo_metadata) from 0.13.1 to 0.14.1.
- [Release notes](https://github.com/oli-obk/cargo_metadata/releases)
- [Commits](https://github.com/oli-obk/cargo_metadata/compare/v0.13.1...0.14.1

)

---
updated-dependencies:
- dependency-name: cargo_metadata
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Intend to reactivate cargo-unleash check

It appears the bug it was deactivated for has been resolved a while ago. Trying to reactivate the checks.

* adding missing cargo.toml metadata for BEEFY crates

* fix wrong version reference

* matching up versions

* disable faulty cache

* switching more versions to prerelease

* Revert "disable faulty cache"

This reverts commit 411a12ae444a9695a8bfea4458a868438d870b06.

* bump minor of sc-allocator to fix already-published-issue

* fixup another pre-released dependency problem

* temp switch to latest unleash

* fixing dependency version and features

* prometheus endpoint has also been changed

* fixing proposer metrics versioning

* fixing hex feature for beefy

* fix generate-bags feature selection

* fixup Cargo.lock

* upgrade prometheus dependencies

* missed one

* switch to latest release

* add TypeInfo for DispatchTime

* upgrade wasm-builder to Rust 2021

* remove resolver

* revert resolver in virtual workspace

* Switch to Rust 2021

* Update trybuild to fix errors

* all reverences of substrate.dev replaced with *correct* docs.substrate.io links

* fix links in non-cargo files, errors in replace

* homepage https://substrate.io



* cargo +nightly fmt

* FMT

Co-authored-by: Bastian Köcher <[email protected]>

* Do not propagate host RUSTFLAGS when checking for WASM toolchain

* Update utils/wasm-builder/src/prerequisites.rs

* Update utils/wasm-builder/src/prerequisites.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Prepare UI tests for rust 1.54

* Delete wrong_page.stderr

* CI: run with a staging CI image

* Revert "CI: run with a staging CI image"

This reverts commit 66f5b00d14b50fd9d8fbf773f7e884f380697591.

* CI: debug, again

* LOG_TARGET is only used in std

* Remove unnecessary unsafe

* Fixes

* Use correct rustc locally

* FMT

* Compile with benchmarking

* Review feedback

* Some ui tests

* I know...

* Fix wasm tests

Co-authored-by: Denis P <[email protected]>
Co-authored-by: thiolliere <[email protected]>

Remove unneeded dependencies and dev-dependencies.
Made self_destruct test not dependent on wasm bin size.  
Updated code related to deprecated warning on tracing-subscriber `scope()` 
( See https://github.com/tokio-rs/tracing/issues/1429 )

* reformat everything again

* manual formatting

* last manual fix

* Fix build

Bumps [walkdir](https://github.com/BurntSushi/walkdir) from 2.3.1 to 2.3.2.
- [Release notes](https://github.com/BurntSushi/walkdir/releases)
- [Commits](https://github.com/BurntSushi/walkdir/compare/2.3.1...2.3.2

)

---
updated-dependencies:
- dependency-name: walkdir
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove extra commas made redundent after rustfmt

* Run cargo fmt on the whole code base

* Second run

* Add CI check

* Fix compilation

* More unnecessary braces

* Handle weights

* Use --all

* Use correct attributes...

* Fix UI tests

* AHHHHHHHHH

* 🤦

* Docs

* Fix compilation

* 🤷

* Please stop

* 🤦

 x 2

* More

* make rustfmt.toml consistent with polkadot

Co-authored-by: André Silva <[email protected]>

* mark template and utils as non-publish

* switch to development version for testing

* activate unleash check

* maybe if I disable all rules...

* Fix isolated compilation of `max-encoded-len-derive` with `syn`

error[E0369]: binary operation `==` cannot be applied to type `syn::Path`
  --> src/lib.rs:88:29
   |
88 |             .filter(|attr| attr.path == parse_quote!(max_encoded_len_crate))
   |                            --------- ^^ ----------------------------------- _
   |                            |
   |                            syn::Path

error: aborting due to previous error

For more information about this error, try `rustc --explain E0369`.
Error: could not compile `max-encoded-len-derive`

* WIP: bump changes crates since v3 tag to next breaking

cargo unleash version bump-breaking --changed-since v3.0.0
cargo unleash version set-pre dev --changed-since v3.0.0

FIXME: Don't modify crates that are not yet released, e.g.
`max-encoded-len-derive`

* Update lockfile

* WIP: Bump sp-transaction-pool as well

* WIP: Bump sp-offchain as well

* WIP: Bump frame-system-rpc-runtime-api as well

* WIP: Bump sp-authority-discovery as well

* Manually deactivate dev-deps before `cargo unleash check`

Otherwise we run into `Cycle detected` error.

* Bump sp-consensus-slots

* Add missing Cargo.lock change

* Bump sp-consensus-vrf as well

* Bump sp-keyring as well

* Bump sp-consensus-pow as well

* Try to speed up the `unleash-check` job

Previously, the job took 106 minutes - let's see if explicitly
specifying a `CARGO_TARGET_DIR` will help

* fixup: Ensure the temp target dir exists for unleash check

* Bump pallet-transaction-payment-rpc-runtime-api as well

Needed for Polkadot

* Bump pallet-transaction-payment-rpc as well

Needed for Polkadot

* Try updating crates after patching in the Polkadot CI job

* Use another approach to update patched Substrate crates

* Try to update all sp-core versions in Polkadot CI job

* Simplify sp-core version checking

* Apply another shellcheck lint

* Just do the simplest thing I guess

* Welp don't do --offline then

* Clean up `unleash-check` job triggers

Co-authored-by: Denis Pisarev <[email protected]>

* Fix a note in unleash-check cache step

* Add a note about temporary optimization in cargo-unleash

* Pin a newer version of cargo-unleash

Co-authored-by: Igor Matuszewski <[email protected]>
Co-authored-by: Denis Pisarev <[email protected]>

* validation extension in sp_io

* need paths

* arc impl

* missing host function in executor

* io to pkdot

* decode function.

* encode primitive.

* trailing tab

* multiple patch

* fix child trie logic

* restore master versionning

* bench compact proof size

* trie-db 22.3 is needed

* line width

* split line

* fixes for bench (additional root may not be needed as original issue was
with empty proof).

* revert compact from block size calculation.

* New error type for compression.

* Adding test (incomplete (failing)).
Also lacking real proof checking (no good primitives in sp-trie crate).

* There is currently no proof recording utility in sp_trie, removing
test.

* small test of child root in proof without a child proof.

* remove empty test.

* remove non compact proof size

* Missing revert.

* proof method to encode decode.

* Upgrade `cargo_metadata` to 0.13.1

* Update utils/wasm-builder/src/wasm_project.rs

Co-authored-by: Bastian Köcher <[email protected]>

* begin maybe-compressed-blob

* fix build

* implement blob compression / decompression

* add some tests

* decode -> decompress

* decompress code if compressed

* make API of compresseed blob crate take limit as parameter

* use new API in sc-executro

* wasm-builder: compress wasm

* fix typo

* simplify

* address review

* fix wasm_project.rs

* Update primitives/maybe-compressed-blob/Cargo.toml

Co-authored-by: Andronik Ordian <[email protected]>

Co-authored-by: Andronik Ordian <[email protected]>

* Init `RuntimeLogger` automatically for each runtime api call

This pr change the runtime api in such a way to always and automatically
enable the `RuntimeLogger`. This enables the user to use `log` or
`tracing` from inside the runtime to create log messages. As logging
introduces some extra code and especially increases the size of the wasm
blob. It is advised to disable all logging completely with
`sp-api/disable-logging` when doing the wasm builds for the on-chain
wasm runtime.

Besides these changes, the pr also brings most of the logging found in
frame to the same format "runtime::*".

* Update frame/im-online/src/lib.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Update test-utils/runtime/Cargo.toml

* Fix test

* Don't use tracing in the runtime, as we don't support it :D

* Fixes

Co-authored-by: Guillaume Thiolliere <[email protected]>

* bumping version for next release
* add changelog
* add guide

* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL

* Build every wasm crate in its own project with wasm-builder

Building all wasm crates in one workspace was a nice idea, however it
just introduced problems:

1. We needed to prune old members, but this didn't worked for old git
deps.
2. We locked the whole wasm workspace while building one crate. This
could lead to infinitely locking the workspace on a crash.

Now we just build every crate in its own project, this means we will
build the dependencies multiple times. While building the dependencies
multiple times, we still decrease the build time by around 30 seconds
for Polkadot and Substrate because of the new parallelism ;)

* Remove the requirement on wasm-builder-runner

This removes the requirement on wasm-builder-runner by using the new
`build_dep` feature of cargo. We use nightly anyway and that enables us
to use this feature. This solves the problem of not mixing
build/proc-macro deps with normal deps. By doing this we get rid off
this complicated project structure and can depend directly on
`wasm-builder`. This also removes all the code from wasm-builder-runner
and mentions that it is deprecated.

* Copy the `Cargo.lock` to the correct folder

* Remove wasm-builder-runner

* Update docs

* Fix deterministic check

Modified-by: Bastian Köcher <[email protected]>

* Try to make the ui test happy

* Switch to `SKIP_WASM_BUILD`

* Rename `SKIP_WASM_BINARY` to the correct name...

* Update utils/wasm-builder/src/builder.rs

Co-authored-by: André Silva <[email protected]>

* Update utils/wasm-builder/src/builder.rs

Co-authored-by: André Silva <[email protected]>

Co-authored-by: André Silva <[email protected]>

* Make `wasm-builder` print the rustc version

This makes `wasm-builder` print the rustc version that is being used to
compile the project. This is rather useful, because people can check
faster if the used rustc version is maybe known for being broken with
Substrate.

* Apply suggestions from code review

* Add some comments

* Fixes bug in wasm-builder with cargo publish

There was a bug in wasm-builder which resulted in generating a
`Cargo.lock` in the project directory because of running `cargo
metadata`. This resulted in commands like `cargo publish` to fail (if
there was no `Cargo.lock` before building), because it checks that the
project directory isn't modified.

* Update utils/wasm-builder/src/wasm_project.rs

Co-authored-by: André Silva <[email protected]>

Co-authored-by: André Silva <[email protected]>

This improves the error message of wasm-builder when the wasm toolchain
isn't installed. Currently we print that the wasm toolchain is not
installed, but the actual problem is that there is a bug in the
packaging in rust. This will now be much easier to debug, by printing
the full error message of the compiler.

Because apparently I can not speak properly ;)

* Ignore gc for debug build.

* alternate implementation

* Update utils/wasm-builder/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* fix

Co-authored-by: Bastian Köcher <[email protected]>

* Better handling of stable-only build

* Fix node template build

* Fix wasm builder node-template version mismatch

* Fix load_spec error

* Add , in parameter

* Add descrptive panic messages in tests

* Add descriptive tests in node/executor benches

* Fix missing compact_code_unwrap

* Add missing wasm_binary_unwrap function for executor integration test

* Only define import_sp_io in no_std

* Small Cargo.toml styling fix

* Bump wasm-builder to 2.0.0

* Fix all `with_wasm_builder_from_crates` version in Substrate

* Use `with_wasm_builder_from_crates` for node-template

Co-authored-by: Gavin Wood <[email protected]>