* Create a macro which automates creation of benchmark test suites.

* bump impl_version

* allow unused on test_bench_by_name

* use proper doctest ignore attribute

* Explicitly hand the Module to the test suite

Much better practice than depending on it showing up implicitly in
the namespace.

* explicitly import what we need into `mod tests`

* bench_module is `ident` not `tt`

Co-authored-by: Guillaume Thiolliere <[email protected]>

* allow end users to specify arguments for new_test_ext

This turned out to be surprisingly easy. On reflection, it turns out
that of course the compiler can't eagerly evaluate the function call,
but needs to paste it in everywhere desired.

* enable explicitly specifying the path to the benchmarks invocation

also enable optional trailing commas

* Revert "bump impl_version"

This reverts commit 0209e4de33fd43873f8cfc6875815d0fd6151e63.

* list failing benchmark tests and the errors which caused the failure

* harden benchmark tests against internal panics

* suppress warning about ignored profiles

unfortunately, setting the profile here doesn't do anything; we'd
need to set it in every leaf package anyway. However, as this was
just making the default explicit anyway, I think it's safe enough
to remove entirely.

* impl_benchmark_test_suite for assets

* impl_benchmark_test_suite for balances

* impl_benchmark_test_suite for bounties

* impl_benchmark_test_suite for Collective

* impl_benchmark_test_suite for Contracts

* impl_benchmark_test_suite for Democracy

* don't impl_benchmark_test_suite for Elections-Phragmen

* impl_benchmark_test_suite for Identity

Note that Identity tests currently fail. They failed in an identical
way before this change, so as far as I'm concerned, the status quo is
good enough for now.

* impl_benchmark_test_suite for ImOnline

* impl_benchmark_test_suite for indices

For this crate also, the test suite fails identically with and without
this change, so we can say that this change is not the cause of the
tests' failure to compile.

* impl_benchmark_test_suite for lottery

* impl_benchmark_test_suite for merkle-mountain-range

* impl_benchmark_test_suite for Multisig

These tests fail identically with and without the change, so the change
seems unlikely to be the origin of the failures.

* impl_benchmark_test_suite for offences

* impl_benchmark_test_suite for Proxy

Fails identically with and without this change.

* impl_benchmark_test_suite for scheduler

* impl_benchmark_test_suite for session

It turns out to be important to be able to exclude items marked
`#[extra]` sometimes. Who knew?

* impl_benchmark_test_suite for staking

* impl_benchmark_test_suite for system

* impl_benchmark_test_suite for timestamp

* impl_benchmark_test_suite for tips

* impl_benchmark_test_suite for treasury

* impl_benchmark_test_suite for utility

Note that benchmark tests fail identically before and after this change.

* impl_benchmark_test_suite for vesting

* fix wrong module name in impl_benchmark_test_suite in Offences

* address line length nits

* enable optional keyword argument: exec_name

Took a _lot_ of macro-wrangling to get the functionality that I want,
but now you have the option to pass in

```rust
impl_benchmark_test_suite!(
	Elections,
	crate::tests::ExtBuilder::default().desired_members(13).desired_runners_up(7),
	crate::tests::Test,
	exec_name = build_and_execute,
);
```

and have it expand out properly. A selected fragment of the expansion:

```rust
        fn test_benchmarks() {
            crate::tests::ExtBuilder::default()
                .desired_members(13)
                .desired_runners_up(7)
                .build_and_execute(|| {
```

* get rid of dead code

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Initial migration of balances pallet

* Fix some errors

* Remove unused imports

* Formatting and removing some todos

* Delete Subtrait

* Add genesis builder impls for tests

* Fix GenesisConfig impl

* Make set_balance visible to tests, rename RawEvent to Event

* Fix tests with Event rename etc.

* More test RawEvent renames

* Even more RawEvent renames

* Rename module to pallet in comments

* Add PalletInfo impl to avid storage collision, fixes tests

* Apply review suggestion: remove trailing a

Co-authored-by: David <[email protected]>

* BalancesEvent alias

* Remove BalancesEvent alias

* Review suggestion: remove redundant comment

* Apply review suggestion: make vis super

* Fis doc links

* Add RawEvent alias

* Add missing Instance parameter to deprecated RawEvent alias

* Fix RawEvent deprecation warnings

Co-authored-by: David <[email protected]>

* contracts: Remove ConfigCache

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Fixup test

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Migrate bounties tests to use construct_runtime

* Migrate contracts tests to use construct_runtime

* Migrate democracy tests to use construct_runtime

* review: rename TreasuryEvent -> TreasuryError

* contracts: Implement refcounting for wasm code

* contracts: Charge rent for code storage

* contracts: Fix dispatchables erroneously refunding base costs

* Fixed typos in comments.

Co-authored-by: Andrew Jones <[email protected]>

* Remove awkward empty line

* Fix more typos in docs

* Fix typos in docs

Co-authored-by: Andrew Jones <[email protected]>

* Split up complicated expression

Co-authored-by: Andrew Jones <[email protected]>

* review: Remove unused return value

* Fix typos

Co-authored-by: Andrew Jones <[email protected]>

* review: Fix refcount being reset to one on re-instrumentation

* Document evictable_code parameter

* Make Executable::execute consume and store itself

* Added comments about stale values

* Disregard struct size in occupied_storage()

Co-authored-by: Andrew Jones <[email protected]>

* contracts: Document seal_input

* contracts: Improve `ReturnCode` docs.

* contracts: Improve seal_restore_to docs

* review: Improved wording

* contracts: Remove redundant bool argument from the eviction event

* contracts: Improve event documentation

* contracts: Emit event on contract termination

* PRINT_PALLET_UPGRADE=1 cargo check -p frame-system

* Copy attributes, imports, mods and type defs

* Copy Config trait

* Annotate constants

* Tabify

* Migrate hooks

* Upgrade template rename interface to hooks

* Migrate pallet call

* Migrate Event

* Migrate Error

* Migrate Origin

* Remove optional validate_unsigned

* Remove remaining TODO_MAYBE_WHERE_CLAUSE

* Overwrite original lib.rs with migrated lib2.rs.

* Add required Event IsType constraint

* Add disable supertrait check

* Fix leftover Trait trait

* Add missing pallet prefix for weight attributes

* Add missing Error type parameter

* Add missing Hooks type parameter

* Private call visibility, restore original helper types and helpers etc

* Fix hooks type parameter

* Rename RawEvent to Event

* Add missing storage type annotations

* Remove unused imports

* Add GenesisConfig helpers for compat

* Fix unused import warnings

* Update frame/support/procedural/src/storage/print_pallet_upgrade.rs

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Fix test errors and warnings

* Fix remaining errors and warnings

* Apply review suggestion: fix formatting

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: annotate BlockLength as constant

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: add triling comma

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: add triling comma

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: add trailing comma

Co-authored-by: Guillaume Thiolliere <[email protected]>

* Apply review suggestion: fix storage type indentation

* Apply review suggestion: remove redundant Origin type alias

* Add missing codec derives for BlockLength

* Restore module docs

* Module -> Pallet renamel

* Revert "Update frame/support/procedural/src/storage/print_pallet_upgrade.rs"

This reverts commit d2a2d5b6

* Apply review suggestion: merge crate imports

Co-authored-by: Alexander Theißen <[email protected]>

* Revert "Upgrade template rename interface to hooks"

This reverts commit 306f0239

* Single line import

* Refactor generated genesis build

* Import sp_io::storage

* Revert previous, fully qualify sp_io::storage

* Fix ui tests

* Fix errors after merge, missing changes

* Set UpgradedToDualRefCount to true in genesis build

* Annotated Runtime version with constant, exposing it via metadata

* Add metadata attribute

Co-authored-by: Guillaume Thiolliere <[email protected]>
Co-authored-by: Alexander Theißen <[email protected]>

* Add rent_payed field to the contract info

* Don't pay out more as reward as was spent in rent

* Make successful evictions free

* Add tests to check that surcharge reward is capped by rent payed

* review: Fixed docs

* Add `len` function that can return the length of a storage item efficiently

* Make use of the new len function in contracts

* Fix benchmarks

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* Remove unused imports

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Pay first rent during instantiation

* Fix and add new tests

* Do not increment trie id counter on failure

* Rework rent parameters

* No need for empty_pair_count any longer

* Parameterize runtime

* commented use of common

* hack to pass tests

* another hack

* remove all commented code

* fix the easy tests

* temp hack

* follow through comma hack until better solution

* patch macro

* missed one

* update benchmarks

* update docs

* fix docs

* removed too much

* fix changes

Co-authored-by: Shawn Tabrizi <[email protected]>

* Prevent contracts from allocating a too large buffer

* Fix possible integer overflow

* Do not evict a contract from within a call stack

We don't want to trigger contract eviction automatically when
a contract is called. This is because those changes can be
reverted due to how storage transactions are used at the moment.
More Information:
https://github.com/paritytech/substrate/issues/6439#issuecomment-648754324

It can be re-introduced once the linked issue is resolved. In the meantime
`claim_surcharge` must be called to evict a contract.

* Lazily delete storage in on_initialize instead of when removing the contract

* Add missing documentation of new error

* Make Module::claim_surcharge public

It being the only dispatchable that is private is an oversight.

* review: Add final newline

* review: Simplify assert statement

* Add test that checks that partial remove of a contract works

* Premote warning to error

* Added missing docs for seal_terminate

* Lazy deletion should only take AVERAGE_ON_INITIALIZE_RATIO of the block

* Added informational about the lazy deletion throughput

* Avoid lazy deletion in case the block is already full

* Prevent queue decoding in case of an already full block

* Add test that checks that on_initialize honors block limits

* Make host functions return TrapReason

This avoids the need to manually store any trap reasons
to the `Runtime` from the host function. This adds the following
benefits:

* It properly composes with the upcoming chain extensions
* Missing to set a trap value is now a compile error

* Add chain extension

The chain extension is a way for the contract author to add new
host functions for contracts to call.

* Add tests for chain extensions

* Fix regression in set_rent.wat fixture

Not all offsets where properly updated when changing the fixtures
for the new salt on instantiate.

* Pre-charge a weight amount based off the specified length

* Improve fn write docs

* Renamed state to phantom

* Fix typo

* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL

* Add SS58Prefix type to the frame_system config trait

* Remove unused chain_id runtime interface

* Make host functions return TrapReason

This avoids the need to manually store any trap reasons
to the `Runtime` from the host function. This adds the following
benefits:

* It properly composes with the upcoming chain extensions
* Missing to set a trap value is now a compile error

* review: Remove superflous .into()

* Make it possible to calculate the storage as often as you want

So, until now each Substrate based blockchain has calculated the storage
root once, at the end of the block. Now there is Frontier that wants to
calculate some intermediate storage root. However this failed on block
import. The problem with that was the extrinsics root. When building the
block we stored `Default::default()` as extrinsics root, because yeah,
we don't know the extrinsics root before finishing the block. At the end
this extrinsics root was then calculated. But on block import we passed
the already known extrinsics root. This was no problem, as we removed
this value at the end of the block. However when you all the storage
root in between, that changes the storage root between block building
and block import.

This pr changes this behavior. It removes the `ExtrinsicsRoot` storage
entry and also doesn't pass it anymore to `System::initialize`. By doing
it, we remove the difference in the storage and fix the storage root mismatch.

* Fix bug with incorrectly calculating the extrinscs root

* Review feedback

This is no longer necessary with storage transactions.

* fix docs

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Alexander Theißen <[email protected]>

Co-authored-by: Alexander Theißen <[email protected]>

* Allow Backend::for_keys_in_child_storage to be aborted by the closure

* Ext::kill_child_storage now takes an upper limit for backend deletion

* Add Storage::storage_kill_limited() runtime interface

* review: Use a new version of kill_storage instead of a new interface

* review: Simplify boolean expression

Co-authored-by: cheme <[email protected]>

* review: Rename for_keys_in_child_storage

Co-authored-by: cheme <[email protected]>

* Basic weights builder.

* Fixing WiP

* Make the tests work.

* Fix weights in node/runtime.

* WiP.

* Update pallets with new weights parameters.

* Validate returns a Result now.

* Count mandatory weight separately.

* DRY

* BREAKING: Updating state root, because of the left-over weight-tracking stuff

* Update tests affected by Mandatory tracking.

* Fixing tests.

* Fix defaults for simple_max

* Update frame/system/src/weights.rs

Co-authored-by: Kian Paimani <[email protected]>

* Rework the API a bit.

* Fix compilation & tests.

* Apply suggestions from code review

Co-authored-by: Kian Paimani <[email protected]>

* Add extra docs & rename few things.

* Fix whitespace in ASCII art.

* Update frame/system/src/limits.rs

Co-authored-by: Kian Paimani <[email protected]>

* Fix max_extrinsic calculations.

* Fix conflicts.

* Fix compilation.

* Fix new code.

* re-remove generic asset

* Fix usage.

* Update state root.

* Update proxy.

* Fix tests.

* Move weights validity to integrity_test

* Remove redundant BlockWeights.

* Add all/non_mandatory comment

* Add test.

* Remove fn block_weights

* Make the macro prettier.

* Fix some docs.

* Make max_total behave more predictabily.

* Add BlockWeights to metadata.

* fix balances test

* Fix utility test.

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Benjamin Kampmann <[email protected]>
Co-authored-by: thiolliere <[email protected]>

* rename Trait to Config

* add test asserting using Trait is still valid.

* fix ui tests

* pallet-contracts: Fix seal_restore_to to output proper module errors

Those errors where part of the decl_error for some time but where
never actually returned. This allows proper debugging of failed
restorations. Previously, any error did return the misleading
`ContractTrapped`.

* Bind UncheckedFrom<T::Hash> + AsRef<[u8]> everywhere

This allows us to make assumptions about the AccoutId
that are necessary for testing and in order to benchmark
the module properly.

This also groups free standing functions into inherent functions
in order to minimize the places where the new bounds need to
be specified.

* Rework contract address determination

* Do not allow override by runtime author
* Instantiate gained a new parameter "salt"

This change is done now in expecation of the upcoming code rent
which needs to change the instantiation dispatchable and
host function anyways.

The situation in where we have only something that is like CREATE2
makes it impossible for UIs to help the user to create an arbitrary
amount of instantiations from the same code.

With this change we have the same functionality as ethereum with
a CREATE and CREATE2 instantation semantic.

* Remove TrieIdGenerator

The new trait bounds allows us to remove this workaround
from the configuration trait.

* Remove default parameters for config trait

It should be solely the responsiblity to determine proper values for
these parameter. As a matter of fact most runtime weren't using these
values anyways.

* Fix tests for new account id type

Because of the new bounds on the trait tests can't get away by using
u64 as accound id. Replacing the 8 byte value by a 32 byte value
creates out quite a bit of code churn.

* Fix benchmarks

The benchmarks need adaption to the new instantiate semantics.

* Fix compile errors caused by adding new trait bounds
* Fix compile errors caused by renaming storage and rent functions
* Adapt host functions and dispatchables to the new salt
* Add tests for instantiate host functions (was not possible before)

* Add benchmark results

* Adapt to the new WeightInfo

The new benchmarks add a new parameter for salt "s" to the instantiate weights
that needs to be applied.

* Fix deploying_wasm_contract_should_work integration test

This test is adapted to use the new instantiate signature.

* Break overlong line

* Break more long lines

Co-authored-by: Parity Benchmarking Bot <[email protected]>

* Thread-local parameter_types for testing.

* Better docs.

* Some minors

* Merge'em

* Update frame/support/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Align more to basti's trick

* Update frame/support/src/lib.rs

* Update frame/support/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* pallet_contracts: Inline benchmark helper that is only used once

* Move all max_* Schedule items into a new struct

* Limit the number of globals a module can declare

* The current limits are too high for wasmi to even execute

* Limit the amount of parameters any wasm function is allowed to have

* Limit the size the BrTable's immediate value

* Add instruction benchmarks

* Add new benchmarks to the schedule and make use of it

* Add Benchmark Results generated by the bench bot

* Add proc macro that implements `Debug` for `Schedule`

* Add missing imports necessary for no_std build

* Make the WeightDebug macro available for no_std

In this case a dummy implementation is derived in order to not
blow up the code size akin to the RuntimeDebug macro.

* Rework instr_memory_grow benchmark to use only the maximum amount of pages allowed

* Add maximum amount of memory when benching (seal_)call/instantiate

* cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_contracts

* Added utility benchmark that allows pretty printing of the real schedule

* review: Add missing header to the proc-macro lib.rs

* review: Clarify why #[allow(dead_code)] attribute is there

* review: Fix pwasm-utils line

* review: Fixup rand usage

* review: Fix typo

* review: Imported -> Exported

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* contracts: Adapt to new weight structure

* contracts: Fixup runtime WeightInfo

* contracts: Remove unneeded fullpath of WeightInfo type

* Apply suggestions from code review

Co-authored-by: Andrew Jones <[email protected]>

* Fix typo in schedule.rs

Co-authored-by: Andrew Jones <[email protected]>

* Fix docs in schedule.rs

* Apply suggestions from code review

Co-authored-by: Nikolay Volf <[email protected]>

* Don't publish proc-macro crate until 3.0.0 is ready

* Optimize imports for less repetition

* Break overlong line

Co-authored-by: Parity Benchmarking Bot <[email protected]>
Co-authored-by: Andrew Jones <[email protected]>
Co-authored-by: Nikolay Volf <[email protected]>

* contracts: Make use of existing type aliases for runtime API types

* contracts: Refactor the contracts call runtime API

* review: Fix comment typo

Co-authored-by: Andrew Jones <[email protected]>

* Update frame/contracts/common/src/lib.rs

Co-authored-by: Nikolay Volf <[email protected]>

* Update frame/contracts/common/src/lib.rs

Co-authored-by: Nikolay Volf <[email protected]>

* Update frame/contracts/common/src/lib.rs

Co-authored-by: Nikolay Volf <[email protected]>

* Update frame/contracts/common/src/lib.rs

Co-authored-by: Nikolay Volf <[email protected]>

* Update frame/contracts/common/src/lib.rs

Co-authored-by: Nikolay Volf <[email protected]>

* Update lib.rs

* review: Group crate imports

Co-authored-by: Andrew Jones <[email protected]>
Co-authored-by: Addie Wagenknecht <[email protected]>
Co-authored-by: Nikolay Volf <[email protected]>

Signed-off-by: koushiro <[email protected]>

* Try to get it compiling

* One more

* Make stable happy

* Make stable even more happy ;)

* Update shell.nix

* seal: Add capability to put uninstrumented code (for benchmarks)

Benchmarks should only measure the overhead of the API calls itself.
For that reason we want to run them without instrumentation.

* seal: Cap the the data length for deposited events

Data used in events has storage implications for archive nodes.
Those need to keep the events in storage forever. For that reason
we want to limit the amount of storage that can be used inside events.

* seal: Fix error reporting in the case out of bound sandbox access

* seal: Refactor existing benchmarks

* seal: Convert benchmark file to tabs

* seal: Add benchmarks for functions called by contracts

* seal: Create a default schedule from benchmark generated WeightInfo

* seal: Make use of WeightInfo in extrinsic weight annotations

* seal: Replace the old schedule by the benchmark generated one

* Review: Fix copy paste typo in schedule construction

* Review: Fix stale docs

* Fix whitespace errors

Co-authored-by: Sergei Shulepov <[email protected]>

* Review: Use checked_div in order to be more defensive

* Review: Rename no_charge to already_charged

* Review: Whitelist caller of extrinsics

* Review: Remove trailing whitespace

* Review: Remove confusing "self::" syntax

* Review: Add docs for the benchmark prepration submodule

* Review: Move code generation functions to own module

* Review: Refactor and document benchmark helper functions

* Remove additional empty line

* Added missing comment on caller_funding

* Update frame/contracts/src/benchmarking/code.rs

Co-authored-by: Sergei Shulepov <[email protected]>

* Fix missing sp_std::prelude import in code.rs

* cargo run --release --features runtime-benchmarks --manifest-path bin/node/cli/Cargo.toml -- benchmark --chain dev --steps 50 --repeat 20 --extrinsic * --execution=wasm --wasm-execution=compiled --output ./bin/node/runtime/src/weights --header ./HEADER --pallet pallet_contracts --heap-pages 4096

* Use weights from the benchmark machine for the substrate node

* Remove prefixes from Schedule members

* Data lengths in the WeightInfo Trait are specified in kilobytes

* Rename ApiWeights to HostFunctionWeights

Co-authored-by: Sergei Shulepov <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Rename `ModuleToIndex` to `PalletRuntimeSetup`

Besides the renaming it also adds support getting the name of a pallet
as configured in the runtime.

* Rename it to `PalletInfo`

* Remove accidentally added files

* WeightInfo for Vesting Pallet

* clean up weight docs

* Update lib.rs

* try to pipe max locks

* Update for new type

* add warning when locks > MaxLocks

* Update lib.rs

* fix compile

* remove aliasing, fix trait def

* Update