* Require `fn` token in `decl_storage` `get`

The `fn` token was already for quite some time as an optional parameter.
It was introduced to make these functions better findable. This pr makes
the `fn` token required.

* Remove `GetterNoFnkeyword`

* impl try_mutate for storage value and storage double map

* Docs + Reuse `try_mutate` in `mutate`

* Improve tracing implementation

* Enable tracing in runtime interfaces

* Switch to `TRACE` level

* Validate transaction timeouts

* Add timing

* Add tracing to transaction validation.

* Fix docs.

Co-authored-by: NikVolf <[email protected]>

* implement iter for doublemap

* fmt

* fix tests

* fix staking mock

* address comment

* update doc and constraint for reversible hasher

Co-authored-by: Gavin Wood <[email protected]>

* Update to latest staking

* generate tests for benchmarking

* add tests, fix warnings

* starting on democracy

* impl_benchmark_tests

* Way more readable

* add test feature flag (does this work?)

* Fix `successful_origin` impl

* democracry benchmark tests

* Fix example benchmarks, add tests

* identity benchmark tests

* Update im-online benchmark tests

* try to add session benchmarking tests (problem with mock)

* staking and timestamp

* add test for treasury, issue with dynamic contains

* utility

* Vesting

* test instead of check

* hide until we figure out what is wrong

* add docs

* close code

* Create custom mock for session-pallet-benchmarking

* Use refcell pattern

* make un-pub

* test-linux-stable includes `runtime-benchmarks` feature

* Revert "test-linux-stable includes `runtime-benchmarks` feature"

This reverts commit a2dab38abd18ac3eb8a6220e4a00e687740bd38c.

* run tests in `--release`

* undo balance change

* build wasm

* Make Dispatchable return the actual weight consumed

Add PostInfo associated type to Dispatchable and have frame implement
Dispatchable { type PostInfo = PostDispatchInfo } where PostDispatchInfo
contains the actual weight consumed.

* Fix whitespace issues in docs

* Repot a bit of democracy code

* Basic logic is drafted

* Lazy democracy builds.

* Add non-locked split-voting and instant-scheduling.

* Introduce delegation that works.

* Builds again.

* Indentation

* Building.

* Docs and migration

* Fix half of the tests

* Fix up & repot tests

* Fix runtime build

* Update docs

* Docs

* Nits.

* Turnout counts full capital

* Delegations could towards capital

* proxy delegation & proxy unvoting

* Fix

* Tests for split-voting

* Add missing file

* Persistent locking.

* System.BlockHash

* Fix hash

* Introduce K/V iteration in all _concat maps

Also move across:
- System.Account (blake2_128_concat)
- Balances.Locks (twox_64_concat)
- ElectionsPhragmen.VotesOf (twox_64_concat)
- ElectionsPhragmen.StakeOf (twox_64_concat)
- Identity.IdentityOf (twox_64_concat)
- Identity.SubsOf (twox_64_concat)
- Society.Payouts (twox_64_concat)
- Session.NextKeys (twox_64_concat)
- Identity.SuperOf (blake2_128_concat)
- Session.KeyOwner (blake2_128_concat)
- Society.SuspendedCandidates (twox_64_concat)
- Society.SuspendedMembers (twox_64_concat)
- Society.Vouching (twox_64_concat)
- Society.Strikes (twox_64_concat)
- System.EventTopics
- Balances.Account

* Build fixes

* Ensure migration happens in correct order

* Staking.*

* Vesting.* Offences.*

* Democracy.*

* Babe.* Collective.*

* Grandpa.*

* Assets.* Benchmark.* Contracts.* Elections.* Asset.* Nicks.*

Also introduce real account list

* ImOnline.*

* Treasury.*

* Recovery.*

* Final bits.

* Docs

* Fix one test

* Fix test

* All passing except the UI tests

* Remove linked_map part 1

* Remove linked_map

* Some iterator utils for double maps.

* Remove old migrations

* Introduce tombstone for LinkedMap type

* Migration for genesis hash

* Fix build

* Fix hash

* Rename Map is_linked -> unused, keeping backwards compat (#5256)

* Update frame/balances/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update frame/elections/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Remove old migration code.

* Update frame/system/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update bin/node/runtime/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Fix hash

* fix session migration

* Fix watning

Co-authored-by: Jaco Greeff <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Robert Habermeier <[email protected]>

* Assign unique storage names to pallets.

* Bump spec

* Upgrade logic for finality tracker (untested)

* Logic for migrating Identity (untested)

* Logic for migrating transaction-payment

* Fix tests

* Fix `decl_storage` build

* Contract -> Contracts

* Update Cargo.lock

* bump spec

* update migration

* Fix merge error

* Migration for contracts

* Remove serde

* Remove some illegal spaces and Options

* Fix types in identity.

* Minor variable rename

Co-authored-by: Gavin Wood <[email protected]>

* Initially scoping out of the problem

* Remove need for exiry in balance locks.

* Remove expiry from locks.

* Remove supefluous balance test

* Amalgamate pieces of balance module

* Split out vesting

* Fix tests

* Fixes for vesting.

* Docs.

* Weight docs.

* Refactor things in terms of set_balances.

* Switch out ED to be free + reserved.

* Remove on_free_balance_zero and some docs.

* Build fixes

* Update frame/vesting/src/lib.rs

Co-Authored-By: Xiliang Chen <[email protected]>

* Update frame/vesting/src/lib.rs

Co-Authored-By: Xiliang Chen <[email protected]>

* Migration

* Remove superfluous code.

* Test fixes

* Fix some tests

* Fix repatriate reserve

* Fixes

* Add test for migration

* Final cleanups

* Fix

* Indentation.

* Undo unneeded referencing

* Bump runtime version

* Fixes

Co-authored-by: Xiliang Chen <[email protected]>

* remove default hasher from decl_storage!

* fix decl_storage declarations

* modify doublemap syntax

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

Gavin Wood authored Jan 09, 2020 and Shawn Tabrizi committed Jan 09, 2020

* First draft

* Initial work on tests

* Add tests.

* Ensure old members can't tip.

* Fix complexity

* Update node runtime

* Build fix.

* build fix

* Fix tests

* Fix tests

* Refactor Contains impl for tests

* Introduce new way to avoid impl Contains conflicts

* Fixes

* Docs.

* Docs.

* Typo

* Whitespace

* Docs

* Typo

* Formatting

* Update frame/treasury/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update frame/treasury/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update frame/treasury/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Add provisional weights.

Co-authored-by: Shawn Tabrizi <[email protected]>

* Add subaccounts functionality

* More work

* Multisig prototyped with tests

* Add timepoints to prevent replay

* Remove TODO

* Check for the right owner in cancel.

* Test the timepoint stuff

* Batch works with any origin

* Refactor tuples into structs.

* Finalise function docs/complexity and also add proper weights.

* Fix wasm

* Module-level docs

* Fix typo

* Runtime fix

* Better deposit system; more tests.

* Fix typo

* Switch +1 for -1

* Add Blake2_128Concat; fix insecurity; change return policy.

* Fix typo

* Update frame/utility/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update frame/utility/src/lib.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update bin/node/runtime/src/lib.rs

Co-Authored-By: Sergei Pepyakin <[email protected]>

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Sergei Pepyakin <[email protected]>

* Make `decl_error!` errors usable

This pr implements support for returning errors of different pallets in
a pallet. These errors need to be declared with `decl_error!`.

The pr changes the following:

- Each dispatchable function now returns a `DispatchResult` which is an
alias for `Result<(), DispatchError>`.
- `DispatchError` is an enum that has 4 variants:
  - `Other`: For storing string error messages
  - `CannotLookup`: Variant that is returned when something returns a
  `sp_runtime::LookupError`
  - `BadOrigin`: Variant that is returned for any kind of bad origin
  - `Module`: The error of a specific module. Contains the `index`,
  `error` and the `message`. The index is the index of the module in
  `construct_runtime!`. `error` is the index of the error in the error
  enum declared by `decl_error!`. `message` is the message to the error
  variant (this will not be encoded).
- `construct_runtime!` now creates a new struct `ModuleToIndex`. This
struct implements the trait `ModuleToIndex`.
- `frame_system::Trait` has a new associated type: `ModuleToIndex` that
expects the `ModuleToIndex` generated by `construct_runtime!`.
- All error strings returned in any module are being converted now to `DispatchError`.
- `BadOrigin` is the default error returned by any type that implements `EnsureOrigin`.

* Fix frame system benchmarks

* Rename: Phase 1.

* Unify codec.

* Fixing: Phase 2

* Fixing: Phase 3.

* Fixing: Phase 4.

* Fixing: Phase 5.

* Fixing: Phase 6.

* Fixing: Phase 7.

* Fixing: Phase 8. Tests

* Fixing: Phase 9. Tests!!!

* Fixing: Phase 10. Moar tests!

* Finally done!

* More fixes.

* Rename primitives:: to sp_core::

* Apply renames in finality-grandpa.

* Fix benches.

* Fix benches 2.

* Revert node-template.

* Fix frame-system in our modules.

* primitives/sr-io -> primitives/io

* fix

* rename

* runtime-io -> sp-io

* git mv

* fix ci

* remove package name

* fix

* fix

* try minimizing diff

* try minimizing diff again

* try minimizing diff again

* primitives/sr-std -> primitives/std

* fix

* fix conflict

* rstd -> sp-std

* git mv

* fix review

* fix merge

* Introduce storage_next allowing iteration.  (without childtries)

* Implement prefixed storage

* impl cache in client_storage_cache (needs test)

* switch overlay change to btreemap

* Revert "impl cache in client_storage_cache"

This reverts commit c91a4848916eba87184b3dc4722cea81aec9339d.

the storage cache cannot be used this way

* Revert "Implement prefixed storage"

This reverts commit 4931088126a427082d7310ed7e83b8eea966bc20.

* Impl StoragePrefixedMap for all map storages

* remove comment

* Move all overlays to BTreeMap

* btreemap iteration improvment

* impl for child tries

* impl tests for childs

* fix

* remove cache comment

* Fix grumble

* Add `swap` and `decode_len` to `DoubleMap`

*  Add tests to `swap` and `decode_len` for `DoubleMap` (WIP)

* Address review comments

* Remove function that is not in scope

* fix test

* better naming

* Adding script for rename, could be applicable for nodes on top of it, too

* add stderr and gitlab ci features

* apply script

* fix now minor details in expected stderr

* Update the Cargo.lock

* fix name: sc-transaction -> sc-tracing

* fix rename in script, too

* Move transaction pool to primitives

* move backend, errors into primitives

* remove unused client depencies

* Move rpc-api into primitives

* Move peerset back to client

* Move rpc/api back to client, move palette/support/rpc into utils

* move support-rpc into subfolder

* move system-rpc into utils

* move transaction-pool  and -graph back into client

* fix broken imports

* Clean up test primitives

* Make support test utils independent of frame

* remove unnecessary node dependencies from service

* Reactivate dependency script:
 - only enforce the now achieved status quo will remain
 - allow for primitives to depend on /client for now without failing
 - more discriptive error message so people understand, what it wants
 - minor fix to differentiative between ../client and /client (which may be a subfolder)
 - don't allow this to fail anylonger.

* fix doc comment

* 'Should not' rather than 'must not'.

* Revert unwanted dependency changes

* fix faulty import

* fixup derive_more version

* fix wrong import path

* interim

* interim

* interim

* first working section

* cleanup

* finished parsing

* cleanup

* added system module search

* added clone and find_entry

* generic find_module_entry

* interim

* working event

* added generic event with no instance error

* cleanup

* added decl origin

* cleanup

* added all modules

* added outer dispatch

* added modules expansion

* refactored transformations

* updated error message

* added resolve mechanics

* added metadata

* finished config

* finished inherents

* added validate_unsigned

* added compares

* cleanup

* cleanup

* cleanup

* fix

* updated modules for last one wins

* cleanup

* made nested modules

* updated impl version

* removed comment

* cleanup

* added ui tests

* added optional comma

* removed unnecessary to string cast

* removed no compile

* cleanup

* fmt

* returned nocompile

* Update srml/support/procedural/src/construct_runtime/parse.rs

Co-Authored-By: thiolliere <[email protected]>

* added where definition

* updated ui tests

* updated ui test cases

* added test case

* updated tests

* interim

* added parse for module part

* removed totokens

* fixes

* fixed multiple iter

* changed TokenStream

* fmt

* updated trybuild

* added test for arguments

* fmt

* fixes + more tests

* fixes

* fmt

* rolled back runtime

* minor fixes

* empty

* fixes

* fmt

* Update paint/support/procedural/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update paint/support/procedural/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update paint/support/procedural/src/construct_runtime/parse.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* interim

* refactored seen_keys

* refactored hash_set

* Update paint/support/procedural/src/construct_runtime/mod.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* refactored find

* fix

* fixed all_modules

* added double declaration check

* small fix

* fmt

* fix

* fix default

* format

* palette -> frame

* PALETTE, Palette -> FRAME

* Move folder pallete -> frame

* Update docs/Structure.adoc

Co-Authored-By: Benjamin Kampmann <[email protected]>

* Update docs/README.adoc

Co-Authored-By: Benjamin Kampmann <[email protected]>

* Update README.adoc

* Implement instrumentation for performance profiling

* Add profiling to runtime functions declared in `decl_module` inc.
`on_initialize` and `on_finalize`

* Remove changes

* add docs, tidy up

* fix versions

* fix copyright date

* switch to hashmap and instant

* update example

* update example

* implement receiver for logger and make default

* fix comment

* use `if_std!` macro

* remove whitespace

* fix whitespace

* fix nits

* Fisr migrated version of weight to paint

* Bump

* Minor nits

* Some review fixes.

* Line width

* Revert spec bump

* Fix build

* Update lock file

* Update palette/executive/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update palette/membership/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* /paint to /palette

* rename paint to palette

* rename the modules in palette to be pallets

* update Structure.adoc

* bump impl

* fix CI directory

* Update docs/Structure.adoc

Co-Authored-By: Benjamin Kampmann <[email protected]>

* Adding first rough ouline of the repository structure

* Remove old CI stuff

* add title

* formatting fixes

* move node-exits job's script to scripts dir

* Move docs into subdir

* move to bin

* move maintainence scripts, configs and helpers into its own dir

* add .local to ignore

* move core->client

* start up 'test' area

* move test client

* move test runtime

* make test move compile

* Add dependencies rule enforcement.

* Fix indexing.

* Update docs to reflect latest changes

* Moving /srml->/paint

* update docs

* move client/sr-* -> primitives/

* clean old readme

* remove old broken code in rhd

* update lock

* Step 1.

* starting to untangle client

* Fix after merge.

* start splitting out client interfaces

* move children and blockchain interfaces

* Move trie and state-machine to primitives.

* Fix WASM builds.

* fixing broken imports

* more interface moves

* move backend and light to interfaces

* move CallExecutor

* move cli off client

* moving around more interfaces

* re-add consensus crates into the mix

* fix subkey path

* relieve client from executor

* starting to pull out client from grandpa

* move is_decendent_of out of client

* grandpa still depends on client directly

* lemme tests pass

* rename srml->paint

* Make it compile.

* rename interfaces->client-api

* Move keyring to primitives.

* fixup libp2p dep

* fix broken use

* allow dependency enforcement to fail

* move fork-tree

* Moving wasm-builder

* make env

* move build-script-utils

* fixup broken crate depdencies and names

* fix imports for authority discovery

* fix typo

* update cargo.lock

* fixing imports

* Fix paths and add missing crates

* re-add missing crates

* Create typed client helpers for querying chain state storage items declared by
'decl_storage!'.

* Update substrate-rpc-custom functions to use async await syntax.

* The implementation of substrate-rpc-custom was a bit verbose and repetitive.
This commit makes the implementation simpler by intruducing a struct which
represents query for a typed value in storage. The new struct is called
StorageQuery. A StorageQuery wraps a raw StorageKey but is not directy
constructable. To construct a StorageQuery, the user must supply an
implementation of a srml_support::storage::generator trait such as
StorageValue or StorageMap.

A type implementing one of the generator traits can be aquired by:
A) marking a storage item as pub within a call to decl_storage (recommended)
or B) implementing one of the generator types manually.

While option B may sometimes me necessary, it's not recommended because
separate manual implementaions may lose sync with the original definition.

* drop unused dependency

* fmt

* Remove unnecessary pub from Authorities field in test-runtime storage
declaration.

This field was added to support a test in an earlier commit. The
test no longer relies on test-runtime so the change can be reverted.

* Move it to srml as support extension.

* Fix long lines.

* Fix.

* Adds first version of traits for generating the host functions

* First steps of the procedural macro

* Implements generation of the host extern functions

* Prefix ext host function with snake case trait name

* Implement host functions implementation on the host

* Change `HostFunctions` interface

* Implement `HostFunctions` for tuples

* Make `WasmExecutor` generic over the host functions

* Begin to add a test and make it compile

* Make the test succeed

* Add test to ensure that host functions are not found

* It's alive! Make the `set_storage` test work

* Add test for mutable references

* Code cleanup and documentation etc

* Add marker trait for types that should be passed as SCALE encoded

* Inherit the visibility from the trait and more improvements

* More impls and move them into their own file

* Code simplification by dropping one trait

* Give it a better name

* Implement traits for arrays

* Refactor code to support pass by codec/inner

* Docs

* Implement pass by inner for some crypto types and add a test

* Implement exchangeable function support

* Rewrite sr-io with as runtime interface

* Start reworking after master merge

* Adds `PassByCodec` derive

* Adds `PassByInner` derive

* Fix compilation errors

* More implementations

* Implement runtime interface traits for `str`

* Make `sr-io` compile again

* Fix more compilation errors

* More progress on getting stuff back to compile

* More compilation fixes

* Fix warnings

* Remove le conversions

* Add support for `wasm_only` interfaces

* Implement `Allocator` interface

* Improve error message

* Move `WasmAllocator` to `sr-io` and more clean ups

* Use correct function signature for wasm functions

* Store the host functions with the Wasm runtime

* Docs update

* Fix compilation after master merge

* Remove `sr-io/without_std`

* Make `srml-support` tests run again

* More compilation error fixes

* Use correct doc syntax

* Fix test-runtime

* Fix compilation

* Catch native panics when executing the wasm runtime

As with the native runtime, we now catch all native panics when we
execute the wasm runtime. The panics inside the wasm runtime were
already catched before by the wasm executor automatically, but any panic
in the host functions could bring down the node. The recent switch to
execute the native counterpart of the host function in `sr-io`, makes
this change required. The native `sr-io` functions just `panic` when
something is not provided or any other error occured.

* Fix compilation

* Don't panic in a panic

* Move `sr-sandbox` to new runtime interface

* Fixes tests after sandbox changes

* Make sure we detect invalid utf8

* Fixes after master merge

* Adds pass by enum strategy

* Fix wasmtime integration

* Some macro structure clean up

* Rework and test exchangebale host functions

* PassBy derive macros documentation

* Docs for `runtime_interface` macro

* Support wild card argument names

* Adds ui tests

* Make sure that we are backwards compatible to the old runtime interfaces

* Documentation

* Fixes after latest master merge

* Make `wasmtime` happy

* Make `full_crypto` work

* Make the new interface versionable

* Rename `Sanboxing` to `Sandbox`

* Don't finalize in test while importing

* Fix Performance regression

* Fix test

* Retire storage_items

* Add storage_items! tests to srml/support/tests

* Assimilate genesis config

* Implement Printable for tuples.

* Add debugging function.

* Add debug 1.

* Implement  for everything.

* RuntimeDebug derive.

* Introduce RuntimeDebug.

* Add some dummy logging.

* Replace RuntimeDebug with Debug.

* Revert "Replace RuntimeDebug with Debug."

This reverts commit bc47070a8cb30241b2b590b2fa29fd195088162f.

* Working on Debug for all.

* Fix bounds.

* Add debug utils.

* Implement runtime logging.

* Add some docs and clean up.

* Clean up derives.

* Fix custom derive impl.

* Bump runtime.

* Fix long lines.

* Fix doc test.

* Use CARGO_CFG_STD.

* Revert "Use CARGO_CFG_STD."

This reverts commit ea429566de18ed0fa052571b359eb9826a64a9f4.

* Use parse_macro_input

* Update lockfile.

* Apply review suggestions.

* Remove stray re-export.

* Add no-std impl.

* Update lockfile.

* parse decl_storage getters with fn keyword

* test for get in decl_storage

* update all decl_storage! getters

* bump version

* adjust missed doc line

This function executes the given closure in a context where the test
externalities are set. This makes the srml tests easier to write, as the
test externalities need to be created anyway.

* Move `Externalities` into `substrate-externalities`

- `Externalities` now support generic extensions
- Split of `primtives-storage` for storage primitive types

* Move the externalities scoping into `substrate-externalities`

* Fix compilation

* Review feedback

* Adds macro for declaring extensions

* Fix benchmarks

* Introduce `ExtensionStore` trait

* Last review comments

* Implement it for `ExtensionStore`

* impl

* patch

* lock

* some refactor

* some avoided copy

* new api without ref for doublemap

* fix

* version bump

* fix

* point to incoming release

* use codec latest

* bumpd impl version

* fix unused

* fix

* Update srml/support/src/storage/mod.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* refactor

* fix

* remove unused vec storages

* address comment

* Macro-ify perthings.

* Refactor fixed64

* Half-workign phragmen refactor.

* Finalize phragmen refactor.

* Fix creation of perquintill

* Fix build errors

* Line-width

* Fix more build errors.

* Line-width

* Fix offence test

* Resolve all TODOs.

* Apply suggestions from code review

Co-Authored-By: Gavin Wood <[email protected]>
Co-Authored-By: thiolliere <[email protected]>

* Fix most of the review comments.

* Updates to multiply by rational

* Fxi build

* Fix abs issue with Fixed64

* Fix tests and improvements.

* Fix build

* Remove more tests from staking.

* Review comments.

* Add fuzzing stuff.

* Better fuzzing

* Better doc.

* Bump.

* Master.into()

* A bit more hardening.

* Final nits.

* Update lock

* Fix indent.

* Revert lock file.

* Bump.