* {core,srml}/authority-discovery: Move generic to specific session keys

* {srml,core}/authority-discovery: Verify signature outside of runtime

Given that the `core/authority-discovery` uses concrete authority
identifiers and signatures, one can verify a signature with the
authority discovery within `core`. Given the above, the `verify` runtime
api is obsolete and thus removed.

* *: Add authority discovery to the set of session keys

* *: Sign authority discovery DHT payload with keystore instead of runtime

Instead of calling a runtime function to sign a dht payload, which then
invokes the keystore, pass the keystore to the authority discovery
module and use it directly.

* core/authority-discovery: Give libp2p Kademlia time to start up

* core/authority-discovery: Move authorities priority group name to const

* node/runtime/src/lib.rs: Bump runtime spec version

* *: Fix lints and node/testing test failures

* *: Fix formatting

* core/authority-discovery: Box dht event channel in unit tests

* node/cli/src/service.rs: Fix future import

* node/cli/src/service.rs: Replace unwrap by expect with proof

* node/cli/src/chain_spec: Remove TODO for testnet key generation

* core/authority-discovery/src/lib: Remove scale encoding TODOs

* srml/authority-discovery: Make comment a doc comment

* core/authority-discovery: Remove unused StreamExt import

* node/runtime: Bump impl version to debug CI

* Test ci.

* Change the line width to 100.

* Revert "Change the line width to 100."

This reverts commit edff1f85.

* Fix a check for polkadot to work on forked repos.

* Revert "node/runtime: Bump impl version to debug CI"

This reverts commit 1a90903b.

* Revert "Test ci."

This reverts commit a2c9df57.

* Cargo.lock: Fix wrong lock file merge

* srml/authority-discovery: Keep track of new validator set not upcoming

* core/authority-discovery: Document key retrieval functions

* Wasm diagnostics

* Pass the error

* Make errno optional

* Cargo.lock

* Log the error

* randomized tests

* destroy clones

* remove commented

* fixes

* add finalization fork

* Initial version that works with proper tests.

* get rid of todos and grumbles and such.

* Cleanup and fix line-width

* fix test runtime test

This lets us drop a couple old dependencies, which is nice.

* update tiny-keccak

* fix

* Update Cargo.toml

* update Cargo.lock

* remove keccak-hasher

* Refactor sr-api to not depend on client anymore

* Fix benches

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <[email protected]>

* Apply suggestions from code review

* Create typed client helpers for querying chain state storage items declared by
'decl_storage!'.

* Update substrate-rpc-custom functions to use async await syntax.

* The implementation of substrate-rpc-custom was a bit verbose and repetitive.
This commit makes the implementation simpler by intruducing a struct which
represents query for a typed value in storage. The new struct is called
StorageQuery. A StorageQuery wraps a raw StorageKey but is not directy
constructable. To construct a StorageQuery, the user must supply an
implementation of a srml_support::storage::generator trait such as
StorageValue or StorageMap.

A type implementing one of the generator traits can be aquired by:
A) marking a storage item as pub within a call to decl_storage (recommended)
or B) implementing one of the generator types manually.

While option B may sometimes me necessary, it's not recommended because
separate manual implementaions may lose sync with the original definition.

* drop unused dependency

* fmt

* Remove unnecessary pub from Authorities field in test-runtime storage
declaration.

This field was added to support a test in an earlier commit. The
test no longer relies on test-runtime so the change can be reverted.

* Move it to srml as support extension.

* Fix long lines.

* Fix.

* Adds first version of traits for generating the host functions

* First steps of the procedural macro

* Implements generation of the host extern functions

* Prefix ext host function with snake case trait name

* Implement host functions implementation on the host

* Change `HostFunctions` interface

* Implement `HostFunctions` for tuples

* Make `WasmExecutor` generic over the host functions

* Begin to add a test and make it compile

* Make the test succeed

* Add test to ensure that host functions are not found

* It's alive! Make the `set_storage` test work

* Add test for mutable references

* Code cleanup and documentation etc

* Add marker trait for types that should be passed as SCALE encoded

* Inherit the visibility from the trait and more improvements

* More impls and move them into their own file

* Code simplification by dropping one trait

* Give it a better name

* Implement traits for arrays

* Refactor code to support pass by codec/inner

* Docs

* Implement pass by inner for some crypto types and add a test

* Implement exchangeable function support

* Rewrite sr-io with as runtime interface

* Start reworking after master merge

* Adds `PassByCodec` derive

* Adds `PassByInner` derive

* Fix compilation errors

* More implementations

* Implement runtime interface traits for `str`

* Make `sr-io` compile again

* Fix more compilation errors

* More progress on getting stuff back to compile

* More compilation fixes

* Fix warnings

* Remove le conversions

* Add support for `wasm_only` interfaces

* Implement `Allocator` interface

* Improve error message

* Move `WasmAllocator` to `sr-io` and more clean ups

* Use correct function signature for wasm functions

* Store the host functions with the Wasm runtime

* Docs update

* Fix compilation after master merge

* Remove `sr-io/without_std`

* Make `srml-support` tests run again

* More compilation error fixes

* Use correct doc syntax

* Fix test-runtime

* Fix compilation

* Catch native panics when executing the wasm runtime

As with the native runtime, we now catch all native panics when we
execute the wasm runtime. The panics inside the wasm runtime were
already catched before by the wasm executor automatically, but any panic
in the host functions could bring down the node. The recent switch to
execute the native counterpart of the host function in `sr-io`, makes
this change required. The native `sr-io` functions just `panic` when
something is not provided or any other error occured.

* Fix compilation

* Don't panic in a panic

* Move `sr-sandbox` to new runtime interface

* Fixes tests after sandbox changes

* Make sure we detect invalid utf8

* Fixes after master merge

* Adds pass by enum strategy

* Fix wasmtime integration

* Some macro structure clean up

* Rework and test exchangebale host functions

* PassBy derive macros documentation

* Docs for `runtime_interface` macro

* Support wild card argument names

* Adds ui tests

* Make sure that we are backwards compatible to the old runtime interfaces

* Documentation

* Fixes after latest master merge

* Make `wasmtime` happy

* Make `full_crypto` work

* Make the new interface versionable

* Rename `Sanboxing` to `Sandbox`

* Don't finalize in test while importing

* Fix Performance regression

* Fix test

* Remove dependency on hyper-tls

This removes one of the dependencies on native-tls, and thus on OpenSSL.
I will remove the other in a separate commit.

* Remove the `HttpClient` enum

It only had one variant.

* Apply suggestions from code review

Co-Authored-By: Pierre Krieger <[email protected]>

* clean node/cli/Cargo.toml

* minor fix

* clean node/runtime/Cargo.toml

* Apply breaking changes of new libp2p versions

* Oops, forgot to update version

* Fix tests

* Fix imports with WASM

* Fix WASM for real

* Update core/network/src/debug_info.rs

Co-Authored-By: Roman Borschel <[email protected]>

* Fix compilation

- Add new crate `substrate-build-script-utils` to unify the code of
`node`, `node-template` and `polkadot-node`.
- The `node-cli` build script needs to search upwards for the
`.git/HEAD` file to find it.

* Bump rpc deps

* Update core/test-runtime/src/genesismap.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* introduced "with_crypto" feature and applied switches like in substrate-api-client fork

* introduced "with_crypto" feature and applied switches like in substraTEE-worker fork

* distinguishing core::hash vs std::hash

* @bkchr's review requests fulfilled

* fixes

* revert dependency upgrade ed25519-dalek

* added full_crypto features to all crates using app_crypto! macro

* fixing CI complaints.

* fix again

* adding CI test for with_crypto feature

* added full_crypto for ecdsa. now builds wit h--no-deafault-features --features with_crypto

* remove --release from CI test

* @bkchr

 requested changes. moved full_crypto CI test to build stage

* fixing no_std issue

* CI fresh copy from srml-staking

* gitlab CI with +nightly

* solved no-feature-in-macro dilemma

* cosmetics

* Update core/application-crypto/src/sr25519.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update core/application-crypto/src/ed25519.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* even more simple

* undo line delete

* refactoring app_crypto macro. splitting functionalities based on full_crypto feature

* whitespace cosmetics

* srml-evm: init the basic structures

* srml-evm: finish executor implementation

* srml-evm: implement balance deposit and withdraw

* srml-evm: implement the actuall call/create

* srml-evm: use crates.io version of evm

* srml-evm: fix no-std compile

* Remove dependency patch

* Update to evm 0.14

* Use double map for account storage

* Add precompiles support

* Add some basic docs

* Use runtime_io::chain_id()

* Update srml/evm/src/lib.rs

Co-Authored-By: Xiliang Chen <[email protected]>

* Update srml/evm/src/lib.rs

Co-Authored-By: Xiliang Chen <[email protected]>

* Fix WithdrawReason

* Unique saturate balance to u128

* Unique saturate withdraw to u128

* Remove extern crate alloc

* Move account code to a separate storage and use ref for convert_account_id

* More match cause for error message

* Fix potential interger overflow

* Use decode_len for fetching code length

* Implement color output for wasm builder

* Fix `Cargo.lock`

* Add serde to phragmen

* Update lock file

* and bump a version

* and bump a version again

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>

* revert impl-tarit for tuple update

* revert session.

* Revert "revert session."

This reverts commit 98086c9d.

* Revert "revert impl-tarit for tuple update"

This reverts commit 28a7fdde.

* executor: Use non wasmi-specific execution in tests.

* executor: Move all runtime execution tests into tests file.

* executor: Use test_case macro to easily execute tests with different
Wasm execution methods.

* executor: Convert errors to strings with Display, not Debug.

* node-executor: Rewrite benchmarks with criterion.

They were not passing compilation before and criterion seems to be more
widely used in Substrate.

* executor: Begin implementation of Wasm runtime.

The implementation demonstrates the outline of the execution, but does
not link against the external host functions.

* executor: Define and implement basic FunctionExecutor.

The SandboxCapabilities::invoke is still left unimplemented.

* executor: Implement host function trampoline generation.

* executor: Instantiate and link runtime module to env module.

* executor: Provide input data during wasmtime execution.

* executor: Implement SandboxCapabilites::invoke for wasmtime executor.

* executor: Integrate and test wasmtime execution method.

* executor: Improve FunctionExecution error messages.

* Scope the unsafe blocks to be smaller.

* Rename TrampolineState to EnvState.

* Let EnvState own its own compiler instead of unsafe lifetime cast.

* Refactor out some common wasmi/wasmtime logic.

* Typos and cosmetic changes.

* More trampoline comments.

* Cargo.lock update.

* cli: CLI option for running Substrate with compiled Wasm execution.

* executor: Switch dependency from fork to official wasmtime repo.

* Quiet down cranelift logs.

* Explicitly catch panics during host calls.

We do this to ensure that panics do not cross language boundaries.

* Additional checks and clarifications in make_trampoline.

* Fixes after merge from master and panic safety for wasmtime
instantiation.

Weiliang Li authored Nov 01, 2019 and Pierre Krieger committed Nov 01, 2019

* authority-discovery: futures 03 Future

* make ci happy

* use futures timer instead of tokio timer

* Update core/authority-discovery/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* remove tokio 01 runtime

* trigger build

* kill futures01

* rename futures

This reverts commit c3b1a983.

* Extract CLI to separate module in node/cli

* Make node/cli compile for WASM

* More work on node/cli browser

* More work on browser node

* More work

* More work

* Purge a bit the CI script

* More clean up

* Remove substrate-finality-grandpa from the CI

Its tests use tokio, which fails to compile.

* Address review

* Add rocksdb feature to the service

* Fix substrate-service WASM CI

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>

* Don't WASM-compile substrate-service altogether

* grandpa: Write Grandpa authorities to well known key.

Instead of requiring execution proofs for Grandpa authorities, this
enables much simpler storage proofs.

* grandpa: Introduce named AuthorityList type.

* grandpa: Storage migration for srml-grandpa module.

* Remove no-longer-used GrandpaApi runtime API.

* grandpa: Write AuthorityList to storage with encoding version.

We expect the AuthorityList type may change (eg. key changes). To make
upgrades smoother, include a version in the stored value.

* Bump node runtime spec version.

* Update srml/grandpa/src/lib.rs

Co-Authored-By: André Silva <[email protected]>

* im-online: account for block authorship

* im-online: add test for block authorship onlineness

* im-online: cleanup

* im-online: fix test

Passing around Vec<Vec<u8>> everywhere is gross and confusing and
breaks encapsulation.

* Put the DB configuration in an enum

* Allow passing a custom database to client-db

* Clean-ups in client-db

* Fix client tests

* Fix service tests

* Hopefully fix tests for good this time 😩

* Address review

* Change DefaultMaxDepth from 1024 to 32

* Switch sr-arithmetic benchmarking to criterion

* Update core/sr-arithmetic/Cargo.toml

Co-Authored-By: Bastian Köcher <[email protected]>

* Update core/sr-arithmetic/benches/bench.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Test on variable limb lengths

* Change license

* Rework division

* Remove footgun around session keys/handlers

- `OpaqueKeys` now has an associated type `KeyTypeIdProviders`. This can
be used in the runtime as input for `SessionHandler` from the session
trait.
- `impl_opaque_keys` now works with modules and extracts the `KeyTypeId`
from the module directly.
- Added some checks to the `session` storage initialization that checks
that the `SessionHandler` and `Keys` use the same number of keys and
that the order is equal.

* Update core/sr-primitives/src/traits.rs

* Working bench for 50 sequental

* configured benches

* fix warnings

* Optimize and fix issues

* add preamble

* Fix benchmarks.

* fix compilation

* remove unneeded features for now

* initial version for testing

* New version that compiles

* optional at block parameter

* Fix some more view grumbles.

* Update srml/transaction-payment/src/lib.rs

The authority discovery module enables authorities to be discoverable
and discover other authorities to improve interconnection among them. In
order to achieve this the module needs to know when the authority set
changes, thus when a session changes.

One has to register a module as a *session handler* in order for it to
be notified of changing sessions. The order and number of these *session
handlers* **MUST** correspond to the order and number of the *session
keys*.

Commit 7fc21cea added the authority discovery to the `SessionHandlers`.
Given that the authority discovery module piggybacks on the Babe session
keys the commit violated the above constraint.

This commit reverts most of 7fc21cea, leaving `core/authority-discovery`
and `srml/authority-discovery` untouched.

* `decl_storage!` check for duplicate `config()`/`get()`

* Fix tests

This reverts commit a57c72be.

* Transaction pool watch intristics.

* Track extrinsic rpc methods.

* Test for pool watcher.

* Track extrinsic rpc test.

* Fix rpc naming.

* review fixes

* Update jsonrpc and use une subcription.

* Naming and dependencies.

* fix node-template
Use MultiSignature to maintain compatibility with substrate-node
Reset version to 1
Remove unused const

* fix chain_spec

* line width

* Add Nicknames module for accounts.

* Integrate into node.

* Fix build

* Update srml/nicks/src/lib.rs

Co-Authored-By: Kian Paimani <[email protected]>

* Bump runtime

* Improve weight docs

* Docs.

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>