Svyatoslav Nikolsky authored Dec 13, 2018 and asynchronous rob committed Dec 13, 2018

* implement AuxStore on light clients

* Update core/client/db/src/light.rs

Co-Authored-By: svyatonik <[email protected]>

* Update core/client/db/src/light.rs

Co-Authored-By: svyatonik <[email protected]>

* Update core/client/db/src/light.rs

Co-Authored-By: svyatonik <[email protected]>

* make offline-reporting infrastructure more generic

* add a listener-trait for watching when the timestamp has been set

* prevent inclusion of empty offline reports

* add test for exclusion

* generate aura-offline reports

* ability to slash many times for being offline "multiple" times

* Logic for punishing validators for missing aura steps

* stub tests

* pave way for verification of timestamp vs slot

* alter aura import queue to wait for timestamp

* check timestamp matches seal

* do inherent check properly

* service compiles

* all tests compile

* test srml-aura logic

* aura tests pass

* everything builds

* some more final tweaks to block authorship for aura

* switch to manual delays before step

* restore substrate-consensus-aura to always std and address grumbles

* update some state roots in executor tests

* node-executor tests pass

* get most tests passing

* address grumbles

* Move pool maintainance to a background task.

* Remove commented code.

* Add issue number to TODOs

* Fix warnings and tests.

* light client contextual method call

* fixed off-by-one in test

* newlines + commas

* Make `decl_runtime_apis!` implement `RuntimeApiInfo` for all runtime
apis

* Make the runtime side generate the info constants as well

* Make `RuntimeApiInfo` implementation use the correct generics

* Adds a test for the runtime api info stuff

* Remove duplicated code by using block from `test-client`

* Adds `compile_fail` tests for `api_version`

* Adds documentation for `api_version`

* Make `impl_runtime_apis!` generate `RUNTIME_API_VERSIONS`

* Update documentation and tests for `RUNTIME_API_VERSIONS`

* Implement `has_api` by using the `RuntimeApiInfo`

* Make `impl_runtime_apis` check that trait identifiers are unique

* Prefix all runtime api function with the corresponding trait

So `execute_block` will be called `Core_execute_block`.
This makes it possible to have traits implement a function with the
same name.

* Rebase master

* Update after master rebase

* core: make block justification optional

* runtime: update wasm binaries

* core: optionally pass justification on finalize_block

* finality-grandpa: add channel to trigger authority set changes

this will allow the `BlockImport` to trigger an authority set change when
importing a change block that provides a justification (when syncing)

* finality-grandpa: move finalize_block to free function

* finality-grandpa: add GrandpaOracle for auth set liveness checking

this will be used by `BlockImport` to check whether the authority set for a
given block is still live, if the authority set isn't live then importing a
change block requires a justification.

* finality-grandpa: store justification on finalized transition blocks

* finality-grandpa: check justification on authority set change blocks

* finality-grandpa: poll grandpa liveness oracle every 10 seconds

* finality-grandpa: spawn grandpa oracle in service setup

* core: support multiple subscriptions per consensus gossip topic

* finality-grandpa: create and verify justifications

* finality-grandpa: update to local branch of grandpa

* finality-grandpa: update to finality-grandpa v0.5.0

* finality-grandpa: move grandpa oracle code

* finality-grandpa: fix canonality check

* finality-grandpa: clean up error handling

* finality-grandpa: fix canonical_at_height

* finality-grandpa: fix tests

* runtime: update wasm binaries

* core: add tests for finalizing block with justification

* finality-grandpa: improve validation of justifications

* core: remove unused IncompleteJustification block import error

* core: test multiple subscribers for same consensus gossip topic

* Revert "finality-grandpa: improve validation of justifications"

This reverts commit 51eb2c58.

* finality-grandpa: fix commit validation

* finality-grandpa: fix commit ancestry validation

* finality-grandpa: use grandpa v0.5.1

* finality-grandpa: add docs

* finality-grandpa: fix failing test

* finality-grandpa: only allow a pending authority set change per fork

* finality-grandpa: fix validator set transition test

* make block builder generic over inherent-data

* construct_runtime has you specify inherent data type

* get all tests to compile

* Rewrites `impl_runtime_apis!` macro as `proc-macro`

* Adds some documentation

* Require the `impl_runtime_apis` to use a path for accessing the trait

* Make the runtime implement `GetNodeBlockType`

* Moves first chunk of runtime api code into the `impl_runtime_apis` macro

This also renames `ClientWithApi` into `RuntimeApi`.

* Make `impl_runtime_apis` use `runtime` api version automatically

* `decl_runtime_apis` automatically adds `Block: BlockT` as generic parameter

* Remove function generic arguments in block builder api

* Remove some unnused stuff from the `decl_runtime_apis` macro

* Make `InherentData` working again

* Make `impl_runtime_apis!` implement the `RuntimeApi` side as well

* Make it compile again after rebasing with master

* Split `sr-api-macros` into multiple files

* Reimplement `decl_runtime_apis!` as proc_macro

* Use `decl_runtime_apis!` for `Core` as well and improve error reporting

* Adds documentation for `decl_runtime_apis!` and `impl_runtime_apis!`

* Move some code

* Adds compile fail tests

* Adds a test and fixes some bugs

* Make `impl_runtime_apis!` support `_` as parameter name

* Fixes build errors with wasm

* Wasm rebuild after master rebase

* Apply suggestions from code review

Co-Authored-By: bkchr <[email protected]>

* Addresses some grumbles

* Adds test to ensure that method signatures need to match

* New wasm files

* get compiling with latest version of grandpa

* generalize UntilImported to prepare for waiting for commit message targets

* extract until_imported out to own module

* logic for blocking commits until enough blocks imported

* add tests for commit message blocking logic

* pass through commit mesage round number as well

* extract communication streams to own module

* add Error implementation for ExitOrError

* introduce stream adapter for checking commit messages

* output sink for commits

* implement the unimplemented

* remove extra line

* update to latest version of grandpa api

* update finality-grandpa to 0.4.0

* Use filter_map earlier when checking incoming commits messages

Co-Authored-By: rphmeier <[email protected]>

* address some grumbles

* fix light client startup

* fix grumbles

* Revert "Pruning changes trie without digests (#952)"

This reverts commit 468d4698.

* pruning changes tries without digests

* u64::max_value()

* Update wasm files and fix merge conflict

* Fixes tests

* pruning changes tries without digests

* u64::max_value()

* Make sure to call the runtime trait functions

The runtime trait functions export their functions as extern c
functions. These functions itself call the trait functions. This commit
makes sure that the trait functions are called and not functions with
the same name that are implemented for the runtime.

* Update the wasm files

* Revert "start to refactor block import wrapper a bit"

This reverts commit b919de49e1a82422a0870b66100832d531174771.

* Refactor grandpa stuff to get tests almost compiling

* Fixes after rebase

* Fixes merge compile errors

* build CHT for changes tries roots

* collect chnages tries roots proof in key_changes_proof

* flush check_changes_proof

* fixed compilation

* LightDataChecker now has a ref to the blockchain

* continue passing proofs

* new light db tests

* more CHT tests

* more tests for key changes proof when headers are missing

* lost files

Shawn Tabrizi authored Nov 14, 2018 and asynchronous rob committed Nov 14, 2018

* adding mac libraries and how to start testnet

* Clean up Justification struct, Issue #1033

* adding mac libraries and how to start testnet

* Move `initialise_block` into `Core` trait as it is crucial calling the API functions

* Switch to first version of new runtime API implementation

* Fixes bug in tests

* Reenable asserts

* Directly use the `TestAPI` in the tests

* Start improving the api traits

:100644 100644 898aadc7 49217199 M	Cargo.lock
:100644 100644 61570436 465ed664 M	core/client/src/backend.rs
:100644 100644 5d0c886b 64d710fd M	core/client/src/block_builder.rs
:100644 100644 c447855e 5ecbe474 M	core/client/src/client.rs
:100644 100644 139cef13 f90dbf3d M	core/client/src/error.rs
:100644 100644 2800c503 3298e66a M	core/client/src/runtime_api.rs
:100644 100644 affa1c5c 809b08bc M	core/primitives/src/lib.rs
:100644 100644 2877dfa9 d5547413 M	core/sr-api/Cargo.toml
:100644 100644 9a49784d 6a625a03 M	core/sr-api/src/lib.rs
:100644 100644 7c28e1c7 a1a444a9 M	core/sr-primitives/src/traits.rs
:100644 100644 2e113ab6 dcc01a6d M	srml/metadata/Cargo.toml
:100644 100644 ea722a70 0809531a M	srml/metadata/src/lib.rs

* Refactoring

* Move `sr-api` into client and more refactoring

* Fixes tests

* Some documentation and cleanup

* Fixes compilation after rebase

* More refactoring and more documentation

* Makes `substrate-client` compilable on `wasm`

On `wasm` it basically just exports the runtime api stuff.

* Fixes grumbles

* Updates wasm files after rebasing the master

* Remove TODO comment

* Remove whitespaces

* Fixes after rebasing master

* Another rebase, another fix commit

* forks support in changes trie storage

* moved convert_hash to primitives

* optimize ChangesTrieRootsStorage::root when anchor is canonicalized

docs: Extends #999. Remove disused package readmes. Rename to sr-metadata. Fix Readme Rustdoc names (#1016)

* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core

* docs: Remove empty package Readmes. Remove old docs links. Rename to Readme

* docs: Remove link to old docs

* misc: Merge latest from master. Renaem to sr-metadata. Fix list of RustDoc package names

* Rename substrate-metadata to sr-metadata, since it is in the "srml" subdirectory

* Change example to use package name that opens all packages in RustDocs

* Fix list of packages names that are available to open in RustDocs

* fix typo

* fix ordering of CLI options and add missing backslash

* Remove superfluous serde requirements.

* Try to ensure hash is serde

* Fixups

* Building again

* Attempt to reenable Block (doesn't build)

* Fixes compilation for node cli

* Fixes test compilation

* Fix wasm

* Fix tests

* Remove unneeded changes

* Fix up comments

* Reenable some code

* Compile error when origin misused.

* Remove unnecessary includes of `serde_derive`

* Cleanups

* [core/primitives] Update crunchy 0.1 => 0.2

* [core/primitives] Update fixed-hash 0.2 => 0.3.0-beta

* [core/primitives] Update `uint` 0.4 => 0.5.0-beta

* [core/state-machine] Fix errors emerged by fixed-hash update

* [core/state-db] Fix errors that emerged from fixed-hash update

* [core/sr-io] Fix errors that emerged from the fixed-hash update

* [core/trie] Fix errors emerged from fixed-hash updates

* [core/trie] Make use of new Hash::as_fixed_bytes_mut method in tests

* [core/sr-primitives] Fix errors emerged from updating fixed-hash

* [core/executor] Fix errors that emerged from fixed-hash update

* [core/test-runtime] Fix errors that emerged from updating fixed-hash

* [core/test-runtime] Fix an error that emerged from fixed-hash update

* [core/transaction-pool] Fix errors that emerged from updating fixed-hash

* Add From<u64> impl for hash types defined in core/primitives

* [core/client/db] Fix errors that emerged from fixed-hash update

* [core/{client/network/rpc}]: Fix errors emerged by fixed-hash update

* [node/{cli/executor/runtime}]: Fix errors emerged by updating fixed-hash

* [core/network]: Fix bug in parsing constant str

* Update Cargo.lock

- Add crunchy 0.2.1
- fixed-hash 0.2.2 => 0.3.0-beta.3
- Add static-assertions 0.2.5
- uint 0.4.1 => 0.5.0-beta.1

* [core/primitives]: Add fixed-hash/rustc-hex feature for FromStr impl

* [core/primitives] No longer provide From<u64> impl for hash types if byteorder support is not enabled

* [core/primitives] Revert to using From impl again in primitives tests

* [core/..] Fix some bugs that emerged by recent fixed-hash updates

* Update a bunch of Cargo.lock files

* [core/state-db] Re-add whitespace between attr and extern crate/mod

* [core/primitives] Fix bug in From<u64> impl for hash types using the wrong feature guard

* Support multi trie in genesis generation

* Fix merge issues

* Generalize BlockImport

 - move ImportBlock, BlockOrigin, ImportResult into shared sr-primitives
 - let Consensus provide  and  traits again
 - update consensus traits to latest development
 - implement traits on client::Client, test_client::TestClient
 - update RHD to use the new import_block API

* Move ImportBlock into consensus-common
* Send import notification in aura tests
* Integrating aura into service
* Make Signatures more generic
* Aura Block Production with the given key
* run aura on the thread pool
* start at exact step start in aura
* Add needed wasm blob, in leiu of better solutions.
* Make API ids consistent with traits and bring upstream for sharing.
* Add decrease_free_balance to Balances module
* Encode `Metadata` once instead of two times
* Bitops include xor
* Upgrade key module.
* Default pages to somewhat bigger.
* Introduce upgrade key into node
* Add `Created` event

* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core

* Implement a non generic version of child delta trie

* Use delta_trie_root in state_machine

* Expand InMemory backend to support multi-storage

* Create Consolidate trait

* Fix all crate compile and remove unused OverlayedChanges::drain

* Implement child storage root support and overlay changes

* Add child storage reader

* Add child storage writer

* Implement child storage cleaning

* Fix light backend compile

* Add all required ext functions for wasm executor

* Add ext def to io

* Add all io functions

* Fix nostd compile

* Add simple test

* Remove unnecessary vec copy in child_storage_root_transaction

* Use values_mut/for_each to make it shorter

* Use extend to shorter a for loop

* Move record_all_keys to trie so it's easier to generic them later

* space -> tab

* Remove to_owned in debug format

* Clean out all to_owned

* Break debug_trace to multiple lines

* Remove 0..

* UserError copy/paste typo

* Replace Vec::from_raw_parts by slice::from_raw_parts

* Use iter::empty()

* Wrap some long lines

* Wrap a missing line

* Remove unnecessary map

https://github.com/paritytech/substrate/pull/856#discussion_r226222663

* Call ext_free after from_raw_parts

* Fix tests in other crates

* Adds new `ProvideInherent` trait

Also implements the new trait for `srml/consensus` and `srml/timestamp`.

* Adds `impl_outer_inherent!` macro

* Reexport macros from `alloc`

* Introduce `RuntimeString` and fix `ProvideInherent` on `no_std`

* Replace `VersionString` with `RuntimeString`

* Improvements for `impl_outer_inherent!`

* Make `construct_runtime!` support `impl_outer_inherent!`

* Fixes after rebase

* Whitespace