* docs: Remove safe-mix since is no longer local dependency

* docs: Fix existing packages link. Still only includes Substrate Core

* Split out Consensus
* Supply ImportQueue through network-service
  - simplify ImportQueue.import_blocks
  - remove Deadlock on import_block
  - Adding Verifier-Trait
  - Implement import_queue provisioning in service; allow cli to import
* Allow to actually customize import queue
* Consensus Gossip: Cache Message hash per Topic

* Adds a script for automatically update all copyrights

* Ran the `update-copyright` script

* Add missing `As` imports.

* Adds new API traits that will be used by the client and runtime

* Switch consensus to new API's

* Switches transaction-pool to new API's

* Move runtime api stuff into its own crate

* Adds `impl_apis!` macro for implementing the new API traits

* Make `metadata` return directly a blob

* Runtime replace `impl_stubs!` with `impl_apis!`

* Switches to none feature based approach for declaring the different API traits

* Fixes compilation error

* Fixes errors

* Make the `decl_apis!` trait usable from the outside

* Make the `test-client` use the new API traits

* Remove last `impl_stubs!` bits and move some of them into wasm executor for tests

* A little bit more documentation

* Removes `rlp` dependency

* Enable warnings about unused crates in the CI

* Remove all the unused crates

* Make tests working again

snd authored Sep 26, 2018 and asynchronous rob committed Sep 26, 2018

* add stub for Client.best_chain_containing_block_hash

* add fn blockchain::Backend::leaf_hashes

* fix typo

* sketch out Client.best_chain_containing_block_hash

* fix indent

* Blockchain.leaf_hashes -> Blockchain.leaves

* add unimplemented! stub impls for Blockchain.leaves

* start impl of Blockchain.leaves for in-memory client db

* Client.best_chain_containing...: check canonical first and make compile

* first rough attempt at maintaining leaf list in in-memory db

* fix tab indent

* add test best_chain_containing_single_block

* add failing test best_chain_containing_with_fork

* pub use client::blockchain; in test-client to prevent circular dep in client tests

* best_chain_containing_with_single_block: improve and test leaves

* far improve in-memory Backend::leaves impl

* test blockchain::Backend::leaves more thoroughly

* handle more edge cases in blockchain::Backend::leaves impl for in memory

* fix test best_chain_containing_with_fork (two distinct test blocks had same hash)

* make best_chain_containing_block_hash pass existing tests

* improve docstring for Blockchain::leaves

* Client.best_chain_containing: some cleanup. support max_block_number

* best_chain_containing: remove broken outcommented fast check for best = canonical

* remove blank line

* best_block_containing: return None if target_hash not found

* best_chain_containing: add unreachable! at end of function

* improve tests for best_chain_containing

* renames

* more elaborate test scenario for best_containing

* best_containing: fix restriction of search through maybe_max_number

* best_containing: tests for restriction of search

* get rid of unnecessary clones

* replace Client::new_in_mem by new_with_backend which is useful for testing backends

* add test_client::new_with_backend for testing different backend impls

* add test for in_mem::Backend::leaves

* remove unused imports

* in_mem test_leaves: simplify

* flesh out tests for in_mem leaves impl

* remove tests for leaves from client which are now covered in implementing module

* improve comment

* add Client.new_in_mem again

* unwrap in test_client::new_with_backend

* make test_client::BlockBuilderExt work not just with in-mem backend

* make test client ext not just work with in mem backend

* add failing Backend.leaves test for client-db

* update Cargo.lock

* replace KeccakHasher with Blake2Hasher

* refactor

address grumble https://github.com/paritytech/substrate/pull/740#discussion_r217822862

* refactor using NumberFor

address grumble https://github.com/paritytech/substrate/pull/740#discussion_r217823341

* add test that exposes possible problem

* update docstring for Client.best_containing

* extract test for Backend.leaves for reuse

* improve test blockchain_header_and_hash_return_blocks_from_canonical_chain_given_block_numbers

* extract test_blockchain_query_by_number_gets_canonical to easily test multiple impls

* remove whitespace

* remove todo

* Client.best_containing: pre-empt search loop when target in canonical

* best_containing: prevent race condition by holding import lock

* add todo

* extract leaf list update code into function

* add comment

* client-db: use in-memory-kvdb for tests

* use BTreeSet to store leaves for in-mem which is faster and simpler

* add docstring

* add comments and fix formatting

* add initial raw version of LeafSet

* remove Client::update_leaves which has been superceded by LeafSet

* use LeafSet in in-mem backend

* keccak -> blake2

* don't reexport codec traits in primitives

addresses https://github.com/paritytech/substrate/pull/740#discussion_r219538185

* fix rebase mistake

* improve LeafSet and use it in state-db

* correct Transfer nonces to fix ApplyExtinsicFailed(Stale)

* use given backend in canoncal test

* kill dead tree-route code in util

* fix warnings

* tests for leafset

* reorganizations in in_mem backend

* fix reorganization canon block logic

* DB commit and safe reversion on write error

* fix style nits

Gav Wood authored Sep 25, 2018 and Arkadiy Paronyan committed Sep 25, 2018

* Bump codec

* Fix tests

* Patricia trie builds

* Introduce trie

* Some yak shaving.

* Some fixes

* Remove RLP ref

* Fixes

* It builds!

* Some tests fixed

* Another test fix

* Rejig more hashes

* substrate-trie::iterator_works test

* Update lock

* Polish

* Docs

* Undo incorrect "fix" for tests

* Fix nits

asynchronous rob authored Sep 24, 2018 and Arkadiy Paronyan committed Sep 24, 2018

* finalization for in_mem

* fetch last finalized block

* pruning: use canonical term instead of final

* finalize blocks in full node

* begin to port light client DB

* add tree-route

* keep number index consistent in full nodes

* fix tests

* disable cache and finish porting light client

* add AsMut to system module

* final leaf is always best

* fix all tests

* Fix comment and trace

* removed unused Into call

* add comment on behavior of `finalize_block`

* move `tree_route` to client common

* tree_route tests

* return slices in TreeRoute

* apply finality up to parent

* add `finalize_block` call

* adjust formatting

* finality notifications and add last finalized block to chain info

* exhaustive match and comments

* fix sync tests by using non-instant finality

* Move module metadata from json string to custom metadata

* Revisit the metadata structures

1. Move the structures into the metadata crate.
2. Switch to using Cow/MaybeOwnedArray to support Encode/Decode

* Adapt to new metadata structures

* Convert event json metadata to new metadata structures

* Convert storage json metadata to new metadata structures

* Convert runtime metadata from json to new metadata structs

* Implements new metadata structures in client and runtime

* Fixes errors on `no_std`

* Fixes errors after rebasing master

* Do not use `Cow` anymore in metadata

Also replace `String` with our own type definition `StringBuf`.
This fixes compilation on `no_std`.

* Wrap `RuntimeMetadata` in `RuntimeMetadataVersioned` to support versioning

* Move metadata into `srml` and make core unaware of the implementation

* changes_trie

* changs_trie: continue

* changes_trie: adding tests

* fixed TODO

* removed obsolete ExtrinsicChanges

* encodable ChangesTrieConfiguration

* removed polkadot fle

* fixed grumbles

* ext_storage_changes_root returns u32

* moved changes trie root to digest

* removed commented code

* read storage values from native code

* fixed grumbles

* fixed grumbles

* missing comma

Gav Wood authored Sep 12, 2018 and Arkadiy Paronyan committed Sep 12, 2018

* Remove unneeded script

* Rename Substrate Demo -> Substrate

* Rename demo -> node

* Build wasm from last rename.

* Merge ed25519 into substrate-primitives

* Minor tweak

* Rename substrate -> core

* Move substrate-runtime-support to core/runtime/support

* Rename/move substrate-runtime-version

* Move codec up a level

* Rename substrate-codec -> parity-codec

* Move environmental up a level

* Move pwasm-* up to top, ready for removal

* Remove requirement of s-r-support from s-r-primitives

* Move core/runtime/primitives into core/runtime-primitives

* Remove s-r-support dep from s-r-version

* Remove dep of s-r-support from bft

* Remove dep of s-r-support from node/consensus

* Sever all other core deps from s-r-support

* Forgot the no_std directive

* Rename non-SRML modules to sr-* to avoid match clashes

* Move runtime/* to srml/*

* Rename substrate-runtime-* -> srml-*

* Move srml to top-level

* storage proofs

* CHT

Gav Wood authored Aug 15, 2018 and asynchronous rob committed Aug 15, 2018

* Genric over hasher

* WIP start adding NodeCodec

* Add codec to TrieBackend

* Typechecks

* Fix error type

* Cleanup

* Tests build (and fail)

* Fix tests: don't use MemoryDB::default()

* Lockfile

* Address grumbles

* Teach environmental! about generics

* Add Finder artifacts

* whitespace

* Add a toy impl of Hasher and plug it in to Externalities

* Use `uint` and `fixed-hash` from `parity-common`
Remove unused U512
Add test to ensure H256 impls heapsizeof

* lock file updates

* Make hashes Encodable/Decodable

* lock file updates

* Impl FromIterator for TestExternalities so we can collect() and use map!

* Use rustc-hex from crates
Use rlp from master so dependencies do not mess up the scope

* Fix tests in runtime-io

* lockfile shenanigans

* Add a BlakeHasher impl

* Use BlakeHasher in runtime-io

* lockfile updates

* ws

* Add a Blake2/RLP-flavoured NodeCodec

* Use Blake-flavoured Hasher and NodeCodec

* lockfile

* Implement PartialEq and Default for TestExternalities

* Add note about limitations of environmental!

* Make it compile, but this is probably broken

* Derive Debug so tests in executor can work

* Make executor use BlakeHasher

* ws

* WIP make client generic

* typechecks

* cleanup

* client tests pass

* Fix client/db

* cleanup

* Fix network

* Fix rpc

* Fix service

* Make TestExternalities work better au lieu d'un HashMap

* Fix tests in council

* Fix tests in contract

* Fix tests in council

* Fix democracy

* Add comment about odd-looking reexports in tests

* Don't need to load branch

* Fix staking

* Fix session

* Some polkadot fixes and lockfile

* Fix executive

* fixup lockfile

* Fix polkadot/api

* Fix polkadot/service

* Fix polkadot/runtime tests

* Fix tests in test-runtime

* Test fixes

* Fix missing component in the `std` feature

* Use PhantomData and Result from core

* Fix paths
Use core

* load heapsize on wasm

* implement `HeapSizeOf` for wasm

* Add toy impl of `blake2_256` for no_std

* lockfile

* Use kvdb* from parity-common and fix errors

* rebuilt lockfile

* Add dummy impl of `on_advance_round` for rhododendron::Context

* Fix build after merge

* Add HeapSizeOf bound where needed

* Sort out dependencies for no_std

* Add HeapSizeOf bound where needed

* use temp branch pending PR merges

* Remove unneeded tests

* Lock file and wasm artifacts

* lockfile

* Use magic commit for libp2p

* Cleanup

* Implement blake2_256 for no_std

* Back on parity-common master

* missing type params

* Update Cargo.lock

* whitespace

* Rename concrete Rlp node codec "RlpCodec" and use everywhere
Implement a Keccak-flavoured Rlp NodeCodec and use everywhere
Add a KeccakHasher

* Switch to use KeccakHasher

* Lock file and runtimes

* fixup lockfile

* Fix outstanding issue using concrete types (thanks @gnunicorn)

* Cleanup

* More cleanup

* Comment out Blake2 Hasher

* implement ext_keccak256

* Address todo: FetchChecker is generic

* all tests passing

Tomasz Drwięga authored Aug 01, 2018 and Svyatoslav Nikolsky committed Aug 01, 2018

* Initial implementation of storage events.

* Attaching storage events.

* Expose storage modification stream over RPC.

* Use FNV for hashing small keys.

* Fix and add tests.

* Swap alias and RPC name.

* Fix demo.

* Addressing review grumbles.

* Fix comment.

* Substrate service

* Splitting polkadot service

* Specialised components

* Specialised components

* Docs and style

* Docs and style

* Final touches

* Added db key assertion

* More timely block import notifications

* Grumbles.

* More wrapping

* Fix build

* Fixes

* Block export and import

* Export and import using std streams

* Made AuthorituId::from_slice private

* Runtime version

* Updated genesis.wasm

* Minor fixes

* Fresh runtime

* Default version for pre Poc-2; Fixed authorship interface check

* Fixed authoring check

* use db in light clients

* fixed comment

* fixed grumbles

* Merge remote-tracking branch 'origin/master' into gav-xts-dont-panic

* Update wasm.

* consensus, session and staking all panic-safe.

* Democracy doesn't panic in apply.

* Fix tests.

* Extra helper macro, council depanicked.

* Fix one test.

* Fix up all council tests. No panics!

* Council voting depanicked.

* Dispatch returns result.

* session & staking tests updated

* Fix democracy tests.

* Fix council tests.

* Fix up polkadot parachains in runtime

* Fix borked merge

* More Slicable support

Support general `Option` and array types.

* Basic storage types.

* Existential deposit for contract creation

* Basic implemnetation along with removals

* Fix tests.

* externalities builder fix.

* Tests.

* Fix up the runtime.

* Fix tests.

* Add generic `Address` type.

* Initial function integration of Address into Extrinsic.

* Fix build

* All tests compile.

* Fix (some) tests.

* Fix signing.

* Push error.

* transfer can accept Address

* Make Address generic over AccountIndex

* Fix test

* Make Council use Address for dispatch.

* Fix build

* Bend over backwards to support braindead derive.

* Repot some files.

* Fix tests.

* Fix grumbles

* Remove Default bound

* Fix build for new nightly.

* Make `apply_extrinsic` never panic, return useful Result.

* More merge hell

* Doesn't build, but might do soon

* Serde woes

* get substrate-runtime-staking compiling

* Polkadot builds again!

* Fix all build.

* Fix tests & binaries.

* Reserve some extra initial byte values of address for future format changes

* Make semantic of `ReservedBalance` clear.

* Fix panic handler.

* Integrate other balance transformations into the new model

Fix up staking tests.

* Fix runtime tests.

* Fix panic build.

* Tests for demonstrating interaction between balance types.

* Repot some runtime code

* Fix checkedblock in non-std builds

* Get rid of `DoLookup` phantom.

* Attempt to make transaction_pool work with lookups.

* Remove vscode settings

* New attempt at making transaction pool work.

* It builds again!

* --all builds

* Fix tests.

* New build.

* Test account nonce reset.

* polkadot transaction pool tests/framework.

* Initial draft (working).

* Address grumbles.

* Revert bad `map_or`

* Rebuild binaries, workaround.

* Avoid checking in vscode

* reconnecting, shared, slog

* CLI options for name and telemetry url

* ensure telemetry url imples enabled

* Avoid casting to usize early.

* Provide on-connect event for session message

* Better port

* heartbeat and some renaming

* transaction pool stuff

* minor renaming.

* report telemetry

* cleanups.

* Fix for previous cleanup

* dump genesis, dev mode, renaming

* Rework chain spec/config &c. to allow for genesis file loading.

* Avoid producing genesis storage when unneeded

* Allow reading JSON genesis state dumps

* tests work again

* better logging.

* Fix wasm build.

* Introduce PoC-1 spec

* Made block message compatible with poc-1

* Squashed changes for dumping genesis block.

* Binaries.

* Made block message compatible with poc-1

* Remove dead code.

* Fix bad merge.

* Argument passing and returning values when invoking sandboxed funcs (#189)

* Fixed block download sequence (#223)

* Trie-based execution proof (#177)

* TrieBasedBackend

* trie tests

* redunant return_value removed

* use Trie::get_with to record trie proofs

* Relaying tx/blocks by light clients (#190)

* do not import external transactions into light tx pool

* do not announce blocks on light clients

* blocks_are_not_announced_by_light_nodes

Gav Wood authored Jun 06, 2018 and asynchronous rob committed Jun 06, 2018

* Some initial work on RPC and client

* Rephrase as params

* More work on traitifying substrate.

* Traitify in_mem.rs

* traitify client.rs

* Make new primitives (mainly traits) build again.

* Many (superficial) build fixes throughout.

* Fix remaining build issues up to bft interface.

* Make bft primitives be generic.

* Switch out MisBehaviorReport for generic version.

* Merge Hashing into Header.

* Update runtime for new generics (with Hashing).

* Update demo runtime.

* Make runtime compile.

* Build fixes for runtime

* Remove old modules.

* port substrate-bft to use generic substrate types

* port client

* port substrate-test-runtime

* mostly port test-runtime to get compiling for std

* Ensure `AccountId` has a `Default`.

* Fix type deps.

* finish porting

* initialize test_runtime from genesis correctly

* remove commented code

* maybe unsigned signatures

* runtimes compile

* port over most of network

* serialization for generic types

* fix comment

* remove some unnecessary trait bounds

* network compiles

* tests compile for sync

* fix deserialization

* temporarily remove deserialize derives

* workarounds for serde issues for deriving deserialization

* get demo-runtime compiling on std

* port extrinsic-pool

* primitives reshuffling

* get network compiling again

* remove debugging file

* runtime tests now passing

* port client-db

* start to port over substrate-rpc

* mostly port over PolkadotApi

* test_runtime follows normal conventions

* substrate runtime tests pass

* deal with inherent extrinsics correctly in polkadot-api

* port transaction-pool

* port polkadot-consensus

* port substrate-rpc

* everything compiles

* tests compile

* fix grumbles

* test-runtime uses its own transfer type

* switch to master branch of jsonrpc

* fix network tests and some warnings

* all tests pass locally

* [ci-skip] add another comment about issue

* remove some curlies

* light client structure + remote call requests

Tomasz Drwięga authored May 01, 2018 and asynchronous rob committed May 01, 2018

* Test client used in RPC tests.

* Use test-client for network tests.

* Expose BlockOrigin and clean up the API.

* CLI options and keystore integration

* Replace multiqueue with future::mpsc

* BFT gossip

* Revert to app_dirs

* generate_from_seed commented

* Refactor event loop

* Start consensus by timer

* Message buffering

* Minor fixes

* Work around duty-roster issue.

* some more minor fixes

* fix compilation

* more consistent formatting

* make bft input stream never conclude

* Minor fixes

* add timestamp module to executive

* more cleanups and logging

* Fixed message propagation

Arkadiy Paronyan authored Apr 03, 2018 and asynchronous rob committed Apr 03, 2018

* CLI options and keystore integration

* Replace multiqueue with future::mpsc

* BFT gossip

* Revert to app_dirs

* generate_from_seed commented

Arkadiy Paronyan authored Mar 30, 2018 and asynchronous rob committed Mar 30, 2018

* Block import notifications

* Build fix

* Consensus messages supported in the networking

* Started consensus service

* BFT service

* Transaction propagation

* Polkadot service

* CLI integration

* Build fix

* Added signatures validation

* Removed executor argument

* Refactored steam loops; Queue size increased

* Limit queue size

* Fixed doc comment

* Fixed wasm build

* Fixed wasm build

* Check id properly

Gav Wood authored Feb 19, 2018 and asynchronous rob committed Feb 19, 2018

* Block builder (substrate)

* Fix wasm build

* Bulid on any block

* Test for block builder.

* Block import tests for client.

* Tidy ups

* Repotted client

* Avoid pointless work

* All backend stuff now manages optional storage.

Also simplified a lot of the backend.

* Native runtime-io now supports empty storage items.

* Finish up the API transition.

* Build fix.

* Fix tests.

* Remerge in changes to client.

* Final fixes.

* Unrevert typos

* Remove accidentally committed change

* Bring back zero copy

* Fix merge.

* reshuffle consensus libraries

* polkadot-useful type definitions for statement table

* begin BftService

* primary selection logic

* bft service implementation without I/O

* extract out `BlockImport` trait

* allow bft primitives to compile on wasm

* Block builder (substrate)

* take polkadot-consensus down to the core.

* test for preemption

* fix test build

* Fix wasm build

* Bulid on any block

* Test for block builder.

* Block import tests for client.

* Tidy ups

* clean up block builder instantiation

* justification verification logic

* JustifiedHeader and import

* Propert block generation for tests

* Fixed rpc tests

Gav Wood authored Feb 15, 2018 and asynchronous rob committed Feb 15, 2018

* Block builder (substrate)

* Fix wasm build

* Bulid on any block

* Test for block builder.

* Block import tests for client.

* Tidy ups

* clean up block builder instantiation

* Propert block generation for tests

* Fixed rpc tests

Gav Wood authored Feb 12, 2018 and asynchronous rob committed Feb 12, 2018

* Remove rustc dependency from ed25519 and refactor a little.

* Runtime support provides more extensive test-key functionality.

* Additional APIs for ed25519 stuff.

* Extensive test for test-runtime.

* Fixes for the new test key API.

* Additional convenience for tests

* Take advantage of more convenient API.

* Redo formating.

* Remove old test identities.

* Remove boilerplate, add test.

* Refactor out unneeded code.

* Clean up algo for determining authorities.

* Remove unneeded API.

* Make `to_*` consume

* Only export keyring when testing

* Fix build & warning

* Extract Keyring into separate library.

* Add tests for Keyring and a trait-based API.

* Address grumbles.

* Introduce simple blockchain runtime for substrate tests.

* Remove bad files.

* Add needed wasm binaries.

* Refactoring.

- Repot files in test-runtime.
- Rename troublesome `Joiner::join` to `Joiner::and`.
- Rework `Slicable` to dedup code.

* More fixes and refactoring

* Rebuild substrate test wasm.

* Fix merge errors.

* Rename the disasterously named `to_vec` to `encode`.

Also rename `as_slice_then` to `with_encoded`.

* Tests for toy runtime.

* Fix doc nit

Re-introduce zero copy codec and add minimal polkadot client API which uses linked native runtime (#65)

* client-api type and move duty roster types to primitives

* tuple implementation for slicable

* mild cleanup of deserialization code

* stubs which handle encoding and decoding themselves

* fancier impl_stubs macro

* zero-copy slicable API

* minimal polkadot-client API

* fix WASM API generation

* move native environment stuff to substrate executor

* fix warnings and grumbles