* Consensus cleanup

* Fixed DB initialization issue

* Spacing

* changes_trie

* changs_trie: continue

* changes_trie: adding tests

* fixed TODO

* removed obsolete ExtrinsicChanges

* encodable ChangesTrieConfiguration

* removed polkadot fle

* fixed grumbles

* ext_storage_changes_root returns u32

* moved changes trie root to digest

* removed commented code

* read storage values from native code

* fixed grumbles

* fixed grumbles

* missing comma

* key changes proof generation + query

* fix grumbles

* check that changes trie config is not changed by block.finalize()

* fixed changes trie config check

* Started substrate tests

* Sync test

* Test updates

* Improved tests

* Use on-chain block delay

* Parallel test execution

* Otimized tests

* Logging

* Fixed racing test

* Fixed compilation

* Fixed timestamp test

* Removed rlp dependency

* Minor fixes

* Fixed tests

* Removed best_block_id and resolved fdlimit issue

* Whitespace

* Use keyring

* Style

* Added API execution setting

* Removed stale import

snd authored Sep 26, 2018 and asynchronous rob committed Sep 26, 2018

* add stub for Client.best_chain_containing_block_hash

* add fn blockchain::Backend::leaf_hashes

* fix typo

* sketch out Client.best_chain_containing_block_hash

* fix indent

* Blockchain.leaf_hashes -> Blockchain.leaves

* add unimplemented! stub impls for Blockchain.leaves

* start impl of Blockchain.leaves for in-memory client db

* Client.best_chain_containing...: check canonical first and make compile

* first rough attempt at maintaining leaf list in in-memory db

* fix tab indent

* add test best_chain_containing_single_block

* add failing test best_chain_containing_with_fork

* pub use client::blockchain; in test-client to prevent circular dep in client tests

* best_chain_containing_with_single_block: improve and test leaves

* far improve in-memory Backend::leaves impl

* test blockchain::Backend::leaves more thoroughly

* handle more edge cases in blockchain::Backend::leaves impl for in memory

* fix test best_chain_containing_with_fork (two distinct test blocks had same hash)

* make best_chain_containing_block_hash pass existing tests

* improve docstring for Blockchain::leaves

* Client.best_chain_containing: some cleanup. support max_block_number

* best_chain_containing: remove broken outcommented fast check for best = canonical

* remove blank line

* best_block_containing: return None if target_hash not found

* best_chain_containing: add unreachable! at end of function

* improve tests for best_chain_containing

* renames

* more elaborate test scenario for best_containing

* best_containing: fix restriction of search through maybe_max_number

* best_containing: tests for restriction of search

* get rid of unnecessary clones

* replace Client::new_in_mem by new_with_backend which is useful for testing backends

* add test_client::new_with_backend for testing different backend impls

* add test for in_mem::Backend::leaves

* remove unused imports

* in_mem test_leaves: simplify

* flesh out tests for in_mem leaves impl

* remove tests for leaves from client which are now covered in implementing module

* improve comment

* add Client.new_in_mem again

* unwrap in test_client::new_with_backend

* make test_client::BlockBuilderExt work not just with in-mem backend

* make test client ext not just work with in mem backend

* add failing Backend.leaves test for client-db

* update Cargo.lock

* replace KeccakHasher with Blake2Hasher

* refactor

address grumble https://github.com/paritytech/substrate/pull/740#discussion_r217822862

* refactor using NumberFor

address grumble https://github.com/paritytech/substrate/pull/740#discussion_r217823341

* add test that exposes possible problem

* update docstring for Client.best_containing

* extract test for Backend.leaves for reuse

* improve test blockchain_header_and_hash_return_blocks_from_canonical_chain_given_block_numbers

* extract test_blockchain_query_by_number_gets_canonical to easily test multiple impls

* remove whitespace

* remove todo

* Client.best_containing: pre-empt search loop when target in canonical

* best_containing: prevent race condition by holding import lock

* add todo

* extract leaf list update code into function

* add comment

* client-db: use in-memory-kvdb for tests

* use BTreeSet to store leaves for in-mem which is faster and simpler

* add docstring

* add comments and fix formatting

* add initial raw version of LeafSet

* remove Client::update_leaves which has been superceded by LeafSet

* use LeafSet in in-mem backend

* keccak -> blake2

* don't reexport codec traits in primitives

addresses https://github.com/paritytech/substrate/pull/740#discussion_r219538185

* fix rebase mistake

* improve LeafSet and use it in state-db

* correct Transfer nonces to fix ApplyExtinsicFailed(Stale)

* use given backend in canoncal test

* kill dead tree-route code in util

* fix warnings

* tests for leafset

* reorganizations in in_mem backend

* fix reorganization canon block logic

* DB commit and safe reversion on write error

* fix style nits

Gav Wood authored Sep 25, 2018 and Arkadiy Paronyan committed Sep 25, 2018

* Bump codec

* Fix tests

* Patricia trie builds

* Introduce trie

* Some yak shaving.

* Some fixes

* Remove RLP ref

* Fixes

* It builds!

* Some tests fixed

* Another test fix

* Rejig more hashes

* substrate-trie::iterator_works test

* Update lock

* Polish

* Docs

* Undo incorrect "fix" for tests

* Fix nits

asynchronous rob authored Sep 24, 2018 and Arkadiy Paronyan committed Sep 24, 2018

* finalization for in_mem

* fetch last finalized block

* pruning: use canonical term instead of final

* finalize blocks in full node

* begin to port light client DB

* add tree-route

* keep number index consistent in full nodes

* fix tests

* disable cache and finish porting light client

* add AsMut to system module

* final leaf is always best

* fix all tests

* Fix comment and trace

* removed unused Into call

* add comment on behavior of `finalize_block`

* move `tree_route` to client common

* tree_route tests

* return slices in TreeRoute

* apply finality up to parent

* add `finalize_block` call

* adjust formatting

* finality notifications and add last finalized block to chain info

* exhaustive match and comments

* fix sync tests by using non-instant finality

* finalization for in_mem

* fetch last finalized block

* pruning: use canonical term instead of final

* finalize blocks in full node

* begin to port light client DB

* add tree-route

* keep number index consistent in full nodes

* fix tests

* disable cache and finish porting light client

* add AsMut to system module

* final leaf is always best

* fix all tests

* Fix comment and trace

* removed unused Into call

* add comment on behavior of `finalize_block`

* Address grumbles

* Fix hash

* Initial groundwork

* A mess.

* Integrate

* Fix tests

* Unit tests

* Tests for unchecked_extrisnic

* fix tab

* Improve binary format.

* fix tests

* Rename extrinsic-pool -> transaction-pool

Closes #770

* Implement unimplemented.

* typo

* Move module metadata from json string to custom metadata

* Revisit the metadata structures

1. Move the structures into the metadata crate.
2. Switch to using Cow/MaybeOwnedArray to support Encode/Decode

* Adapt to new metadata structures

* Convert event json metadata to new metadata structures

* Convert storage json metadata to new metadata structures

* Convert runtime metadata from json to new metadata structs

* Implements new metadata structures in client and runtime

* Fixes errors on `no_std`

* Fixes errors after rebasing master

* Do not use `Cow` anymore in metadata

Also replace `String` with our own type definition `StringBuf`.
This fixes compilation on `no_std`.

* Wrap `RuntimeMetadata` in `RuntimeMetadataVersioned` to support versioning

* Move metadata into `srml` and make core unaware of the implementation

* Extract well known keys into a single place

* Fixes.

* changes_trie

* changs_trie: continue

* changes_trie: adding tests

* fixed TODO

* removed obsolete ExtrinsicChanges

* encodable ChangesTrieConfiguration

* removed polkadot fle

* fixed grumbles

* ext_storage_changes_root returns u32

* moved changes trie root to digest

* removed commented code

* read storage values from native code

* fixed grumbles

* fixed grumbles

* missing comma

Gav Wood authored Sep 12, 2018 and Arkadiy Paronyan committed Sep 12, 2018

* Remove unneeded script

* Rename Substrate Demo -> Substrate

* Rename demo -> node

* Build wasm from last rename.

* Merge ed25519 into substrate-primitives

* Minor tweak

* Rename substrate -> core

* Move substrate-runtime-support to core/runtime/support

* Rename/move substrate-runtime-version

* Move codec up a level

* Rename substrate-codec -> parity-codec

* Move environmental up a level

* Move pwasm-* up to top, ready for removal

* Remove requirement of s-r-support from s-r-primitives

* Move core/runtime/primitives into core/runtime-primitives

* Remove s-r-support dep from s-r-version

* Remove dep of s-r-support from bft

* Remove dep of s-r-support from node/consensus

* Sever all other core deps from s-r-support

* Forgot the no_std directive

* Rename non-SRML modules to sr-* to avoid match clashes

* Move runtime/* to srml/*

* Rename substrate-runtime-* -> srml-*

* Move srml to top-level

* Remove PoC-1 back-compat hacks

* Fix lingering Keccak references

* More keccak exorcism

* Fix test

* Final fixes

* Introduce `impl_event!` for declaring the Event/RawEvent for a module

* Implements a function for generating the event json metadata

* Update runtime `json_metadata` to include the new event json metadata

* Switch to `impl_event!` for new modules

* Remove unused module declaration in `impl_event!`

* Rename `impl_event!` to `decl_event!`

* Fixes some style nits

* Adds `impl_json_metadata!` for generating all metadata of a runtime

* Adds `json_metadata` RPC call

* Adds simple test for the `json_metadata` RPC call

* Implements json metadata in the demo runtime

* Fix indent

* Adds missing copyright headers

* Dispatch json metadata renamings and improvements

* Replaces `format!` & `String` with `Vec<JSONMetadata`

* Implements `Encode` and `Decode` for JSONMetadata

* Make `impl_json_metadata!` compileable on `no_std`

* Adapt the client to decode the correct type for `json_metadata`

* Fixes compile error and warning

* Whitespace

* Updating substrate-demo

* Consenus fixes

* Reverted toolchain change

* Adjusted timeout formula

* Simplfied proposal creation

* Fixed tests

* Fixed a few small issues

* 2017->2018

* Style

* More style

* Renamed demo executable to substrate

* Style

* Fixed compilation after merge

* Style

* storage proofs

* CHT

* more accurate consensus superseding logic

* mild revision to `can_build_on` logic

* fix a deadlock when spawning agreement as non-authority

* dropping BFT future before poll doesn't lead to service deadlock

* push cancel to BFTfuture rather than waiting for task

Gav Wood authored Aug 15, 2018 and asynchronous rob committed Aug 15, 2018

* Genric over hasher

* WIP start adding NodeCodec

* Add codec to TrieBackend

* Typechecks

* Fix error type

* Cleanup

* Tests build (and fail)

* Fix tests: don't use MemoryDB::default()

* Lockfile

* Address grumbles

* Teach environmental! about generics

* Add Finder artifacts

* whitespace

* Add a toy impl of Hasher and plug it in to Externalities

* Use `uint` and `fixed-hash` from `parity-common`
Remove unused U512
Add test to ensure H256 impls heapsizeof

* lock file updates

* Make hashes Encodable/Decodable

* lock file updates

* Impl FromIterator for TestExternalities so we can collect() and use map!

* Use rustc-hex from crates
Use rlp from master so dependencies do not mess up the scope

* Fix tests in runtime-io

* lockfile shenanigans

* Add a BlakeHasher impl

* Use BlakeHasher in runtime-io

* lockfile updates

* ws

* Add a Blake2/RLP-flavoured NodeCodec

* Use Blake-flavoured Hasher and NodeCodec

* lockfile

* Implement PartialEq and Default for TestExternalities

* Add note about limitations of environmental!

* Make it compile, but this is probably broken

* Derive Debug so tests in executor can work

* Make executor use BlakeHasher

* ws

* WIP make client generic

* typechecks

* cleanup

* client tests pass

* Fix client/db

* cleanup

* Fix network

* Fix rpc

* Fix service

* Make TestExternalities work better au lieu d'un HashMap

* Fix tests in council

* Fix tests in contract

* Fix tests in council

* Fix democracy

* Add comment about odd-looking reexports in tests

* Don't need to load branch

* Fix staking

* Fix session

* Some polkadot fixes and lockfile

* Fix executive

* fixup lockfile

* Fix polkadot/api

* Fix polkadot/service

* Fix polkadot/runtime tests

* Fix tests in test-runtime

* Test fixes

* Fix missing component in the `std` feature

* Use PhantomData and Result from core

* Fix paths
Use core

* load heapsize on wasm

* implement `HeapSizeOf` for wasm

* Add toy impl of `blake2_256` for no_std

* lockfile

* Use kvdb* from parity-common and fix errors

* rebuilt lockfile

* Add dummy impl of `on_advance_round` for rhododendron::Context

* Fix build after merge

* Add HeapSizeOf bound where needed

* Sort out dependencies for no_std

* Add HeapSizeOf bound where needed

* use temp branch pending PR merges

* Remove unneeded tests

* Lock file and wasm artifacts

* lockfile

* Use magic commit for libp2p

* Cleanup

* Implement blake2_256 for no_std

* Back on parity-common master

* missing type params

* Update Cargo.lock

* whitespace

* Rename concrete Rlp node codec "RlpCodec" and use everywhere
Implement a Keccak-flavoured Rlp NodeCodec and use everywhere
Add a KeccakHasher

* Switch to use KeccakHasher

* Lock file and runtimes

* fixup lockfile

* Fix outstanding issue using concrete types (thanks @gnunicorn)

* Cleanup

* More cleanup

* Comment out Blake2 Hasher

* implement ext_keccak256

* Address todo: FetchChecker is generic

* all tests passing

* storage proofs

* fixed grumbles

* Update lib.rs

* substrate: return Option in all storage related RPC methods

* substrate: remove unused imports

* substrate: remove unused NoValueForKey error variant

* substrate: don't return Option on code_at since code is always defined

* Send initial value for heads.

* Send initial values for subscriptions.

Tomasz Drwięga authored Aug 01, 2018 and Svyatoslav Nikolsky committed Aug 01, 2018

* Initial implementation of storage events.

* Attaching storage events.

* Expose storage modification stream over RPC.

* Use FNV for hashing small keys.

* Fix and add tests.

* Swap alias and RPC name.

* Fix demo.

* Addressing review grumbles.

* Fix comment.

* use db in light clients

* fixed comment

* fixed grumbles

* blockchain data cache

* fixed grumbles

* fix compilation

* keep the last entry in the cache

* fixed grumbles

* fixed reporting of pruned entries

* Commit extrinsics changes

* Removed panic=abort

* Commit when needed

* Resotre default hook for the native call

* Revert test

* Proper test

* Sorted errors and fixed wasm build

* Revert chain command

* BLOCKS -> NUM

* Fixed warning

* Refactored Slicable

* Docs

* Wasm build

* Wasm build

* Renamed traits

* Review nits

* Renamed Slicable as well

Gav Wood authored Jul 15, 2018 and Arkadiy Paronyan committed Jul 15, 2018

* Cleanups (remove genesis.wasm & nicer errors)

- Pretty errors for version mismatch
- Remove the need for genesis wasm

* Remove unneeded wasm files

* Improve code of conduct

* Leaner code

* Test fixes

* fix tests

* Fix consensus checking

* Substrate service

* Splitting polkadot service

* Specialised components

* Specialised components

* Docs and style

* Docs and style

* Final touches

* Added db key assertion

* Initial logic

* Remove accidental file

* Config

* Remove accidental

* Apply CLI config

* Additional work. Sadly pointless.

* Rearrange everything

* Loop into CLI param

* Implement dual execution

* typo

* fix tests.

* Better docs

* Fix bug

* Add some tests

* Report block information on consensus failure, tests

* Fix test

* More timely block import notifications

* Grumbles.

* More wrapping

* Fix build

* Fixes

* Refactor to provide performance improvements for RuntimeVersion lookup, too.

Provides a new runtime_version function which, given a specific , returns
the runtime version for that block, either cached or by executing the block
and caching the result. Exposes that function through all clients to make
most use of that performance improvements.

* Block export and import

* Export and import using std streams

* Made AuthorituId::from_slice private

* Runtime version

* Updated genesis.wasm

* Minor fixes

* Fresh runtime

* Default version for pre Poc-2; Fixed authorship interface check

* Fixed authoring check