* blockchain cache pruning strategy

* added some internal docs to cache_pruning_strategy

* Update core/client/db/src/cache/mod.rs

Co-Authored-By: DemiMarie-parity <[email protected]>

* Storage tracing

* Whitepsaces

Co-Authored-By: Sergei Pepyakin <[email protected]>

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>

* Update Cargo.lock

* multiple remote values in single message

* keys_str: String ->keys_str: closure

* Move trait `Printable` into `sr-primitives`

* Cleanup runtime io trie_root interfaces

* Remove last generic bits from sr-io interface

* Fix srml-sudo after master merge

* Fix benchmarks

* Runtime bump

* Print version when panic

* Fix tests.

* removing fetcher dependency from light backend

* fix compilation

* do not panic during execution proof check

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <[email protected]>

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <[email protected]>

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <[email protected]>

* BackendTrustLevel enum

* up runtime version

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <[email protected]>

* Update core/state-machine/src/lib.rs

Co-Authored-By: DemiMarie-parity <[email protected]>

* fixed some grumbles

* Update core/state-machine/src/testing.rs

Co-Authored-By: Gavin Wood <[email protected]>

* Update core/state-machine/src/lib.rs

Co-Authored-By: Gavin Wood <[email protected]>

* mov where

* spaces -> tabs (to restart build)

* Start refactoring state-machine crate

* More improvement to state-machine

* Fix tests compilation on master and remove warnings

* Fix compilation

* Apply suggestions from code review

Co-Authored-By: Sergei Pepyakin <[email protected]>

* Update core/state-machine/src/basic.rs

Co-Authored-By: DemiMarie-parity <[email protected]>

* Line width

* Update core/primitives/src/storage.rs

Co-Authored-By: Benjamin Kampmann <[email protected]>

* Update core/state-machine/src/error.rs

Co-Authored-By: Benjamin Kampmann <[email protected]>

* Review feedback

* Add transactions from retracted blocks back to the pool

* Line width

* Reverse retracted

* changes tries build cache

added CT build cache test

* fix lines width

* fixed some grumbles

* clear cache when: digests disabled, top-level or skewed digest is built

* cached_changed_keys -> with_cached_changed_keys

* client: only report block import to telemetry if new best

* grandpa: fix tests

* consensus: derive Default for ImportedAux

* network: fix test

* srml-system checks

* wip

* more modules compiles

* node-runtime checks

* build.sh passes

* include dispatch error in failed event

* revert some unnecessary changes

* refactor based on comments

* more compile error fixes

* avoid unnecessary into

* reorder code

* fixes some tests

* manually implement encode & decode to avoid i8 workaround

* more test fixes

* more fixes

* more error fixes

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <[email protected]>

* address comments

* test for DispatchError encoding

* tyep alias for democracy

* make error printable

* line width

* fix balances tests

* fix executive test

* fix system tests

* bump version

* ensure consistent method signature

* Apply suggestions from code review

Co-Authored-By: Gavin Wood <[email protected]>

* changes based on review

* Add issue number for TODOs

* fix

* line width

* fix test

* Update core/sr-primitives/src/lib.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update core/sr-primitives/src/traits.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update srml/council/src/motions.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update srml/council/src/motions.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* update based on review

* More concrete macro matching

* fix test build issue

* Update hex-literal dependency version. (#3141)

* Update hex-literal dep version.

* Update lock file.

* Start to rework the new error handling

* More work to get it back compiling

* Start to fix after master merge

* The great transaction error handling refactoring

* Make `decl_error` errors convertible to `&'static str`

* Make srml-executive build again

* Fix `sr-primitives` tests

* More fixes

* Last round of fix ups

* Fix build

* Fix build

* Apply suggestions from code review

Co-Authored-By: Tomasz Drwięga <[email protected]>

* Rename some stuff

* Fixes after master merge

* Adds `CheckBlockGasLimit` signed extension

* Remove debug stuff

* Fix srml-balances test

* Rename `InvalidIndex` to `CannotLookup`

* Remove weird generic parameters

* Rename function again

* Fix import

* Document the signed extension

* Change from `Into` to `From`

* Update srml/contracts/src/lib.rs

Co-Authored-By: Sergei Pepyakin <[email protected]>

* Fix compilation

* Update srml/contracts/src/lib.rs

Co-Authored-By: Tomasz Drwięga <[email protected]>

* Update core/sr-primitives/src/transaction_validity.rs

Co-Authored-By: Tomasz Drwięga <[email protected]>

* Remove unused code

* Fix compilation

* Some cleanups

* Fix compile errors

* Make `TransactionValidity` a `Result`

* Apply suggestions from code review

Co-Authored-By: Gavin Wood <[email protected]>

* Beautify the code a little bit and fix test

* Make `CannotLookup` an inherent error declared by `decl_error!`

* Adds some documentation

* Make `ApplyOutcome` a result

* Up the spec_version

* Apply suggestions from code review

Co-Authored-By: Gavin Wood <[email protected]>
Co-Authored-By: DemiMarie-parity <[email protected]>

cheme authored Sep 02, 2019 and Svyatoslav Nikolsky committed Sep 02, 2019

* Initial implementation, some redundancy is awkward and there is
some useless computation (but there is a pending pr for that).
Next are tests.

* Minimal tests and fix extend child.

* implement iterator for change child trie.

* prune child trie.

* Fix pruning test.

* bump spec version.

* Avoid empty child trie (could also be checked before)

* tabs.

* Fix child digest overriding each others.

* Restore doc deleted on merge.

* Check correct child value on extrinsics build.

* Revert runtime version update.

* chain+state RPCs are async now

* wrapped too long lines

* create full/light RPC impls from service

* use ordering

* post-merge fix

* missing on_custom for child read.

* fix and complete, add test cases.

* shorten line.

* replace vecs of key value tuple with maps.

* generalize tree_root to remove client.backend dependency

* replace client.backend.blockchain.header with client.header

* move used_state_cache_size into client info

* Create intermediate Setup State. Fixes #1134

* remove client.backend from finality proof

* update node-template

* move memory backend into test helper mode

* move test helper into client

* starting the big refactor, remove unused functions

* apply_finality

* apply_finality

* replacing more .backend from environment with client directly

* remove .backend from grandpa by using traits

* remove .backend from babe

* remove .backend from tests where it is not needed

* remove .backend from tests

* fixing tests

* fixing tests

* fixing more tests

* fixing tests

* fix all forks test

* fix style

* fixing unnecessary allocation

* remove old test.

* fix service docs

* apply suggestion

* minor clean ups

* turns out the test-helper features actually is being used!

* fixing line length.

* fix line length

* minor cleaning

* Apply suggestions from code review

thanks, @Basti

Co-Authored-By: Bastian Köcher <[email protected]>

* address grumbles

* simplify finalize block on client

* move block back into inner function

* Apply suggestions from code review

Co-Authored-By: DemiMarie-parity <[email protected]>

* use as.ref instead of match

* Update core/client/src/backend.rs

Co-Authored-By: DemiMarie-parity <[email protected]>

* CT config methods are accepting zero

* intrduce && use configuration range in CT

* Update core/client/db/src/lib.rs

Co-Authored-By: cheme <[email protected]>

* full PR URI

* updated next_max_level_digest_range comment

* added skewed digest description

* added comment for loop

* call next_max_level_digest_range from prev_max_level_digest_block

* more test cases

* Update core/state-machine/src/changes_trie/surface_iterator.rs

Co-Authored-By: cheme <[email protected]>

* updated comment

* Add ability to supply extra storage in test-client

* Don't use native_equivalent in tests.

* Get rid of native_equivalent

* Try to make fields private

* Apply Basti's suggestions

Co-Authored-By: Bastian Köcher <[email protected]>

* Introduce capabilities filtering for calls.

* Bump impl version.

* Allow RichOffchainCall to only read offchain db.

* Fix code.

* Panic on invalid calls.

* Merge execution contexts and expose capabilities.

* Fix repr

* Re-enable keystore for offchain calls.

* value range in blockchain cache

* revert me (testing for spurious failure)

* Revert "revert me (testing for spurious failure)"

This reverts commit 21a4a3cf.

* client: don't import blocks that revert finality

* client: test import of blocks that revert finality

* client: replace tempdir with tempfile in test

* Using child storage, (srml-support only), test failing .

* fix simple tests.

* Enumerable by requiring owned struct (previous form only allow
&'static).
Broken tests are from genesis init.

* implement for_child_keys_with_prefix

* indent

* clear_child_prefix fix.

* clear_child_prefix fix 2.

* fix for storage_impl, if/when allowing child and not child this could be
reverted.

* Fix lot of urlinked child genesis, still need to look upon actual
genesis srml module code.
Probably still a lot of broken code needing debugging.

* switch well_known_key to their associated module child trie.
Fix a genesis init (balance).
Complete some testing.
Comment some tests before using.

* fixing test runtime child keys

* latest commit fix broken genesis init

* fix system balances child name.

* Important fix: storage_root from test externalities need children (it is
already the case for ext).

* executive root with child calculation

* Avoid empty trie on test ext.

* Symetric removal of key for system.

* commenting changes related tests.

* Remove child module specifics.

* fix issues.

* fix some formatting

* fix bench and bump runtime

* Remove extend_storage_overlays, assimilate_storage do the same as is
proper considering srml macro.

* Fix warning for assimilate.

* Removing kill as they do not impact any test cases.

* Use tuple of storage map instead of two parameters. This changes the
behavior of decl_storage genesis build closure (breaking api).

* Do not use build storage before assimilate.

* fix error

* Update core/state-machine/src/backend.rs

* Cache idle state

* Display import error by default

* Include uncles

* Filter missing uncles

* Moved inherent registration to a new crate

* Ignore invalid inherent encoding

* Add Call type to extensible transactions.

Cleanup some naming

* Merge Resource and BlockExhausted into just Exhausted

* Fix

* Another fix

* Call

* Some fixes

* Fix srml tests.

* Fix all tests.

* Refactor crypto so each application of it has its own type.

* Introduce new AuthorityProvider API into Aura

This will eventually allow for dynamic determination of authority
keys and avoid having to set them directly on CLI.

* Introduce authority determinator for Babe.

Experiment with modular consensus API.

* Work in progress to introduce KeyTypeId and avoid polluting API
with validator IDs

* Finish up drafting imonline

* Rework offchain workers API.

* Rework API implementation.

* Make it compile for wasm, simplify app_crypto.

* Fix compilation of im-online.

* Fix compilation of im-online.

* Fix more compilation errors.

* Make it compile.

* Fixing tests.

* Rewrite `keystore`

* Fix session tests

* Bring back `TryFrom`'s'

* Fix `srml-grandpa`

* Fix `srml-aura`

* Fix consensus babe

* More fixes

* Make service generate keys from dev_seed

* Build fixes

* Remove offchain tests

* More fixes and cleanups

* Fixes finality grandpa

* Fix `consensus-aura`

* Fix cli

* Fix `node-cli`

* Fix chain_spec builder

* Fix doc tests

* Add authority getter for grandpa.

* Test fix

* Fixes

* Make keystore accessible from the runtime

* Move app crypto to its own crate

* Update `Cargo.lock`

* Make the crypto stuff usable from the runtime

* Adds some runtime crypto tests

* Use last finalized block for grandpa authority

* Fix warning

* Adds `SessionKeys` runtime api

* Remove `FinalityPair` and `ConsensusPair`

* Minor governance tweaks to get it inline with docs.

* Make the governance be up to date with the docs.

* Build fixes.

* Generate the inital session keys

* Failing keystore is a hard error

* Make babe work again

* Fix grandpa

* Fix tests

* Disable `keystore` in consensus critical stuff

* Build fix.

* ImOnline supports multiple authorities at once.

* Update core/application-crypto/src/ed25519.rs

* Merge branch 'master' into gav-in-progress

* Remove unneeded code for now.

* Some `session` testing

* Support querying the public keys

* Cleanup offchain

* Remove warnings

* More cleanup

* Apply suggestions from code review

Co-Authored-By: Benjamin Kampmann <[email protected]>

* More cleanups

* JSONRPC API for setting keys.

Also, rename traits::KeyStore* -> traits::BareCryptoStore*

* Bad merge

* Fix integration tests

* Fix test build

* Test fix

* Fixes

* Warnings

* Another warning

* Bump version.

* WIP: update codec

* WIP

* compiling

* WIP

* rename parity-scale-codec to codec

* WIP

* fix

* remove old comments

* use published crates

* fix expected error msg

* bump version

* fmt and fix

* remove old comment

* fix wrong decoding impl

* implement encode like for structures

* undo removal of old pending changes

* trailingzeroinput

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <[email protected]>
Co-Authored-By: DemiMarie-parity <[email protected]>

* update codec

* fmt

* version is 1.0.0

* show more error

* fmt

Pierre Krieger authored Aug 03, 2019 and Arkadiy Paronyan committed Aug 03, 2019

* Mention that in_mem::Backend doesn't work

* Bigger warning

* switch to simple codec, trie broken for now

* Actualy use trie_root_noext

* align some hash, failing test on EMCH comment

* Fix trie code over layout instead of hash, revert legacy code for legacy
mainnet ??

* stub behind LayOut

* fix no_std

* temp solution for legacy trie behind feature legacy-key in various crate

* use remote project

* rc client db need prefix

* update trie deps

* bum spec runtime version

* Removing legacy as default.

* Switch mode to non legacy.

* bump runtime version

* Remove legacy trie compatibility features.

* fix warning

* bump version

* change hash on new test.

* Move dependency (#11 trie PR) patched to a parity repo.
Bench reverted to correct hasher.
Some renaming and doc improvments.

* ChildBitmap renaming to BitMap.

* Renaming of LayOut to Layout.

* formatting.

* Removing abreviation such as _ix nb_ or bm.

* Update deps and apply renaming 'Buff' -> 'Buffer'.

* Align to latest trie crates naming changes.

* Update trie dependency.

* Update trie dependency.

* change block_import test hash

* update trie deps (trie use new scale codec but it does not seems to be
an issue).

* update to use latest trie version (no mgmt of multiple radix).

* tabify

* Restoring test to 10 000.

* Use published crate, trie bench is currently down until publishing
(require another pr to update version).

* Update trie-bench.

* Check the genesis hash in transactions regardless of era.

* Fix check-fees, too.

* Undo.

* Subkey supports new signing.

* Remove unneeded type param.

* Bump tx version

* Build.

* Another build fix

* Build again

* Cleanup

* Another fix.

* Fix

* Fixes

* 6 second blocks.

* Fixes

* Build fix

* Fix

* Fix.

* Reorg test

* Fixed informant misreporting reorgs

* Update cache when reorg is caused by applying finality

* Test for finality reorg

* Simplified test

* Typo

Co-Authored-By: André Silva <[email protected]>

* refactor sr_primitives.

* Fix try build error.

* Line-width

* Ui test.

* Final fixes.

* Fix build again.

* bring back ui test.

* Fix unsigned import.

* Another ui fix.

* Also refactor substrate-primitives

* Fix benchmarks.

* Fix doc test.

* fix doc tests

* Minor cleanup in native_executor.rs

* Fix tests

* Add test from original bug report

Original is from @pepyakin

 in 3d7b27f3421818e8d6de568e02fbc2947a06246b.
I adapted it to work with the latest master.

* No longer cleanup module instance

* Replace runtime cache with synchronous clone

* Fix test

* Preserve initial runtime memory and restore it on fetch

* Remove leftover comment

* Fix style

* Improve variable naming

* Replace get_into() with get()

* Handle missing memory export better

* Return earlier when creating runtime first time

* Improve comments

* fmt

* Fix #2967.

* Eradicate `code` from `Error::InvalidCode`

* tidy

* A state snapshot doc.

* Store multiple runtimes by hash.

* Get rid of deref.

* Docs

* Use Self for instantiate_module

* REVERT ME

* Should be ok

* Commit

* Remove dbg

* Use fast-memory's erase

* Clean and undo hacks.

* Introduce a dedicated error for heap_base

* Ban the start function.

* Clean, docs and refactor

* Add rustflags.

* Update Cargo.lock

* Apply Basti's suggestions

Co-Authored-By: Bastian Köcher <[email protected]>

* Rename allocates_huge_stack_array

* Extend TestClientBuilder with set_heap_pages

* Update the test.

* Update core/executor/src/wasm_executor.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update core/executor/src/wasm_runtimes_cache.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update core/executor/src/error.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Update core/executor/src/error.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Fix tests.

* Update cargo-lock

* Use wasmi master

* Use master wasmi

* Move tests.

* Use wasmi crates.io

* Update Cargo.lock

* Fix build.rs

* Bump runtime version

* Revert initial_heap_pages renaming

* Bump wasmi up to 0.5.0

* Bump runtime version

* Don't restore an instance every now and then

* Update core/executor/src/wasm_runtimes_cache.rs

Co-Authored-By: DemiMarie-parity <[email protected]>

* Propagate error in CacheError

* Clarify the get_heap_base call in instantiation

* Supply --export=__heap_base

See https://reviews.llvm.org/D62744

Co-authored-by: Jim Posen <[email protected]>

* Bump version.

* Use combinators for segments.

* Fix build.rs

* Fix build.rs for runtime-test