Gav Wood authored Jun 06, 2018 and asynchronous rob committed Jun 06, 2018

* Some initial work on RPC and client

* Rephrase as params

* More work on traitifying substrate.

* Traitify in_mem.rs

* traitify client.rs

* Make new primitives (mainly traits) build again.

* Many (superficial) build fixes throughout.

* Fix remaining build issues up to bft interface.

* Make bft primitives be generic.

* Switch out MisBehaviorReport for generic version.

* Merge Hashing into Header.

* Update runtime for new generics (with Hashing).

* Update demo runtime.

* Make runtime compile.

* Build fixes for runtime

* Remove old modules.

* port substrate-bft to use generic substrate types

* port client

* port substrate-test-runtime

* mostly port test-runtime to get compiling for std

* Ensure `AccountId` has a `Default`.

* Fix type deps.

* finish porting

* initialize test_runtime from genesis correctly

* remove commented code

* maybe unsigned signatures

* runtimes compile

* port over most of network

* serialization for generic types

* fix comment

* remove some unnecessary trait bounds

* network compiles

* tests compile for sync

* fix deserialization

* temporarily remove deserialize derives

* workarounds for serde issues for deriving deserialization

* get demo-runtime compiling on std

* port extrinsic-pool

* primitives reshuffling

* get network compiling again

* remove debugging file

* runtime tests now passing

* port client-db

* start to port over substrate-rpc

* mostly port over PolkadotApi

* test_runtime follows normal conventions

* substrate runtime tests pass

* deal with inherent extrinsics correctly in polkadot-api

* port transaction-pool

* port polkadot-consensus

* port substrate-rpc

* everything compiles

* tests compile

* fix grumbles

* test-runtime uses its own transfer type

* switch to master branch of jsonrpc

* fix network tests and some warnings

* all tests pass locally

* [ci-skip] add another comment about issue

* remove some curlies

* Apply inversion of control to contract module.

* Extract contract to it's own crate.

* Rebuild binaries.

* Use [panic_implementation] instead of lang item

* Rebuild binaries.

* Merge remote-tracking branch 'origin/master' into gav-xts-dont-panic

* Update wasm.

* consensus, session and staking all panic-safe.

* Democracy doesn't panic in apply.

* Fix tests.

* Extra helper macro, council depanicked.

* Fix one test.

* Fix up all council tests. No panics!

* Council voting depanicked.

* Dispatch returns result.

* session & staking tests updated

* Fix democracy tests.

* Fix council tests.

* Fix up polkadot parachains in runtime

* Fix borked merge

* Merge remote-tracking branch 'origin/master' into gav-xts-dont-panic

* Update wasm.

* consensus, session and staking all panic-safe.

* Democracy doesn't panic in apply.

* Fix tests.

* Extra helper macro, council depanicked.

* Fix one test.

* Fix up all council tests. No panics!

* Council voting depanicked.

* utilise hygene

* dynamic inclusion threshold calculator

* collators interface

* collation helpers

* initial proposal-creation future

* create proposer when asked to propose

* remove local_availability duty

* statement table tracks includable parachain count

* beginnings of timing future

* finish proposal logic

* remove stray println

* extract shared table to separate module

* change ordering

* includability tracking

* fix doc

* initial changes to parachains module

* initialise dummy block before API calls

* give polkadot control over round proposer based on random seed

* propose only after enough candidates

* flesh out parachains module a bit more

* set_heads

* actually introduce set_heads to runtime

* update block_builder to accept parachains

* split block validity errors from real errors in evaluation

* update WASM runtimes

* polkadot-api methods for parachains additions

* delay evaluation until candidates are ready

* comments

* fix dynamic inclusion with zero initial

* test for includability tracker

* wasm validation of parachain candidates

* move primitives to primitives crate

* remove runtime-std dependency from codec

* adjust doc

* polkadot-parachain-primitives

* kill legacy polkadot-validator crate

* basic-add test chain

* test for basic_add parachain

* move to test-chains dir

* use wasm-build

* new wasm directory layout

* reorganize a bit more

* Fix for rh-minimal-parachain (#141)

* Remove extern "C"

We already encountered such behavior (bug?) in pwasm-std, I believe.

* Fix `panic_fmt` signature by adding `_col`

Wrong `panic_fmt` signature can inhibit some optimizations in LTO mode.

* Add linker flags and use wasm-gc in build script

Pass --import-memory to LLD to emit wasm binary with imported memory.

Also use wasm-gc instead of wasm-build.

* Fix effective_max.

I'm not sure why it was the way it was actually.

* Recompile wasm.

* Fix indent

* more basic_add tests

* validate parachain WASM

* produce statements on receiving statements

* tests for reactive statement production

* fix build

* add OOM lang item to runtime-io

* use dynamic_inclusion when evaluating as well

* fix update_includable_count

* remove dead code

* grumbles

* actually defer round_proposer logic

* update wasm

* address a few more grumbles

* grumbles

* update WASM checkins

* remove dependency on tokio-timer

Gav Wood authored May 24, 2018 and asynchronous rob committed May 24, 2018

* Add 128-bit shim and use for balance.

* Fix tests.

* Fix wasm build.

* Add teardowns.

* Rebuild binaries.

* Make Trait::Balance: Slicable

* Rebuild binaries.

* Extract mock module.

* Wasm smart-contract instrumentation

* Rebuild binaries.

* Memory limits and tests.

* initialise dummy block before API calls

* test for random-seed

* revert polkadot-runtime changes

* Fix initialisations and add a test.

* Fix test.

* Fix overflow bug.

* Minor refactoring and fixes.

* Fix vote threshold.

* Add note.

* Fixes for latest rust and the readme.

* Better readme.

* An extra validator for PoC-1

* Update README.

* PoC-1 bootnodes.

* don't return async::notready for messages without scheduling wakeup

* Fix endowed account

* give polkadot control over round proposer based on random seed

* address grumbles.

* DB backend

* DB backend

* Cleanup

* Clean build files after running tests

* Fixed comment

* add OOM lang item to runtime-io

Sergey Pepyakin authored May 01, 2018 and asynchronous rob committed May 01, 2018

* Add primitives for sandboxing.

* Add sandbox module.

* Implement the runtime part of the sandbox.

* Rebuild binaries.

* Implement smart-contract execution.

* Add more documentation.

Gav Wood authored Apr 16, 2018 and asynchronous rob committed Apr 16, 2018

* Disable CORS validation, improve logging.

* Fix build from nightly API change.

* Rebuilt binaries

* kill post_finalise

* update wasm runtimes

Gav Wood authored Apr 12, 2018 and asynchronous rob committed Apr 12, 2018

* extrinsics root is calculated as part of block-building.

* Fix build.

* Fix xt root.

* Couple of fixes.

* Logging and more useful APIs.

* Fix test.

* Update log version.

* Switch back to correct version of log.

Sergey Pepyakin authored Apr 11, 2018 and asynchronous rob committed Apr 11, 2018

* Refactor Externalities to AccountDb.

* Recompile wasm.

* Implement transaction queue RPC.

* whitespace

* Support without_std env in environmental!  (#110)

* Make environmental crate support without_std env.

* Small doc fixes.

* Remove dead code.

* CLI options and keystore integration

* Replace multiqueue with future::mpsc

* BFT gossip

* Revert to app_dirs

* generate_from_seed commented

* Refactor event loop

* Start consensus by timer

* Message buffering

* Minor fixes

* Work around duty-roster issue.

* some more minor fixes

* fix compilation

* more consistent formatting

* make bft input stream never conclude

* Minor fixes

* add timestamp module to executive

* more cleanups and logging

* Fixed message propagation

Gav Wood authored Apr 05, 2018 and asynchronous rob committed Apr 05, 2018

* Initial stuff.

* Various fixes.

* Fix tests.

* Fix another test

* Fix another test.

* Docs in polkadot runtime.

* Fix up ser/de tests.

* Update god keys

* Syntax

* Fix

* Merge remote-tracking branch 'origin/master' into gav-merge-runtime

* Permissions on init.sh

* Port-over the whitespace from @rphmeier

* Rename

* Merge branch 'master' into gav-merge-runtime

* Fix typo.

* Fix grumbles.

* Make more idiomatic.

* Move `Ed25519Signature` out of traits.

* Factor out safe-mix and dispatch

* Refactor dispatch into something more modular.

* Fix wasm build.

* Fix up timestamp

* fix warnings.

* Borked timestamp example

* Fix build

* Timestamp as skeleton for traity runtime.

* New storage macro.

* Dispatch module has traity API.

* Move consensus module to new API

* Refactoring and outer dispatch

* Avoid unnecessary derives.

* Abstract the low-level half of system.

* nicer outer dispatch syntax.

* Make runtime compile again (albeit in a heavily simplified state)

* Reworking runtime and the upper levels of system.

* Initial reworking of runtime:

- Introduced executive module;
- Introduced trait primitives module;
- Provided an API endpoint.

* Expose an additional function in system

* Another couple of functions traitified in executive.

* another function in executive traitified.

* One more function traitified.

* Finish traitifying executive!

* Traitify session module.

* Cleanups and ensure session gets run.

* First part of traitification of staking module.

* Bit more of staking traitified.

* Additional stuff in staking. Fix up session.

* Penultimate part of staking module.

* Final part of staking (code)

* Update demo runtime to include staking.

* Final tweaks for staking integration.

* Remove old runtime files.

* Schedule staking.

* Minor fixes

* First bits of democracy.

* Democracy module integrated.

* Fix warning.

* Traitify and integrate council module

* Council voting.

* Runtime binary and tweaks.

* Binary update.

* Fix `*Type` grumble.

* Fix up genesis_map

* Remove NonTrivialSlicable

* Staking "test externalities" stuff along with refactor.

* Add session test externalities constructor

* Fixed executor tests.

* Make one test in executive module work.

* Remove test framework stuff into common module.

* Enable other tests in executive

* Session tests reinstated, minor refactoring of keyring.

* Fix staking tests.

* Fix up democracy tests.

* First few tests in council.

* Council tests reinstated :)

* Avoid hardcoding blake2 into Header.

* Fix last few tests.

* Make all primitives generic.

* Fix tests.

* Refactor runtime to remove genesismap.

* Streamline runtime more with macrofied config.

* Clean paths

* Fix warning.

* Consolidate demo runtime crate.

* Remove stale code.

* Refactor away dodgy trait.

* Add corresponding Aux type.

* Fixes

* Rename Digesty -> Digest

* Rename Headery -> Header

* Blocky -> Block

* Fix wasm build.

* kill warnings

* more docs

* minor cleanups

* Reserve pages for heap at initialization time

Since it is freshly allocated memory it won't collide with stack, data or whatever else.

* Compile all wasm files with LTO.

Modules compiled without LTO contain undesired imports and outright invalid (e.g. type mismatches inside the wasm).

* Completely rework dispatch mechanism into something modular.

Not yet complete but 75% there.

* Council vote tests.

* Fix tests.

* whitespace.

* Fix demo runtime tests.

* Fix up tests.

* Remove dead code.

* Initial util code for random beacon

* Timestamp uses new storage API.

* Move over system module to new API.

* Much nicer storage API, moved over staking module.

* More refactoring.

* Democracy uses new storage API.

* Council uses new RPC.

* Fix more tests.

* Use match for Id

* Generic mix.

* Integrate random beacon

* Update binaries.

* Fixes relating to with_ext removal.

* Remove dead code.

* Rework mixer into an iterator adaptor.

* Link to paper.

* Algorithm cleanups

* Merge and fix test.

* Docs.

* Fix typo.

* rename

* Fix tests.

* Completely rework dispatch mechanism into something modular.

Not yet complete but 75% there.

* Council vote tests.

* Fix tests.

* whitespace.

* Fix demo runtime tests.

* Fix up tests.

* Remove dead code.

* Timestamp uses new storage API.

* Move over system module to new API.

* Much nicer storage API, moved over staking module.

* More refactoring.

* Democracy uses new storage API.

* Council uses new RPC.

* Fix more tests.

* Use match for Id

* Use match for Id

* Make PrivPass better protected.

* Address other grumbles.

* Give PrivPass a private member.

* Testing PrivPass.

* Add docs.

* Recompile binaries after merge.

* Remove duplicated code.

* New binaries.

* Docs

* Docs

* avoid use of (arguably) confusing terminology.

* Completely rework dispatch mechanism into something modular.

Not yet complete but 75% there.

* Council vote tests.

* Fix tests.

* whitespace.

* Fix demo runtime tests.

* Fix up tests.

* Remove dead code.

* Use match for Id

* Make PrivPass better protected.

* Address other grumbles.

* Give PrivPass a private member.

* Testing PrivPass.

* Add docs.

* double-commit and prepare misbehavior

* get misbehavior on completion

* collect misbehavior on drop, not only on success

* kill unused transaction_index field

* add primitive misbehavior report type

* add misbehavior report transaction

* store prior session

* fix set_items

* basic checks for misbehavior reports

* crate for substrate bft misbehavior checking

* integrate misbehavior check crate

* fix comment

* new wasm binaries

* fix hash in test

* import misbehavior transactions into queue

* fix test build

* sign on digest and full proposal when proposing

* detect proposal misbehavior

* fix fallout

* restore balance/bondage types

* reshuffle consensus libraries

* polkadot-useful type definitions for statement table

* begin BftService

* primary selection logic

* bft service implementation without I/O

* extract out `BlockImport` trait

* allow bft primitives to compile on wasm

* Block builder (substrate)

* take polkadot-consensus down to the core.

* test for preemption

* fix test build

* Fix wasm build

* Bulid on any block

* Test for block builder.

* Block import tests for client.

* Tidy ups

* clean up block builder instantiation

* justification verification logic

* JustifiedHeader and import

* Propert block generation for tests

* network and tablerouter trait

* use statement import to drive creation of further statements

* Fixed rpc tests

* custom error type for consensus

* create proposer

* asynchronous proposal evaluation

* inherent transactions in polkadot runtime

* fix tests to match real polkadot block constraints

* implicitly generate inherent functions

* add inherent transaction functionality to block body

* block builder logic for polkadot

* some tests for the polkadot API

* avoid redundancy in native code compatibility check

* helper for extracting nonce

* transaction pool implementation

* transaction pool

* integrate transaction pool with proposer

* indentation

* kill storage keys module

* accept new transactions to replace old

* reshuffle consensus libraries

* polkadot-useful type definitions for statement table

* begin BftService

* primary selection logic

* bft service implementation without I/O

* extract out `BlockImport` trait

* allow bft primitives to compile on wasm

* Block builder (substrate)

* take polkadot-consensus down to the core.

* test for preemption

* fix test build

* Fix wasm build

* Bulid on any block

* Test for block builder.

* Block import tests for client.

* Tidy ups

* clean up block builder instantiation

* justification verification logic

* JustifiedHeader and import

* Propert block generation for tests

* network and tablerouter trait

* use statement import to drive creation of further statements

* Fixed rpc tests

* custom error type for consensus

* create proposer

* asynchronous proposal evaluation

* inherent transactions in polkadot runtime

* fix tests to match real polkadot block constraints

* implicitly generate inherent functions

* add inherent transaction functionality to block body

* block builder logic for polkadot

* some tests for the polkadot API

Gav Wood authored Feb 19, 2018 and asynchronous rob committed Feb 19, 2018

* Block builder (substrate)

* Fix wasm build

* Bulid on any block

* Test for block builder.

* Block import tests for client.

* Tidy ups

* Repotted client

* Avoid pointless work

* All backend stuff now manages optional storage.

Also simplified a lot of the backend.

* Native runtime-io now supports empty storage items.

* Finish up the API transition.

* Build fix.

* Fix tests.

* Remerge in changes to client.

* Final fixes.

* Unrevert typos

* Remove accidentally committed change

* Bring back zero copy

* Fix merge.

Gav Wood authored Feb 15, 2018 and asynchronous rob committed Feb 15, 2018

* Block builder (substrate)

* Fix wasm build

* Bulid on any block

* Test for block builder.

* Block import tests for client.

* Tidy ups

* clean up block builder instantiation

* Propert block generation for tests

* Fixed rpc tests

* reshuffle consensus libraries

* polkadot-useful type definitions for statement table

* begin BftService

* primary selection logic

* bft service implementation without I/O

* extract out `BlockImport` trait

* allow bft primitives to compile on wasm

* take polkadot-consensus down to the core.

* test for preemption

* fix test build

Gav Wood authored Feb 12, 2018 and asynchronous rob committed Feb 12, 2018

* Remove rustc dependency from ed25519 and refactor a little.

* Runtime support provides more extensive test-key functionality.

* Additional APIs for ed25519 stuff.

* Extensive test for test-runtime.

* Fixes for the new test key API.

* Additional convenience for tests

* Take advantage of more convenient API.

* Redo formating.

* Remove old test identities.

* Remove boilerplate, add test.

* Refactor out unneeded code.

* Clean up algo for determining authorities.

* Remove unneeded API.

* Make `to_*` consume

* Only export keyring when testing

* Fix build & warning

* Extract Keyring into separate library.

* Add tests for Keyring and a trait-based API.

* Address grumbles.

* Introduce simple blockchain runtime for substrate tests.

* Remove bad files.

* Add needed wasm binaries.

* Refactoring.

- Repot files in test-runtime.
- Rename troublesome `Joiner::join` to `Joiner::and`.
- Rework `Slicable` to dedup code.

* More fixes and refactoring

* Rebuild substrate test wasm.

* Fix merge errors.

* Rename the disasterously named `to_vec` to `encode`.

Also rename `as_slice_then` to `with_encoded`.

* Tests for toy runtime.

* Fix doc nit