* add new apis to externalities

* Update mod.rs

* update for new host function

* Rename unused

* cargo update -p sp-io

* Add whitelists

* Apply changes to polkadot runtime as well

Co-authored-by: kianenigma <kian.peymani@gmail.com>

* Companion for Substrate #6269 (nested storage tx)

* Adjust to PR changes

* Bump to newest substrate

* Bump spec

* Some fixes to compile for Android

* Revert change to cli

* add storage_append

* Bump versions

Co-authored-by: Gav Wood <gavin@parity.io>

* Companion pr for Substrate #5741

* update to substrate master

Co-authored-by: André Silva <andre.beat@gmail.com>

The `TaskExecutor` extension is now required by Substrate to batch verify signatures.

Co-authored-by: NikVolf <nikvolf@gmail.com>

* add extension registrations

* update to latest

* merge&update

* upgrade primitives to allow changing validation function

* set up storage schema for old parachains code

* fix compilation errors

* fix test compilation

* add some tests for past code meta

* most of the runtime logic for code upgrades

* implement old-code pruning

* add a couple tests

* clean up remaining TODOs

* add a whole bunch of tests for runtime functionality

* remove unused function

* fix runtime compilation

* extract some primitives to parachain crate

* add validation-code upgrades to validation params and result

* extend validation params with code upgrade fields

* provide maximums to validation params

* port test-parachains

* add a code-upgrader test-parachain and tests

* fix collator tests

* move test-parachains to own folder to work around compilation errors

* fix test compilation

* update the Cargo.lock

* fix parachains tests

* remove dbg! invocation

* use new pool in code-upgrader

* bump lockfile

* link TODO to issue

* refactor out validation hosts to pool struct

* make web-wasm compatible

* typo

* remove now-unused static hosts

* Flag to force kusama runtime

* Chainspecs for kusama

* Polkadot config for westend

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* network/src/legacy/gossip: Wrap GossipEngine in Arc Mutex & lock it on use

`GossipEngine` in itself has no need to be Send and Sync, given that it
does not rely on separately spawned background tasks anymore.
`RegisteredMessageValidator` needs to be `Send` and `Sync` due to the
inherited trait bounds from implementing `GossipService`. In addition
`RegisteredMessageValidator` derives `Clone`. Thereby `GossipEngine`
needs to be wrapped in an `Arc` and `Mutex` to keep the status quo.

* Needed fixes.

* Fixes

* Fixed build

* Fixed build w benchmarking CLI

* Fixed building tests

* Added --dev shortcut

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>

* expunge legacy code from polkadot-network

* mostly rip out old legacy protocol from service

* ensure validation work is spawned by incoming messages

* decouple availabliity store from network logic; clean up data flow

* av_store: test helpers and use futures-abort

* update polkadot-validation to pass n_validators when submitting chunks

* fallible erasure-chunk fetching

* implement `ErasureNetworking` for new network prot

* API for registering availability store in network

* fully integrate new network service into service

* fix validation tests

* scaffolding for porting collator over to new network

* track connected validators' peer IDs and distribute collators' collations

* helper in network for fetching all checked statements

* fix adder-collator

* actually register notifications protocol

* Update service/src/lib.rs

* Make needed changes to service

* Merge two companion PRs.

- #880
- #881

* Some effort towards compilation

* Fix

* remove `NetworkSpecialization` references from network

* fix compilation errors in service and collator

* ensure protocol name is valid

* Fixes

* Fix

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>

* Switch branch

* Small changes

* Update substrate branch

* Switch

* Revert "Switch branch"

This reverts commit b9d48b2c.

* fix

* add `wipe` and `commit`

* Remove deprecated_host_interface

* Switch branch

* HasherFor -> HashFor

* More HasherFor changes

* Final touches

* Revert "Switch branch"

This reverts commit d0da2731

.

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* encode the candidate statement as only the hash

* refactor CandidateReceipt and CollationInfo

* introduce an abridged candidate receipt type

* erasure coding stores candidate receipt

* store omitted data instead and introduce AvailableData type

* refactor availability-store schema

* tweak schema and APIs a bit more

* get availability-store tests passing

* accept AbridgedCandidateReceipt in `set_heads`

* change statement type in primitives to be hash-only

* fix parachains runtime tests

* fix bad merge

* rewrite validation pipeline

* remove evaluation module

* use abridged candidate hash as canonical

* statement table uses abridged candidate receipts

* kill availability_store::Data struct

* port shared table to new validation pipelines

* extract full validation pipeline to helper

* remove old validation pipeline from collation module

* polkadot-validation compiles

* polkadot-validation tests compile

* make local collation available in validation service

* port legacy network code

* polkadot-network fully ported

* network: ensure fresh statement is propagated

* remove pov_block_hash from LocalValidationData

* remove candidate_hash field from AttestedCandidate and update runtime

* port runtimes to new ParachainHost definition

* port over polkadot-collator

* fix test compilation

* better fix

* remove unrelated validation work dispatch fix

* address grumbles

* fix equality check

* Remove TargetedMessage

* Nitpicks

* update latest substrate polkadot-master

* fix test compilation

* bump version to 0.7.18

* bump impl_version

* update substrate

* Revert "Instantiate environment with asynchronous API (#768)"

This reverts commit 989db4b8.

* update substrate

* remove unused parameter type

* bump trie-db version for tests

* fix collator test

* update substrate

* remove unnecessary service changes

* Add some more bounds

* More bounds

* More fixes

* More fixes

* Bump versions

* Update for tipping treasury

* Bump substrate

* Fixes

* Put send_consensus back in.

* Fix test

* Fixes

* Fixes

* Fix warning

* Make use of `runtime_interface` for parachain externalities

This also changes the encoding of the `ValidationResult` return value to
match the default encoding used in Substrate.

* Fix compilation for web

* Update `Cargo.lock`

* Include feedback

* Move proc macro

* Update parachain/src/lib.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

Signed-off-by: yjhmelody <465402634@qq.com>

Signed-off-by: yjhmelody <465402634@qq.com>

Bastian Köcher authored Oct 08, 2019 and asynchronous rob committed Oct 08, 2019

Also fix the weird file structure by making `wasm_executor.rs` ->
`wasm_executor/mod.rs`.

Arkadiy Paronyan authored Aug 24, 2019 and asynchronous rob committed Aug 24, 2019

* Allow parallel parachain validation

* Fixed test interference

* Switch to pooled implementation

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update validation_host.rs

* Minor cleanups

* Fixed build

* Integrate srml/im-online

* Fix all build errors with old aura.

* Fix most of the build errors.

* Builds and tests seem to pass (I will not trust this commit yet)

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Kill some warnings.

* fix panics on 0 validators

* Fix dev chain.

* Fix author stuff

* fix im online integration.

* Some tweaks

* Introduce app-crypto

* Initial build work

* codec update / tweaks

* patch polkadot-erasure-coding input

* More fixes for new crypto

* More fixes

* Update parachains module

* evamp parachain crypto

* More crypto work.

* Chain spec and service.

* ChainSpec stuff

* Last bits for a clean build

* Tweak coment

* adapt polkadot-validation to the new keystore

* polkadot-network compiles, but tests don't

* Integrate the new parachain validation stuff

* delete message_routing file

* make polkadot-network tests compile and pass

* runtime tests compile and pass

* update substrate ref

* service compiles

* all tests pass

* Add TODO, change branch back to polkadot-master

* Lock file

* TODOs done

* Issue number

* Remove old tODO

* Remove commented code

* Improved execution & tests

* Style

* Made CLI arg const

* Moved Upwards message

* CLI subcommand for validation worker

* Build halting parachain

* Build halting parachain

* Made stuff private

* Reorganized parachain tests

* Comment

* Whitespace

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Fixed call data size check and introduced an enum

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Make `polkadot-parachain` call `validate_block` instead of `validate`

Also switch to rust 2018 in the crate

* Use `rstd`

* Make `load_params` a pointer

* Convert validation error

* Convert wasm_executor error

* Convert block evaluation error

* Convert collation errors and the compilation

* Remove error-chain dep from service

* Remove unused Result type

* Remove unused error variants

* Remove redundant intos

* Add missing comments

* Update validation/src/collation.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Fix new error variant

* Slots module

* Integrate slots

* More drafting

* Minor updates

* Update parachains to use trati

* More build fixes

* Full code now compiles

* Add renew bid function

* Implement calculate_winner

* Warning remove

* Update gitignore

* Test framework

* Tests

* Further testing

* More tests, new parameterisation.

* Fix and new test

* Thread-safe tests

* Test off-boarding and a fix.

* Test onboarding

* Allow late onboarding.

* Another test and fix

* Avoid println in nostd

* Compact representation of paraids

* Introduce documentation.

* Introduce events.

* Additional test and fix

* Additional test

* Tidy up line lengths.

* Remove printlns

* Use later substrate utils.

* Allow parachains to send messages.

* Fix build/test

* Make slots work with latest substrate

* Update runtime/src/slot_range.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Polish logic

* Rewind to earlier substrate master

* Remove dead code.

* Fix build

* Update substrate ref to master

* Update to new inherent digests API

* address grumbles

* fix

* Fix a warning.

* Reworded a comment.

* Check that receipt matches expectations

* Add test for final checks

* Split out queuing logic.

* Test final piece of queuing logic

* Fix up docs.

* More docs fixes

* define Id type in polkadot-parachain and depend on that in primitives.

* fix tests

* read head-data directly out of WASM memory

* implement ext_post_message for parachain WASM

* further refactoring of the parachain module

* add externalities error type

* accumulate posted messages when validating parachain candidate

* define Extrinsic type in primitives

* availability-store: store extrinsic data

* compute extrinsic and check against candidate

* add some egress queue tests

* grumbles & substrate update

* ensure everything builds

* Refactored Slicable

* Docs

* Wasm build

* Wasm build

* Renamed traits

* Review nits

* Renamed Slicable as well

* dynamic inclusion threshold calculator

* collators interface

* collation helpers

* initial proposal-creation future

* create proposer when asked to propose

* remove local_availability duty

* statement table tracks includable parachain count

* beginnings of timing future

* finish proposal logic

* remove stray println

* extract shared table to separate module

* change ordering

* includability tracking

* fix doc

* initial changes to parachains module

* initialise dummy block before API calls

* give polkadot control over round proposer based on random seed

* propose only after enough candidates

* flesh out parachains module a bit more

* set_heads

* actually introduce set_heads to runtime

* update block_builder to accept parachains

* split block validity errors from real errors in evaluation

* update WASM runtimes

* polkadot-api methods for parachains additions

* delay evaluation until candidates are ready

* comments

* fix dynamic inclusion with zero initial

* test for includability tracker

* wasm validation of parachain candidates

* move primitives to primitives crate

* remove runtime-std dependency from codec

* adjust doc

* polkadot-parachain-primitives

* kill legacy polkadot-validator crate

* basic-add test chain

* test for basic_add parachain

* move to test-chains dir

* use wasm-build

* new wasm directory layout

* reorganize a bit more

* Fix for rh-minimal-parachain (#141)

* Remove extern "C"

We already encountered such behavior (bug?) in pwasm-std, I believe.

* Fix `panic_fmt` signature by adding `_col`

Wrong `panic_fmt` signature can inhibit some optimizations in LTO mode.

* Add linker flags and use wasm-gc in build script

Pass --import-memory to LLD to emit wasm binary with imported memory.

Also use wasm-gc instead of wasm-build.

* Fix effective_max.

I'm not sure why it was the way it was actually.

* Recompile wasm.

* Fix indent

* more basic_add tests

* validate parachain WASM

* produce statements on receiving statements

* tests for reactive statement production

* fix build

* add OOM lang item to runtime-io

* use dynamic_inclusion when evaluating as well

* fix update_includable_count

* remove dead code

* grumbles

* actually defer round_proposer logic

* update wasm

* address a few more grumbles

* grumbles

* update WASM checkins

* remove dependency on tokio-timer