This reverts commit f20ae654.

* Make compat with exit-future updates

* Update exit-future entirely

* Tidy

* Bump Substrate

Fedor Sakharov authored Dec 03, 2019 and asynchronous rob committed Dec 03, 2019

* Erasure encoding availability initial commit

 * Modifications to availability store to keep chunks as well as
   reconstructed blocks and extrinsics.
 * Gossip messages containig signed erasure chunks.
 * Requesting eraure chunks with polkadot-specific messages.
 * Validation of erasure chunk messages.

* Apply suggestions from code review

Co-Authored-By: Luke Schoen <[email protected]>

* Fix build after a merge

* Gossip erasure chunk messages under their own topic

* erasure_chunks should use the appropriate topic

* Updates Cargo.lock

* Fixes after merge

* Removes a couple of leftover pieces of code

* Fixes simple stuff from review

* Updates erasure and storage for more flexible logic

* Changes validation and candidate receipt production.

* Adds add_erasure_chunks method

* Fixes most of the nits

* Better validate_collation and validate_receipt functions

* Fixes the tests

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <[email protected]>

* Removes unwrap() calls

* Removes ErasureChunks primitive

* Removes redundant fields from ErasureChunk struct

* AvailabilityStore should store CandidateReceipt

* Changes the way chunk messages are imported and validated.

 * Availability store now stores a validator_index and n_validators for
 each relay_parent.
 * Availability store now also stores candidate receipts.
 * Removes importing chunks in the table and moves it into network
 gossip validation.
 * Validation of erasure messages id done against receipts that are
 stored in the availability store.

* Correctly compute topics for erasure messages

* Removes an unused parameter

* Refactors availability db querying into a helper

* Adds the apis described in the writeup

* Adds a runtime api to extract erasure roots form raw extrinsics.

* Adds a barebone BlockImport impl for avalability store

* Adds the implementation of the availability worker

* Fix build after the merge with master.

* Make availability store API async

* Bring back the default wasmtime feature

* Lines width

* Bump runtime version

* Formatting and dead code elimination

* some style nits (#1)

* More nits and api cleanup

* Disable wasm CI for availability-store

* Another nit

* Formatting

* rename crates as appropriate

* Rename to use master - easy to revert, but shows it's working

* Bump runtime version

* Bump version

* Bump Substrate again

* Update to latest Substrate master (#615)

* Update to latest Substrate master

* Remove unneeded patch + warning

* Update `Cargo.lock`

* Fix tests

* Update again

* Bump Substrate (#616)

* Update lock

* Fix

* Few fixes

* Bump to latest Substrate

* Fixes

* fix pre-tx-pool compilation

* more compilation fixes

* Updates for the injection period

- Liberal slash-refunding
- Instant unbonding

* *: Enable refactored authority discovery (#624)

* *: Enable authority discovery module

* *: List authority discovery id after parachain validator id

Make sure existing key types don't change their order by appending the
authority discovery id instead of injecting it between im online id and
parachain validator id.

* *: Gate authority discovery module behind feature flag

* cli/src/lib.rs: Fix warnings

* cli/src/lib.rs: Shorten line length

* Bump Substrate

* Bump Substrate

* Line widths

* Line widths again

* Revert bump.

* Update to latest Substrate master (#570)

* Bump substrate/version (#557)

* Bump version and Substrate (#560)

* Bump version and Substrate

* Bump version and Substrate

* Bump versions

* bump substrate to release specific v0.6.15

* Update lock

* Prepare Polkadot update for Substrate runtime interface 2.0 (#563)

* Prepare Polkadot update for Substrate runtime interface 2.0

* bump substrate to release specific v0.6.15

* Switch to `polkadot-master`

* Version bump

* Master backports

* Bump runtime

* Fix tests

* Fix tests

* Another fix.

* core logic for ICMP gossip

* refactor gossip to make more extension friendly

* move files aroun

* extract attestation-gossip logic to its own module

* message validation and broadcast logic

* fix upstream crates' compilation

* add a test

* another test for overlapping

* Some grammar and phrasing tweaks

Co-Authored-By: Luke Schoen <[email protected]>

* add since parameter to ingress runtime API

* broadcast out known unrouted message queues

* fix compilation of service and collator

* remove useless index_mapping

* some tests for icmp propagation

* fix decoding bug and test icmp queue validation

* simplify engine-id definition

Co-Authored-By: Sergei Pepyakin <[email protected]>

* address some grumbles

* some cleanup of old circulation code

* give network a handle to extrinsic store on startup

* an honest collator ensures data available as well

* address some grumbles

* add docs; rename the attestation session to "leaf work"

* module docs

* move gossip back to gossip.rs

* clean up and document attestation-gossip a bit

* some more docs on the availability store

* store all outgoing message queues in the availability store

* filter `Extrinsic` out of validation crate

* expunge Extrinsic from network

* expunge Extrinsic from erasure-coding

* expunge Extrinsic from collator

* expunge from adder-collator

* rename ExtrinsicStore to AvailabilityStore everywhere

* annotate and clean up message-routing tests

* Integrate srml/im-online

* Fix all build errors with old aura.

* Fix most of the build errors.

* Builds and tests seem to pass (I will not trust this commit yet)

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <[email protected]>

* Kill some warnings.

* fix panics on 0 validators

* Fix dev chain.

* Fix author stuff

* fix im online integration.

* Some tweaks

* Introduce app-crypto

* Initial build work

* codec update / tweaks

* patch polkadot-erasure-coding input

* More fixes for new crypto

* More fixes

* Update parachains module

* evamp parachain crypto

* More crypto work.

* Chain spec and service.

* ChainSpec stuff

* Last bits for a clean build

* Tweak coment

* adapt polkadot-validation to the new keystore

* polkadot-network compiles, but tests don't

* Integrate the new parachain validation stuff

* delete message_routing file

* make polkadot-network tests compile and pass

* runtime tests compile and pass

* update substrate ref

* service compiles

* all tests pass

* Add TODO, change branch back to polkadot-master

* Lock file

* TODOs done

* Issue number

* Remove old tODO

* Remove commented code

* Make `communication_for` exit when we end a round

* Fix compilation

* Best effort to bring up to date.

* Fix the executor stuff

* Update verisons.

* Finish fixing

* Final fixes and warnings.

* add some docs and bump Wasm versions

* Fix tests

* Fix final test

* Introduce `BuildParachainContext` trait

* Change the structure and hide the actual network implementation behind a
trait

* Add functions to collator `Network` trait

* Expose `collator_id_to_peer_id`

* `ValidationNetwork` expose `checked_statements`

* Style nit

By using a strongly typed network service, we make sure that we send and
receive the correct messages. Before there was a bug, a `SignedStatement`
was sent and a `GossipMessage` was decoded, but this could never work.

Bastian Köcher authored Jun 20, 2019 and asynchronous rob committed Jun 20, 2019

* Update to latest Substrate master + warning fixes

* Update runtime/src/lib.rs

Co-Authored-By: thiolliere <[email protected]>

Fix typo

* Update substrate deps and fix runtime compilation

* Fix compilation

* Enable `std` feature

* strip out all ICMP code and begin gossip refactor

* validate incoming statements

* message_allowed logic

* compiles

* do reporting and neighbor packet validation

* tests compile

* propagate gossip messages

* test message_allowed

* some more tests

* address grumbles

thiolliere authored May 15, 2019 and asynchronous rob committed May 15, 2019

* WIP

* merging select_chain

* WIP

* update to point to gui-polkadot-master

* Fix collator

* update gui-polkadot-master and fix

* fix unwraps

* better returning an error

Stanislav Tkach authored May 08, 2019 and asynchronous rob committed May 08, 2019

* Don't pass validators' public keys with attestations

* Update statement-table's Misbehaviour typedef

* Update network/router

* Expand MessageValidationData

* Try to fix tests

* Extend ApiContext

* Remove 'index_mapping' from the SessionParams

* Construct index_mapping from authorities

* Move index_mapping to TableContext

* Add test for index_mapping order

* port polkadot_runtime and polkadot_validation

* update storages build (#245)

* all tests pass

* rebuild wasm

* validators expect collators to give them parachain messages

* mostly port network to use pov_block

* network tests pass

* verify ingress when fetching pov block

* fix runtime compilation

* all tests build

* fix some grumbles

* Update validation/src/collation.rs

Co-Authored-By: rphmeier <[email protected]>

* Update primitives/src/parachain.rs

Co-Authored-By: rphmeier <[email protected]>

* Update network/src/lib.rs

Co-Authored-By: rphmeier <[email protected]>

Bastian Köcher authored Mar 28, 2019 and asynchronous rob committed Mar 28, 2019

* Rebuild runtime

* Remove invalid value from chainspec (#68)

* service: use grandpa block import for locally sealed aura blocks (#85)

* bump version to v0.3.1

* Update lock file.

* limit number of transactions when building blocks (#91)

* Update to latest Substrate

* Bump to 0.3.2

* Actually bump.

* v0.3.2 (#98)

* bump substrate version

* fix polkadot-collator

* point to alexander-backports of substrate

* bump version

* cli: fix node shutdown (#100)

* update to latest substrate, change to v0.3.4

* update to latest substrate, bump version to 0.3.5

* v0.3.6

* try to build on every v0.3 commit and update alexander-backports

* bump to v0.3.7

* bump to 0.3.8

* Bump to 0.3.9: network and pruning improvements

* Bump to 0.3.10: reduce network bandwidth usage

* Use libp2p-kad 0.3.2 (#122)

* Bump libp2p-identify to 0.3.1 (#123)

* Bump to 0.3.12 (#127)

* Update Substrate again (#128)

* update substrate and bump version to v0.3.13

* bump version to v0.3.14: fix --reserved-nodes

* add a manually curated grandpa module (#136)

* updating v0.3 to use substrate v0.10 (#146)

* updating to latest substrate v0.10

* better handling of outer poll

* nit

* fix tests

* remove comment

* reduce indentation

* use self.poll

* bring oneshot into scope

* spaces

* wrap

* remove match

* wrap

* Update primitives/Cargo.toml

Co-Authored-By: gterzian <[email protected]>

* Update runtime/wasm/Cargo.toml

Co-Authored-By: gterzian <[email protected]>

* Update runtime/wasm/Cargo.toml

Co-Authored-By: gterzian <[email protected]>

* Update test-parachains/adder/collator/src/main.rs

Co-Authored-By: gterzian <[email protected]>

* indent

* add paranthese

* config: fix wrong ip for alexander bootnode (#161)

* fix curated-grandpa and rebuild wasm (#162)

* [v0.3] Integrates new gossip system into Polkadot (#166)

* new gossip validation in network

* integrate new gossip into service

* network: guard validation network future under exit signal (#168)

* bump version to v0.3.15: substrate v0.10

* [v0.3] update to substrate master (#175)

* update to substrate master

* fix test

* service: fix telemetry endpoints on alexander chainspec (#169) (#178)

* Update v0.3 to latest Substrate master (#177)

* update substrate v0.3 to latest master

* bump spec version

* update to latest master: remove fees module

* update runtime blobs

* bump version to 0.3.16

* replace sr25519 accountid with anysigner

* bump version to v0.3.17

* Some PoC-3 GRANDPA tweaks (#181)

* call on_finalise after triggering curated_grandpa change

* make grandpa rounds shorter for faster finalization

* use authorities when calculating duty roster (#185)

* [v0.3] Update to substrate master (#183)

* update to latest substrate master

* bump version to 0.3.18

* update to latest substrate master

* bump spec version

* update runtime wasm blobs

* remove current_offline_slash from chain spec

* update to substrate master: bump version to v0.3.19 (#188)

* update to substrate master: bump version to v0.3.19

libp2p network improvements

* network: replace NodeIndex with PeerId

* network: fix tests

* polkadot v0.3.20 (#190)

* update to substrate master: bump version to 0.3.20

* runtime: add offchain worker trait

* runtime: rebuild wasm blobs

* bump spec version (#191)

* Fix compilation

* Update version to 0.4.0

* Switch to use `polkadot-master` branch from substrate

* Remove unused struct

* Remove `grandpa::SyncedAuthorities` from `OnSessionChange`

* refactor out a consensus data fetcher from table router

* move statement checking logic into router

* refuse to start authority if collator

* support building the table router asynchronously

* instantiate_consensus does not overwrite old

* update key in new consensus if there was none before

* collator collects ingress from network

* test produced egress roots

* fix adder-collator compilation

* address first grumbles

* integrate new gossip with collator network launch

* address review

* Update to master

This introduces a new type `CollatorId`, currently just `SessionKey`
but which would forseeably change to its own thing. It seems to work
like this (despite there being a lot of the new-incompatible
`AccountId` replaced). No idea if it does anything sensible, though.

* Cleanups

* Fix tests

* Remove commented code

* Specify commit hash

* Remove commented code

* Correct version

* Update runtime/Cargo.toml

Co-Authored-By: gavofyork <[email protected]>

* PairT instead of _Pair

* Update lock file

* Remove rev causing upset

* Integrates new gossip system into Polkadot (#166)

* new gossip validation in network

* integrate new gossip into service

* Fix build

* Fix claims module

* fix warning

* update to latest master again

* update runtime

Andrew Jones authored Feb 21, 2019 and asynchronous rob committed Feb 21, 2019

* Initial rename of consensus -> validation

* Rename crate imports

* network: rename consensus to validation

* network: rename consensus in comments and logs

* Grumbles

* Rename tests consensus -> validation

* compute ingress and routing in polkadot runtime

* extract parent candidates from block when beginning consensus

* fetch incoming messages when validating

* fix consensus tests

* parachain wasm execution uses messages

* update parachain tests to check if messages are executed

* abstract out network service to make room for network tests

* skeleton for incoming data fetch

* collate ingress from consensus-gossip

* keep track of validated candidates in the shared-table

* add some shared_table tests for new behavior

* broadcast egress messages on gossip

* test compute_ingress

* move network tests to module folder

* dummy network for consensus-network tests

* make consensus network generic over executor

* test egress broadcast and ingress fetch

* fix test compilation

* address some grumbles

* address grumbles and fix parachain shuffle

* remove broadcast parameter from consensus network trait

* update to latest substrate - protocol API update

* nits

* remove the wait on a future

* use std mpsc for oneshot

* read head-data directly out of WASM memory

* implement ext_post_message for parachain WASM

* further refactoring of the parachain module

* add externalities error type

* accumulate posted messages when validating parachain candidate

* define Extrinsic type in primitives

* availability-store: store extrinsic data

* compute extrinsic and check against candidate

* add some egress queue tests

* grumbles & substrate update

* ensure everything builds

* remove availability vote type from statement-table

* expunge availability statement from consensus module

* expunge availability from duty roster

* rename StatementProducer to ParachainWork

* fix runtime tests and remove availability statement variant

* update wasm

* multiple consensus sessions in network

* tests compile, add a test for RecentSessionKeys

* track recently received session keys from validators

* add a test for desired key-sending behavior

* Fix up wasm runtime build

* Fixes for runtime

* Fix.

* More fixes

* Runtime builds on native.

* Native and wasm both build without warnings.

* Fix runtime tests.

* Merge #20

* Final fix for native runtime.

* Compile polkadot wo consensus

* Reverted changes to polkadot-consensus

* reintroduce minimal subset of consensus

* reintroduce checked_block to runtime for std

* polkadot_consensus compiles without most of the code

* remove checked_block again and do more checks in parachains for runtime

* uncomment proposer

* remove offline tracker

* extract out parachain-attestation logic from proposal directly

* reintroduce transaction_pool

* write some custom aura verification logic for the block verifier

* use transaction pool in more generic way

* service compiles again

* polkadot-network and tests pass

* remove unused session_key function from router

* everything but CLI compiles due to service hell

* Fixes compilation of `polkadot_cli`

* everything compiles

* update adder wasm