* update to latest substrate master

* Fix compilation

* core logic for ICMP gossip

* refactor gossip to make more extension friendly

* move files aroun

* extract attestation-gossip logic to its own module

* message validation and broadcast logic

* fix upstream crates' compilation

* add a test

* another test for overlapping

* Some grammar and phrasing tweaks

Co-Authored-By: Luke Schoen <[email protected]>

* add since parameter to ingress runtime API

* broadcast out known unrouted message queues

* fix compilation of service and collator

* remove useless index_mapping

* some tests for icmp propagation

* fix decoding bug and test icmp queue validation

* simplify engine-id definition

Co-Authored-By: Sergei Pepyakin <[email protected]>

* address some grumbles

* some cleanup of old circulation code

* give network a handle to extrinsic store on startup

* an honest collator ensures data available as well

* address some grumbles

* add docs; rename the attestation session to "leaf work"

* module docs

* move gossip back to gossip.rs

* clean up and document attestation-gossip a bit

* some more docs on the availability store

* store all outgoing message queues in the availability store

* filter `Extrinsic` out of validation crate

* expunge Extrinsic from network

* expunge Extrinsic from erasure-coding

* expunge Extrinsic from collator

* expunge from adder-collator

* rename ExtrinsicStore to AvailabilityStore everywhere

* annotate and clean up message-routing tests

* Integrate srml/im-online

* Fix all build errors with old aura.

* Fix most of the build errors.

* Builds and tests seem to pass (I will not trust this commit yet)

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <[email protected]>

* Kill some warnings.

* fix panics on 0 validators

* Fix dev chain.

* Fix author stuff

* fix im online integration.

* Some tweaks

* Introduce app-crypto

* Initial build work

* codec update / tweaks

* patch polkadot-erasure-coding input

* More fixes for new crypto

* More fixes

* Update parachains module

* evamp parachain crypto

* More crypto work.

* Chain spec and service.

* ChainSpec stuff

* Last bits for a clean build

* Tweak coment

* adapt polkadot-validation to the new keystore

* polkadot-network compiles, but tests don't

* Integrate the new parachain validation stuff

* delete message_routing file

* make polkadot-network tests compile and pass

* runtime tests compile and pass

* update substrate ref

* service compiles

* all tests pass

* Add TODO, change branch back to polkadot-master

* Lock file

* TODOs done

* Issue number

* Remove old tODO

* Remove commented code

* Make `collator::Network` require `Send + Sync` to make it work

* Update packages

* Update to latest Substrate

* Make it compile and make tests work

* Use `polkadot-master`

* Fix CI

* Remove `build.sh` from readmes

* Delete old stuff

* Bring one back

* Best effort to bring up to date.

* Fix the executor stuff

* Update verisons.

* Finish fixing

* Final fixes and warnings.

* add some docs and bump Wasm versions

* Fix tests

* Fix final test

* Expose `collator_id_to_peer_id`

* `ValidationNetwork` expose `checked_statements`

* Style nit

By using a strongly typed network service, we make sure that we send and
receive the correct messages. Before there was a bug, a `SignedStatement`
was sent and a `GossipMessage` was decoded, but this could never work.

Bastian Köcher authored Jun 20, 2019 and asynchronous rob committed Jun 20, 2019

* Update to latest Substrate master + warning fixes

* Update runtime/src/lib.rs

Co-Authored-By: thiolliere <[email protected]>

* track unrouted ingress in runtime

* test ingress routing

* fix compilation

* add space

Co-Authored-By: Gavin Wood <[email protected]>

* Slots module

* Integrate slots

* More drafting

* Minor updates

* Update parachains to use trati

* More build fixes

* Full code now compiles

* Add renew bid function

* Implement calculate_winner

* Warning remove

* Update gitignore

* Test framework

* Tests

* Further testing

* More tests, new parameterisation.

* Fix and new test

* Thread-safe tests

* Test off-boarding and a fix.

* Test onboarding

* Allow late onboarding.

* Another test and fix

* Avoid println in nostd

* Compact representation of paraids

* Introduce documentation.

* Introduce events.

* Additional test and fix

* Additional test

* Tidy up line lengths.

* Remove printlns

* Use later substrate utils.

* Allow parachains to send messages.

* Fix build/test

* Make slots work with latest substrate

* Update runtime/src/slot_range.rs

Co-Authored-By: Robert Habermeier <[email protected]>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Polish logic

* Rewind to earlier substrate master

* Remove dead code.

* Fix build

* Update substrate ref to master

* Update to new inherent digests API

* address grumbles

* fix

* Fix a warning.

* Reworded a comment.

* Check that receipt matches expectations

* Add test for final checks

* Split out queuing logic.

* Test final piece of queuing logic

* Fix up docs.

* More docs fixes

* strip out all ICMP code and begin gossip refactor

* validate incoming statements

* message_allowed logic

* compiles

* do reporting and neighbor packet validation

* tests compile

* propagate gossip messages

* test message_allowed

* some more tests

* address grumbles

* positive updates to reputations when good things happen

* address review

* branch on good or bad pov blocks

* add a new constant for bad pov blocks

* update substrate reference

* bump wasm

* port polkadot_runtime and polkadot_validation

* update storages build (#245)

* all tests pass

* rebuild wasm

* validators expect collators to give them parachain messages

* mostly port network to use pov_block

* network tests pass

* verify ingress when fetching pov block

* fix runtime compilation

* all tests build

* fix some grumbles

* Update validation/src/collation.rs

Co-Authored-By: rphmeier <[email protected]>

* Update primitives/src/parachain.rs

Co-Authored-By: rphmeier <[email protected]>

* Update network/src/lib.rs

Co-Authored-By: rphmeier <[email protected]>

joe petrowski authored Apr 08, 2019 and asynchronous rob committed Apr 08, 2019

* network and validation

* remainder of todos

* ref new issue

* change issue number

Bastian Köcher authored Mar 28, 2019 and asynchronous rob committed Mar 28, 2019

* Rebuild runtime

* Remove invalid value from chainspec (#68)

* service: use grandpa block import for locally sealed aura blocks (#85)

* bump version to v0.3.1

* Update lock file.

* limit number of transactions when building blocks (#91)

* Update to latest Substrate

* Bump to 0.3.2

* Actually bump.

* v0.3.2 (#98)

* bump substrate version

* fix polkadot-collator

* point to alexander-backports of substrate

* bump version

* cli: fix node shutdown (#100)

* update to latest substrate, change to v0.3.4

* update to latest substrate, bump version to 0.3.5

* v0.3.6

* try to build on every v0.3 commit and update alexander-backports

* bump to v0.3.7

* bump to 0.3.8

* Bump to 0.3.9: network and pruning improvements

* Bump to 0.3.10: reduce network bandwidth usage

* Use libp2p-kad 0.3.2 (#122)

* Bump libp2p-identify to 0.3.1 (#123)

* Bump to 0.3.12 (#127)

* Update Substrate again (#128)

* update substrate and bump version to v0.3.13

* bump version to v0.3.14: fix --reserved-nodes

* add a manually curated grandpa module (#136)

* updating v0.3 to use substrate v0.10 (#146)

* updating to latest substrate v0.10

* better handling of outer poll

* nit

* fix tests

* remove comment

* reduce indentation

* use self.poll

* bring oneshot into scope

* spaces

* wrap

* remove match

* wrap

* Update primitives/Cargo.toml

Co-Authored-By: gterzian <[email protected]>

* Update runtime/wasm/Cargo.toml

Co-Authored-By: gterzian <[email protected]>

* Update runtime/wasm/Cargo.toml

Co-Authored-By: gterzian <[email protected]>

* Update test-parachains/adder/collator/src/main.rs

Co-Authored-By: gterzian <[email protected]>

* indent

* add paranthese

* config: fix wrong ip for alexander bootnode (#161)

* fix curated-grandpa and rebuild wasm (#162)

* [v0.3] Integrates new gossip system into Polkadot (#166)

* new gossip validation in network

* integrate new gossip into service

* network: guard validation network future under exit signal (#168)

* bump version to v0.3.15: substrate v0.10

* [v0.3] update to substrate master (#175)

* update to substrate master

* fix test

* service: fix telemetry endpoints on alexander chainspec (#169) (#178)

* Update v0.3 to latest Substrate master (#177)

* update substrate v0.3 to latest master

* bump spec version

* update to latest master: remove fees module

* update runtime blobs

* bump version to 0.3.16

* replace sr25519 accountid with anysigner

* bump version to v0.3.17

* Some PoC-3 GRANDPA tweaks (#181)

* call on_finalise after triggering curated_grandpa change

* make grandpa rounds shorter for faster finalization

* use authorities when calculating duty roster (#185)

* [v0.3] Update to substrate master (#183)

* update to latest substrate master

* bump version to 0.3.18

* update to latest substrate master

* bump spec version

* update runtime wasm blobs

* remove current_offline_slash from chain spec

* update to substrate master: bump version to v0.3.19 (#188)

* update to substrate master: bump version to v0.3.19

libp2p network improvements

* network: replace NodeIndex with PeerId

* network: fix tests

* polkadot v0.3.20 (#190)

* update to substrate master: bump version to 0.3.20

* runtime: add offchain worker trait

* runtime: rebuild wasm blobs

* bump spec version (#191)

* Fix compilation

* Update version to 0.4.0

* Switch to use `polkadot-master` branch from substrate

* Remove unused struct

* Remove `grandpa::SyncedAuthorities` from `OnSessionChange`

* refactor out a consensus data fetcher from table router

* move statement checking logic into router

* refuse to start authority if collator

* support building the table router asynchronously

* instantiate_consensus does not overwrite old

* update key in new consensus if there was none before

* collator collects ingress from network

* test produced egress roots

* fix adder-collator compilation

* address first grumbles

* integrate new gossip with collator network launch

* address review

* Update to master

This introduces a new type `CollatorId`, currently just `SessionKey`
but which would forseeably change to its own thing. It seems to work
like this (despite there being a lot of the new-incompatible
`AccountId` replaced). No idea if it does anything sensible, though.

* Cleanups

* Fix tests

* Remove commented code

* Specify commit hash

* Remove commented code

* Correct version

* Update runtime/Cargo.toml

Co-Authored-By: gavofyork <[email protected]>

* PairT instead of _Pair

* Update lock file

* Remove rev causing upset

* Integrates new gossip system into Polkadot (#166)

* new gossip validation in network

* integrate new gossip into service

* Fix build

* Fix claims module

* fix warning

* update to latest master again

* update runtime

Andrew Jones authored Feb 21, 2019 and asynchronous rob committed Feb 21, 2019

* Initial rename of consensus -> validation

* Rename crate imports

* network: rename consensus to validation

* network: rename consensus in comments and logs

* Grumbles

* Rename tests consensus -> validation

* compute ingress and routing in polkadot runtime

* extract parent candidates from block when beginning consensus

* fetch incoming messages when validating

* fix consensus tests

* parachain wasm execution uses messages

* update parachain tests to check if messages are executed

* abstract out network service to make room for network tests

* skeleton for incoming data fetch

* collate ingress from consensus-gossip

* keep track of validated candidates in the shared-table

* add some shared_table tests for new behavior

* broadcast egress messages on gossip

* test compute_ingress

* move network tests to module folder

* dummy network for consensus-network tests

* make consensus network generic over executor

* test egress broadcast and ingress fetch

* fix test compilation

* address some grumbles

* address grumbles and fix parachain shuffle

* remove broadcast parameter from consensus network trait

* update to latest substrate - protocol API update

* nits

* remove the wait on a future

* use std mpsc for oneshot

* multiple consensus sessions in network

* tests compile, add a test for RecentSessionKeys

* track recently received session keys from validators

* add a test for desired key-sending behavior

* Fix up wasm runtime build

* Fixes for runtime

* Fix.

* More fixes

* Runtime builds on native.

* Native and wasm both build without warnings.

* Fix runtime tests.

* Merge #20

* Final fix for native runtime.

* Compile polkadot wo consensus

* Reverted changes to polkadot-consensus

* reintroduce minimal subset of consensus

* reintroduce checked_block to runtime for std

* polkadot_consensus compiles without most of the code

* remove checked_block again and do more checks in parachains for runtime

* uncomment proposer

* remove offline tracker

* extract out parachain-attestation logic from proposal directly

* reintroduce transaction_pool

* write some custom aura verification logic for the block verifier

* use transaction pool in more generic way

* service compiles again

* polkadot-network and tests pass

* remove unused session_key function from router

* everything but CLI compiles due to service hell

* Fixes compilation of `polkadot_cli`

* everything compiles

* update adder wasm

Gav Wood authored Sep 12, 2018 and Arkadiy Paronyan committed Sep 12, 2018

* Fixups for 646

* Fixes for API

* For for #678

* Fix runtime

* Update and build

* Tests build

* Fix tests

* Derive macro for simple structs.

* Derive Encode/Decode wherever we can.

* Derive for enums.

* Support discriminant.

* Get rid of some repetition.

* Support custom indices.

* Derive codec for enums wherever possible.

* Fix no_std

* WASM rebuilt

* Avoid excessive import.

* Fix compilation.

* Address review grumbles.

* update basic_add wasm

* wasm feature and collator feature

* move test parachains around a little

* fix wasm build for basic_add

* move basic_add to adder, introduce README

* minimal basic_add collator

* ensure collator messages are sent in the right order

* more logging

* route consensus statements to all peers

* minor bugfixes for parachains

* genesis builder accounts for parachain heads

* fix parachains tests

* targets for txpool

* tweak runtime + collator

* fix version in adder-collator

* consistency for overflowing

* adjust comment

* fix stable test run

* remove dummy registration test

* final grumbles

* Fix p2p

* Cosmetic stuff

* More cosmetics

* Whitespace

* Whitespace

* Whitespace

* Renames

* Most cosmetics

* typo

* minor rename

* Remote-end disconnect message should be info!

* invalid tab

* Avoid ignoring sustained bad pings

* Remove workarounds.

* Refactor out disable/disconnect peer to make API more declarative

* Minor fixes.

* rename `disconnect_peer` to `drop_peer` in low-level

* skeleton of collators object

* awaiting and handling collations. rename `collators` to CollationPool

* add some tests

* add tests

* implement Collators trait for ConsensusNetwork

* plug collators into main polkadot-network

* ignore collator role message

* add a couple more tests

* garbage collection for collations

* extract session-key tracking from consensus

* add local_collations.rs

* finish polish of local_collations

* integrate local_collations into network layer

* introduce API for adding local collations

* mostly finish collator implementation pending service fix

* Specialized network()

* push collations to the network

* grumbles

* substrate-service has custom configuration

* initialize network in collator mode as necessary

* Added some traces

* Gossip for all

* Fixed formatting

Arkadiy Paronyan authored Jul 16, 2018 and asynchronous rob committed Jul 16, 2018

* Use substrate codec for network messages

* Version bump

* Removed redundant format