* Update to latest Substrate master (#615)

* Update to latest Substrate master

* Remove unneeded patch + warning

* Update `Cargo.lock`

* Fix tests

* Update again

* Bump Substrate (#616)

* Update lock

* Fix

* Few fixes

* Bump to latest Substrate

* Fixes

* fix pre-tx-pool compilation

* more compilation fixes

* Updates for the injection period

- Liberal slash-refunding
- Instant unbonding

* *: Enable refactored authority discovery (#624)

* *: Enable authority discovery module

* *: List authority discovery id after parachain validator id

Make sure existing key types don't change their order by appending the
authority discovery id instead of injecting it between im online id and
parachain validator id.

* *: Gate authority discovery module behind feature flag

* cli/src/lib.rs: Fix warnings

* cli/src/lib.rs: Shorten line length

* Bump Substrate

* Bump Substrate

* Line widths

* Line widths again

* Revert bump.

* Update to latest Substrate master (#570)

* Bump substrate/version (#557)

* Bump version and Substrate (#560)

* Bump version and Substrate

* Bump version and Substrate

* Bump versions

* bump substrate to release specific v0.6.15

* Update lock

* Prepare Polkadot update for Substrate runtime interface 2.0 (#563)

* Prepare Polkadot update for Substrate runtime interface 2.0

* bump substrate to release specific v0.6.15

* Switch to `polkadot-master`

* Version bump

* Master backports

* Bump runtime

* Fix tests

* Fix tests

* Another fix.

* core logic for ICMP gossip

* refactor gossip to make more extension friendly

* move files aroun

* extract attestation-gossip logic to its own module

* message validation and broadcast logic

* fix upstream crates' compilation

* add a test

* another test for overlapping

* Some grammar and phrasing tweaks

Co-Authored-By: Luke Schoen <[email protected]>

* add since parameter to ingress runtime API

* broadcast out known unrouted message queues

* fix compilation of service and collator

* remove useless index_mapping

* some tests for icmp propagation

* fix decoding bug and test icmp queue validation

* simplify engine-id definition

Co-Authored-By: Sergei Pepyakin <[email protected]>

* address some grumbles

* some cleanup of old circulation code

* give network a handle to extrinsic store on startup

* an honest collator ensures data available as well

* address some grumbles

* add docs; rename the attestation session to "leaf work"

* module docs

* move gossip back to gossip.rs

* clean up and document attestation-gossip a bit

* some more docs on the availability store

* store all outgoing message queues in the availability store

* filter `Extrinsic` out of validation crate

* expunge Extrinsic from network

* expunge Extrinsic from erasure-coding

* expunge Extrinsic from collator

* expunge from adder-collator

* rename ExtrinsicStore to AvailabilityStore everywhere

* annotate and clean up message-routing tests

* Integrate srml/im-online

* Fix all build errors with old aura.

* Fix most of the build errors.

* Builds and tests seem to pass (I will not trust this commit yet)

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <[email protected]>

* Kill some warnings.

* fix panics on 0 validators

* Fix dev chain.

* Fix author stuff

* fix im online integration.

* Some tweaks

* Introduce app-crypto

* Initial build work

* codec update / tweaks

* patch polkadot-erasure-coding input

* More fixes for new crypto

* More fixes

* Update parachains module

* evamp parachain crypto

* More crypto work.

* Chain spec and service.

* ChainSpec stuff

* Last bits for a clean build

* Tweak coment

* adapt polkadot-validation to the new keystore

* polkadot-network compiles, but tests don't

* Integrate the new parachain validation stuff

* delete message_routing file

* make polkadot-network tests compile and pass

* runtime tests compile and pass

* update substrate ref

* service compiles

* all tests pass

* Add TODO, change branch back to polkadot-master

* Lock file

* TODOs done

* Issue number

* Remove old tODO

* Remove commented code

* Make `collator::Network` require `Send + Sync` to make it work

* Update packages

* Update to latest Substrate

* Make it compile and make tests work

* Use `polkadot-master`

* Fix CI

* Remove `build.sh` from readmes

* Delete old stuff

* Bring one back

* track unrouted ingress in runtime

* test ingress routing

* fix compilation

* add space

Co-Authored-By: Gavin Wood <[email protected]>

* Slots module

* Integrate slots

* More drafting

* Minor updates

* Update parachains to use trati

* More build fixes

* Full code now compiles

* Add renew bid function

* Implement calculate_winner

* Warning remove

* Update gitignore

* Test framework

* Tests

* Further testing

* More tests, new parameterisation.

* Fix and new test

* Thread-safe tests

* Test off-boarding and a fix.

* Test onboarding

* Allow late onboarding.

* Another test and fix

* Avoid println in nostd

* Compact representation of paraids

* Introduce documentation.

* Introduce events.

* Additional test and fix

* Additional test

* Tidy up line lengths.

* Remove printlns

* Use later substrate utils.

* Allow parachains to send messages.

* Fix build/test

* Make slots work with latest substrate

* Update runtime/src/slot_range.rs

Co-Authored-By: Robert Habermeier <[email protected]>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <[email protected]>

* Polish logic

* Rewind to earlier substrate master

* Remove dead code.

* Fix build

* Update substrate ref to master

* Update to new inherent digests API

* address grumbles

* fix

* Fix a warning.

* Reworded a comment.

* Check that receipt matches expectations

* Add test for final checks

* Split out queuing logic.

* Test final piece of queuing logic

* Fix up docs.

* More docs fixes

* strip out all ICMP code and begin gossip refactor

* validate incoming statements

* message_allowed logic

* compiles

* do reporting and neighbor packet validation

* tests compile

* propagate gossip messages

* test message_allowed

* some more tests

* address grumbles

thiolliere authored May 15, 2019 and asynchronous rob committed May 15, 2019

* WIP

* merging select_chain

* WIP

* update to point to gui-polkadot-master

* Fix collator

* update gui-polkadot-master and fix

* fix unwraps

* better returning an error

* positive updates to reputations when good things happen

* address review

* branch on good or bad pov blocks

* add a new constant for bad pov blocks

* update substrate reference

* bump wasm

* port polkadot_runtime and polkadot_validation

* update storages build (#245)

* all tests pass

* rebuild wasm

* validators expect collators to give them parachain messages

* mostly port network to use pov_block

* network tests pass

* verify ingress when fetching pov block

* fix runtime compilation

* all tests build

* fix some grumbles

* Update validation/src/collation.rs

Co-Authored-By: rphmeier <[email protected]>

* Update primitives/src/parachain.rs

Co-Authored-By: rphmeier <[email protected]>

* Update network/src/lib.rs

Co-Authored-By: rphmeier <[email protected]>

Bastian Köcher authored Mar 28, 2019 and asynchronous rob committed Mar 28, 2019

* Rebuild runtime

* Remove invalid value from chainspec (#68)

* service: use grandpa block import for locally sealed aura blocks (#85)

* bump version to v0.3.1

* Update lock file.

* limit number of transactions when building blocks (#91)

* Update to latest Substrate

* Bump to 0.3.2

* Actually bump.

* v0.3.2 (#98)

* bump substrate version

* fix polkadot-collator

* point to alexander-backports of substrate

* bump version

* cli: fix node shutdown (#100)

* update to latest substrate, change to v0.3.4

* update to latest substrate, bump version to 0.3.5

* v0.3.6

* try to build on every v0.3 commit and update alexander-backports

* bump to v0.3.7

* bump to 0.3.8

* Bump to 0.3.9: network and pruning improvements

* Bump to 0.3.10: reduce network bandwidth usage

* Use libp2p-kad 0.3.2 (#122)

* Bump libp2p-identify to 0.3.1 (#123)

* Bump to 0.3.12 (#127)

* Update Substrate again (#128)

* update substrate and bump version to v0.3.13

* bump version to v0.3.14: fix --reserved-nodes

* add a manually curated grandpa module (#136)

* updating v0.3 to use substrate v0.10 (#146)

* updating to latest substrate v0.10

* better handling of outer poll

* nit

* fix tests

* remove comment

* reduce indentation

* use self.poll

* bring oneshot into scope

* spaces

* wrap

* remove match

* wrap

* Update primitives/Cargo.toml

Co-Authored-By: gterzian <[email protected]>

* Update runtime/wasm/Cargo.toml

Co-Authored-By: gterzian <[email protected]>

* Update runtime/wasm/Cargo.toml

Co-Authored-By: gterzian <[email protected]>

* Update test-parachains/adder/collator/src/main.rs

Co-Authored-By: gterzian <[email protected]>

* indent

* add paranthese

* config: fix wrong ip for alexander bootnode (#161)

* fix curated-grandpa and rebuild wasm (#162)

* [v0.3] Integrates new gossip system into Polkadot (#166)

* new gossip validation in network

* integrate new gossip into service

* network: guard validation network future under exit signal (#168)

* bump version to v0.3.15: substrate v0.10

* [v0.3] update to substrate master (#175)

* update to substrate master

* fix test

* service: fix telemetry endpoints on alexander chainspec (#169) (#178)

* Update v0.3 to latest Substrate master (#177)

* update substrate v0.3 to latest master

* bump spec version

* update to latest master: remove fees module

* update runtime blobs

* bump version to 0.3.16

* replace sr25519 accountid with anysigner

* bump version to v0.3.17

* Some PoC-3 GRANDPA tweaks (#181)

* call on_finalise after triggering curated_grandpa change

* make grandpa rounds shorter for faster finalization

* use authorities when calculating duty roster (#185)

* [v0.3] Update to substrate master (#183)

* update to latest substrate master

* bump version to 0.3.18

* update to latest substrate master

* bump spec version

* update runtime wasm blobs

* remove current_offline_slash from chain spec

* update to substrate master: bump version to v0.3.19 (#188)

* update to substrate master: bump version to v0.3.19

libp2p network improvements

* network: replace NodeIndex with PeerId

* network: fix tests

* polkadot v0.3.20 (#190)

* update to substrate master: bump version to 0.3.20

* runtime: add offchain worker trait

* runtime: rebuild wasm blobs

* bump spec version (#191)

* Fix compilation

* Update version to 0.4.0

* Switch to use `polkadot-master` branch from substrate

* Remove unused struct

* Remove `grandpa::SyncedAuthorities` from `OnSessionChange`

* refactor out a consensus data fetcher from table router

* move statement checking logic into router

* refuse to start authority if collator

* support building the table router asynchronously

* instantiate_consensus does not overwrite old

* update key in new consensus if there was none before

* collator collects ingress from network

* test produced egress roots

* fix adder-collator compilation

* address first grumbles

* integrate new gossip with collator network launch

* address review

* Update to master

This introduces a new type `CollatorId`, currently just `SessionKey`
but which would forseeably change to its own thing. It seems to work
like this (despite there being a lot of the new-incompatible
`AccountId` replaced). No idea if it does anything sensible, though.

* Cleanups

* Fix tests

* Remove commented code

* Specify commit hash

* Remove commented code

* Correct version

* Update runtime/Cargo.toml

Co-Authored-By: gavofyork <[email protected]>

* PairT instead of _Pair

* Update lock file

* Remove rev causing upset

* Integrates new gossip system into Polkadot (#166)

* new gossip validation in network

* integrate new gossip into service

* Fix build

* Fix claims module

* fix warning

* update to latest master again

* update runtime

Andrew Jones authored Feb 21, 2019 and asynchronous rob committed Feb 21, 2019

* Initial rename of consensus -> validation

* Rename crate imports

* network: rename consensus to validation

* network: rename consensus in comments and logs

* Grumbles

* Rename tests consensus -> validation

* compute ingress and routing in polkadot runtime

* extract parent candidates from block when beginning consensus

* fetch incoming messages when validating

* fix consensus tests

* parachain wasm execution uses messages

* update parachain tests to check if messages are executed

* abstract out network service to make room for network tests

* skeleton for incoming data fetch

* collate ingress from consensus-gossip

* keep track of validated candidates in the shared-table

* add some shared_table tests for new behavior

* broadcast egress messages on gossip

* test compute_ingress

* move network tests to module folder

* dummy network for consensus-network tests

* make consensus network generic over executor

* test egress broadcast and ingress fetch

* fix test compilation

* address some grumbles

* address grumbles and fix parachain shuffle

* remove broadcast parameter from consensus network trait

* multiple consensus sessions in network

* tests compile, add a test for RecentSessionKeys

* track recently received session keys from validators

* add a test for desired key-sending behavior

* Fix up wasm runtime build

* Fixes for runtime

* Fix.

* More fixes

* Runtime builds on native.

* Native and wasm both build without warnings.

* Fix runtime tests.

* Merge #20

* Final fix for native runtime.

* Compile polkadot wo consensus

* Reverted changes to polkadot-consensus

* reintroduce minimal subset of consensus

* reintroduce checked_block to runtime for std

* polkadot_consensus compiles without most of the code

* remove checked_block again and do more checks in parachains for runtime

* uncomment proposer

* remove offline tracker

* extract out parachain-attestation logic from proposal directly

* reintroduce transaction_pool

* write some custom aura verification logic for the block verifier

* use transaction pool in more generic way

* service compiles again

* polkadot-network and tests pass

* remove unused session_key function from router

* everything but CLI compiles due to service hell

* Fixes compilation of `polkadot_cli`

* everything compiles

* update adder wasm

* Fix p2p

* Cosmetic stuff

* More cosmetics

* Whitespace

* Whitespace

* Whitespace

* Renames

* Most cosmetics

* typo

* minor rename

* Remote-end disconnect message should be info!

* invalid tab

* Avoid ignoring sustained bad pings

* Remove workarounds.

* Refactor out disable/disconnect peer to make API more declarative

* Minor fixes.

* rename `disconnect_peer` to `drop_peer` in low-level

* skeleton of collators object

* awaiting and handling collations. rename `collators` to CollationPool

* add some tests

* add tests

* implement Collators trait for ConsensusNetwork

* plug collators into main polkadot-network

* ignore collator role message

* add a couple more tests

* garbage collection for collations

* extract session-key tracking from consensus

* add local_collations.rs

* finish polish of local_collations

* integrate local_collations into network layer

* introduce API for adding local collations

* mostly finish collator implementation pending service fix

* Specialized network()

* push collations to the network

* grumbles

* substrate-service has custom configuration

* initialize network in collator mode as necessary

Arkadiy Paronyan authored Jul 16, 2018 and asynchronous rob committed Jul 16, 2018

* Use substrate codec for network messages

* Version bump

* Removed redundant format

* Fix warnings in libp2p

* Force a reason when you use the fatalist disable_peer

* Print more information

* Slightly more concise ref-fu

* Tracing for figuring out what's going into genesis

* Merge

* Fxi test

* Refactored Slicable

* Docs

* Wasm build

* Wasm build

* Renamed traits

* Review nits

* Renamed Slicable as well

* skeleton of collators object

* awaiting and handling collations. rename `collators` to CollationPool

* add some tests

* add tests

* implement Collators trait for ConsensusNetwork

* plug collators into main polkadot-network

* ignore collator role message

* add a couple more tests

* garbage collection for collations

* ensure disconnected backup collator is removed from pool

* address other grumbles

* arbitrary application logic in CLI

* collation work

* split up exit and work futures in application

* collation node workflow

* typo

* indentation fix

* doc grumbles

* rename Application to Worker

* refactor Worker::exit to exit_only

* dynamic inclusion threshold calculator

* collators interface

* collation helpers

* initial proposal-creation future

* create proposer when asked to propose

* remove local_availability duty

* statement table tracks includable parachain count

* beginnings of timing future

* finish proposal logic

* remove stray println

* extract shared table to separate module

* change ordering

* includability tracking

* fix doc

* initial changes to parachains module

* initialise dummy block before API calls

* give polkadot control over round proposer based on random seed

* propose only after enough candidates

* flesh out parachains module a bit more

* set_heads

* actually introduce set_heads to runtime

* update block_builder to accept parachains

* split block validity errors from real errors in evaluation

* update WASM runtimes

* polkadot-api methods for parachains additions

* delay evaluation until candidates are ready

* comments

* fix dynamic inclusion with zero initial

* test for includability tracker

* wasm validation of parachain candidates

* move primitives to primitives crate

* remove runtime-std dependency from codec

* adjust doc

* polkadot-parachain-primitives

* kill legacy polkadot-validator crate

* basic-add test chain

* test for basic_add parachain

* move to test-chains dir

* use wasm-build

* new wasm directory layout

* reorganize a bit more

* Fix for rh-minimal-parachain (#141)

* Remove extern "C"

We already encountered such behavior (bug?) in pwasm-std, I believe.

* Fix `panic_fmt` signature by adding `_col`

Wrong `panic_fmt` signature can inhibit some optimizations in LTO mode.

* Add linker flags and use wasm-gc in build script

Pass --import-memory to LLD to emit wasm binary with imported memory.

Also use wasm-gc instead of wasm-build.

* Fix effective_max.

I'm not sure why it was the way it was actually.

* Recompile wasm.

* Fix indent

* more basic_add tests

* validate parachain WASM

* produce statements on receiving statements

* tests for reactive statement production

* fix build

* add OOM lang item to runtime-io

* use dynamic_inclusion when evaluating as well

* fix update_includable_count

* remove dead code

* grumbles

* actually defer round_proposer logic

* update wasm

* address a few more grumbles

* schedule collation work as soon as BFT is started

* impl future in collator

* fix comment

* governance proposals for adding and removing parachains

* bump protocol version

* tear out polkadot-specific pieces of substrate-network

* extract out polkadot-specific stuff from substrate-network

* begin polkadot network subsystem

* grumbles

* update WASM checkins

* parse status from polkadot peer

* allow invoke of network specialization

* begin statement router implementation

* remove dependency on tokio-timer

* fix sanity check and have proposer factory create communication streams

* pull out statement routing from consensus library

* fix comments

* adjust typedefs

* extract consensus_gossip out of main network protocol handler

* port substrate-bft to new tokio

* port polkadot-consensus to new tokio

* fix typo

* start message processing task

* initial consensus network implementation

* remove known tracking from statement-table crate

* extract router into separate module

* defer statements until later

* double signature is invalid

* propagating statements

* grumbles

* request block data

* fix compilation

* embed new consensus network into service

* port demo CLI to tokio

* all test crates compile

* some tests for fetching block data

* whitespace

* adjusting some tokio stuff

* update exit-future

* remove overly noisy warning

* clean up collation work a bit

* address review grumbles

* fix lock order in protocol handler

* rebuild wasm artifacts

* tag AuthorityId::from_slice for std only

* address formatting grumbles

* rename event_loop to executor

* some more docs for polkadot-network crate