Skip to content
  1. Jul 16, 2020
  3. Jul 15, 2020
    • s3krit's avatar
      [CI] check_runtime.sh redux (#1390) · 756b4380
      s3krit authored 
      * Initial check_runtime.sh redux

* Skip further checks if no change to runtime

* remove polkadot-master reference

* Fix issues found during review :)

* replace static common dir with common_dirs array
      756b4380
  5. Jul 14, 2020
    • Gavin Wood's avatar
      Bump Substrate & versions (#1406) · 168c0922
      Gavin Wood authored
      168c0922
    • Peter Goodspeed-Niklaus's avatar
      mod subsystem-util (#1376) · d7684115
      Peter Goodspeed-Niklaus authored 
      * Add subsystem-util crate.

Start by moving the JobCanceler here.

* copy utility functions for requesting runtime data; generalize

* convert subsystem-util from crate to module in subsystem

The point of making a sub-crate is to ensure that only the necessary
parts of a program get compiled; if a dependent package needed only
subsystem-util, and not subsystem, then subsystem wouldn't need to
be compiled.

However, that will never happen: subsystem-util depends on
subsystem::messages, so subsystem will always be compiled.

Therefore, it makes more sense to add it as a module in the existing
crate than as a new and distinct crate.

* make runtime request sender type generic

* candidate backing subsystem uses util for api requests

* add struct Validator representing the local validator

This struct can be constructed when the local node is a validator;
the constructor fails otherwise. It stores a bit of local data, and
provides some utility methods.

* add alternate constructor for better efficiency

* refactor candidate backing to use utility methods

* fix test breakage caused by reordering tests

* restore test which accidentally got deleted during merge

* start extracting jobs management into helper traits + structs

* use util::{JobHandle, Jobs} in CandidateBackingSubsystem

* implement generic job-manager subsystem impl

This means that the work of implementing a subsystem boils down
to implementing the job, and then writing an appropriate
type definition, i.e.

pub type CandidateBackingSubsystem<Spawner, Context> =
	util::JobManager<Spawner, Context, CandidateBackingJob>;

* add hash-extraction helper to messages

* fix errors caused by improper rebase

* doc improvement

* simplify conversion from overseer communication to job message

* document fn hash for all messages

* rename fn hash() -> fn relay_parent

* gracefully shut down running futures on Conclude

* ensure we're validating with the proper validator index

* rename: handle_unhashed_msg -> handle_orphan_msg

* impl Stream for Jobs<Spawner, Job>

This turns out to be relatively complicated and requires some
unsafe code, so we'll want either detailed review, or to choose
to revert this commit.

* add missing documentation for public items

* use pin-project to eliminate unsafe code from this codebase

* rename SenderMessage -> FromJob

* reenvision the subsystem requests as an extension trait

This works within `util.rs`, but fails in `core/backing/src/lib.rs`,
because we don't actually create the struct soon enough. Continuing
down this path would imply substantial rewriting.

* Revert "reenvision the subsystem requests as an extension trait"

This reverts commit a5639e36.

The fact is, the new API is more complicated to no real benefit.

* apply suggested futuresunordered join_all impl

* CandidateValidationMessage variants have no top-level relay parents

* rename handle_orphan_msg -> handle_unanchored_msg

* make most node-core-backing types private

Now the only public types exposed in that module are
CandidateBackingSubsystem and ToJob. While ideally we could reduce
the public interface to only the former type, that doesn't work
because ToJob appears in the public interface of CandidateBackingSubsystem.

This also involves changing the definition of CandidateBackingSubsystem;
it is no longer a typedef, but a struct wrapping the job manager.
      d7684115
    • gabriel klawitter's avatar
      ci: fix subdirectory upload to s3 (#1398) · 587c6858
      gabriel klawitter authored
      587c6858
  7. Jul 13, 2020
    • Max Inden's avatar
      *: Enable authority discovery by default (#1395) · bc6e1e77
      Max Inden authored 
      * *: Enable authority discovery by default

Instead of having to explicitly enable the authority discovery module on
validator and sentry nodes, this commit enables the module by default.

Today there is no way for non validator or sentry nodes to run the
module. That might change in the future.

* service/src/lib: Fix typo in new_full! for test
      bc6e1e77
    • Gavin Wood's avatar
      Tweak poll end block (#1397) · d0e553fe
      Gavin Wood authored
      v0.8.14
      d0e553fe
  9. Jul 11, 2020
  11. Jul 10, 2020
    • Bastian Köcher's avatar
      Revert log rotation (#1389) · feb5039e
      Bastian Köcher authored 
      * Revert log rotation

https://github.com/paritytech/substrate/pull/6627

* 'Update substrate'

Co-authored-by: parity-processbot <>
      feb5039e
    • Peter Goodspeed-Niklaus's avatar
      add more detail to provisioner block production section (#1370) · 64d0c7ae
      Peter Goodspeed-Niklaus authored 
      Per https://github.com/paritytech/polkadot/commit/69ce9ff#r40392855
      64d0c7ae
    • Joseph Mark's avatar
      Add Process.toml (#1361) · 9f1016d9
      Joseph Mark authored 
      

* Add Process.toml

* Apply suggestions from code review

Co-authored-by: default avatarRobert Habermeier <[email protected]>
      9f1016d9
    • asynchronous rob's avatar
      Refactor primitives (#1383) · 96af6ead
      asynchronous rob authored 
      * create a v1 primitives module

* Improve guide on availability types

* punctuate

* new parachains runtime uses new primitives

* tests of new runtime now use new primitives

* add ErasureChunk to guide

* export erasure chunk from v1 primitives

* subsystem crate uses v1 primitives

* node-primitives uses new v1 primitives

* port overseer to new primitives

* new-proposer uses v1 primitives (no ParachainHost anymore)

* fix no-std compilation for primitives

* service-new uses v1 primitives

* network-bridge uses new primitives

* statement distribution uses v1 primitives

* PoV distribution uses v1 primitives; add PoV::hash fn

* move parachain to v0

* remove inclusion_inherent module and place into v1

* remove everything from primitives crate root

* remove some unused old types from v0 primitives

* point everything else at primitives::v0

* squanch some warns up

* add RuntimeDebug import to no-std as well

* port over statement-table and validation

* fix final errors in validation and node-primitives

* add dummy Ord impl to committed candidate receipt

* guide: update CandidateValidationMessage

* add primitive for validationoutputs

* expand CandidateValidationMessage further

* bikeshed

* add some impls to omitted-validation-data and available-data

* expand CandidateValidationMessage

* make erasure-coding generic over v1/v0

* update usages of erasure-coding

* implement commitments.hash()

* use Arc<Pov> for CandidateValidation

* improve new erasure-coding method names

* fix up candidate backing

* update docs a bit

* fix most tests and add short-circuiting to make_pov_available

* fix remainder of candidate backing tests

* squanching warns

* squanch it up

* some fallout

* overseer fallout

* free from polkadot-test-service hell
      96af6ead
  13. Jul 09, 2020
  15. Jul 08, 2020
  17. Jul 07, 2020
  19. Jul 06, 2020
    • asynchronous rob's avatar
      Implement the Statement Distribution Subsystem (#1326) · 8348cc4c
      asynchronous rob authored 
      * set up data types and control flow for statement distribution

* add some set-like methods to View

* implement sending to peers

* start fixing equivocation handling

* Add a section to the statement distribution subsystem on equivocations and flood protection

* fix typo and amend wording

* implement flood protection

* have peer knowledge tracker follow when peer first learns about a candidate

* send dependents after circulating

* add another TODO

* trigger send in one more place

* refactors from review

* send new statements to candidate backing

* instantiate active head data with runtime API values

* track our view changes and peer view changes

* apply a benefit to peers who send us statements we want

* remove unneeded TODO

* add some comments and improve Hash implementation

* start tests and fix `note_statement`

* test active_head seconding logic

* test that the per-peer tracking logic works

* test per-peer knowledge tracker

* test that peer view updates lead to messages being sent

* test statement circulation

* address review comments

* have view set methods return references
      8348cc4c
    • Bastian Köcher's avatar
      Companion for #6576 (#1351) · 0499212f
      Bastian Köcher authored 
      * Companion for #6576

* 'Update substrate'

Co-authored-by: parity-processbot <>
      0499212f
    • Shawn Tabrizi's avatar
      Companion for #6567 (Benchmark Writer CLI) (#1349) · 7afb386d
      Shawn Tabrizi authored 
      * Update usage of macro

* 'Update substrate'

* cargo update -p sp-io

Co-authored-by: parity-processbot <>
      7afb386d
  21. Jul 05, 2020
    • Peter Goodspeed-Niklaus's avatar
      implement custom proposer (#1320) · 69ce9ff3
      Peter Goodspeed-Niklaus authored 
      

* network bridge skeleton

* move some primitives around and add debug impls

* protocol registration glue & abstract network interface

* add send_msgs to subsystemctx

* select logic

* transform different events into actions and handle

* implement remaining network bridge state machine

* start test skeleton

* make network methods asynchronous

* extract subsystem out to subsystem crate

* port over overseer to subsystem context trait

* fix minimal example

* fix overseer doc test

* update network-bridge crate

* write a subsystem test-helpers crate

* write a network test helper for network-bridge

* set up (broken) view test

* Revamp network to be more async-friendly and not require Sync

* fix spacing

* fix test compilation

* insert side-channel for actions

* Add some more message types to AllMessages

* introduce a test harness

* impl ProvideInherent for InclusionInherent

* reduce import churn; correct expect message

* move inclusion inherent identifier into primitives

It's not clear precisely why this is desired, but it's a pattern
I've seen in several places, so I'm going this to be on the
safe side. Worst case, we can revert this commit pretty easily.

* bump kusama spec_version to placate CI

* copy sc_basic_authorship::{ProposerFactory, Proposer}

We have from the problem description:

> This Proposer will require an OverseerHandle to make requests via.

That's next on the plate.

* use polkadot custom proposer instead of basic-authorship one

* add some tests

* ensure service compiles and passes tests

* fix typo

* fix service-new compilation

* Subsystem test helpers send messages synchronously

* remove smelly action inspector

* remove superfluous let binding

* fix warnings

* add license header

* empty commit; maybe github will notice the one with changes

* Update node/network/bridge/src/lib.rs

Co-authored-by: default avatarPeter Goodspeed-Niklaus <[email protected]>

* add sanity check to only include valid inherents

* stub: encapsulate block production mechanics instead of copying them

The goal is to end up with something like what's in
validation::block_production::*, which encapsulates
basic block production mechanics. This is a better idea than
just straight-up copying those mechanics.

* partial implementation of propose fn

Doesn't actually compile yet; need to bring in some other
commits to ensure ProvisionerMessage is a thing, and also
figure out how to get the block hash given the current
context.

* fix compilation

* clear a few more compile errors

* finish fn propose

* broken: add timeout to proposal

* add timeout to proposal

* guide: provisioner is responsible for selecting parachain candidates

* implement ProvisionerMessage::RequestInherentData & update fn propose

* impl CreateProposer::init; clean up

* impl std::error::Error for Error

* document error-handling rationale

* cause polkadot-service-new to compile correctly

* Move potentially-blocking call from fn init -> fn propose

This means that we can wrap the delayed call into the same
timeout check used elsewhere.

* document struct Proposer

* extract provisioner data fetch

This satisfies two requirements:

- only applies the timeout to actually fetching the provisioner data,
  not to constructing the block after
- simplifies the problem of injecting default data if we could not
  get the real provisioner data in time.

Co-authored-by: default avatarRobert Habermeier <[email protected]>
Co-authored-by: default avatarGavin Wood <[email protected]>
      69ce9ff3
  23. Jul 04, 2020
    • André Silva's avatar
      babe: enable equivocation reporting on all runtimes (#1330) · 78e6e08e
      André Silva authored 
      

* enable BABE equivocation reporting on all runtimes

* runtime: fix parachains mock test runtime

* cargo update -p sp-io

Co-authored-by: default avatarShawn Tabrizi <[email protected]>
      78e6e08e
    • Bastian Köcher's avatar
      Make a collator send a collation as backup as well (#1353) · 21f31f78
      Bastian Köcher authored 
      Currently a collator will only send a collation to validators it is a
primary for. While testing this could lead to the situation that the
same collator was registered as prime for all Parachain validators but
failed for other reasons to generate a PoVBlock. However no other
collator was sending a collation, which stopped the Parachain until the
faulty collator was stopped.

This pr solves this problem by making sure that every collator sends a
collation to one of his validators he is connected to, but registered as backup.
      21f31f78
  25. Jul 03, 2020