1. 10 Mar, 2021 2 commits
  2. 09 Mar, 2021 1 commit
  3. 26 Feb, 2021 1 commit
    • Robert Klotzner's avatar
      Request based availability distribution (#2423) · 950447e1
      Robert Klotzner authored
      * WIP
      
      * availability distribution, still very wip.
      
      Work on the requesting side of things.
      
      * Some docs on what I intend to do.
      
      * Checkpoint of session cache implementation
      
      as I will likely replace it with something smarter.
      
      * More work, mostly on cache
      
      and getting things to type check.
      
      * Only derive MallocSizeOf and Debug for std.
      
      * availability-distribution: Cache feature complete.
      
      * Sketch out logic in `FetchTask` for actual fetching.
      
      - Compile fixes.
      - Cleanup.
      
      * Format cleanup.
      
      * More format fixes.
      
      * Almost feature complete `fetch_task`.
      
      Missing:
      
      - Check for cancel
      - Actual querying of peer ids.
      
      * Finish FetchTask so far.
      
      * Directly use AuthorityDiscoveryId in protocol and cache.
      
      * Resolve `AuthorityDiscoveryId` on sending requests.
      
      * Rework fetch_task
      
      - also make it impossible to check the wrong chunk index.
      - Export needed function in validator_discovery.
      
      * From<u32> implementation for `ValidatorIndex`.
      
      * Fixes and more integration work.
      
      * Make session cache proper lru cache.
      
      * Use proper lru cache.
      
      * Requester finished.
      
      * ProtocolState -> Requester
      
      Also make sure to not fetch our own chunk.
      
      * Cleanup + fixes.
      
      * Remove unused functions
      
      - FetchTask::is_finished
      - SessionCache::fetch_session_info
      
      * availability-distribution responding side.
      
      * Cleanup + Fixes.
      
      * More fixes.
      
      * More fixes.
      
      adder-collator is running!
      
      * Some docs.
      
      * Docs.
      
      * Fix reporting of bad guys.
      
      * Fix tests
      
      * Make all tests compile.
      
      * Fix test.
      
      * Cleanup + get rid of some warnings.
      
      * state -> requester
      
      * Mostly doc fixes.
      
      * Fix test suite.
      
      * Get rid of now redundant message types.
      
      * WIP
      
      * Rob's review remarks.
      
      * Fix test suite.
      
      * core.relay_parent -> leaf for session request.
      
      * Style fix.
      
      * Decrease request timeout.
      
      * Cleanup obsolete errors.
      
      * Metrics + don't fail on non fatal errors.
      
      * requester.rs -> requester/mod.rs
      
      * Panic on invalid BadValidator report.
      
      * Fix indentation.
      
      * Use typed default timeout constant.
      
      * Make channel size 0, as each sender gets one slot anyways.
      
      * Fix incorrect metrics initialization.
      
      * Fix build after merge.
      
      * More fixes.
      
      * Hopefully valid metrics names.
      
      * Better metrics names.
      
      * Some tests that already work.
      
      * Slightly better docs.
      
      * Some more tests.
      
      * Fix network bridge test.
      950447e1
  4. 23 Feb, 2021 1 commit
  5. 22 Feb, 2021 1 commit
  6. 19 Feb, 2021 1 commit
    • Bernhard Schuster's avatar
      feat/jaeger: more spans, more stages (#2477) · f9186eb2
      Bernhard Schuster authored
      * feat/jaeger: more spans, more stages
      
      Stage numbers are still arbitrarily picked.
      
      * feat/jaeger: additional spans
      
      * chore/spellcheck: improve the dictionary
      
      * fix/jaeger JaegerSpan -> jaeger::Span
      f9186eb2
  7. 18 Feb, 2021 1 commit
  8. 17 Feb, 2021 1 commit
    • Bernhard Schuster's avatar
      refactor/reputation: unify the values used (#2462) · 8c1b2e87
      Bernhard Schuster authored
      * refactor/reputation: unify the values used
      
      * chore/rep: rename Annoy* to Cost*, make duplicate message Cost*Repeated
      
      * fix/reputation: lost and found, convert at the boundary to substrate
      
      * refactor/rep: move conversion to base reputation one level down, left conversions
      
      * fix/rep: order of magnitude adjustments
      
      Thanks pierre!
      
      * remove spaces
      
      * chore/rep: give rationale for order of magnitude
      
      * refactor/rep: move UnifiedReputationChange to separate file
      
      * fix/rep: order of magnitudes correction
      8c1b2e87
  9. 14 Feb, 2021 1 commit
    • Bastian Köcher's avatar
      Notify collators about seconded collation (#2430) · ac5ef00e
      Bastian Köcher authored
      * Notify collators about seconded collation
      
      This pr adds functionality to inform a collator that its collation was
      seconded by a parachain validator. Before this signed statement was only
      gossiped over the validation substream. Now, we explicitly send the
      seconded statement to the collator after it was validated successfully.
      
      Besides that it changes the `CollatorFn` to return an optional result
      sender that is informed when the build collation was seconded by a
      parachain validator.
      
      * Add test
      
      * Make sure we only send `Seconded` statements
      
      * Make sure we only receive valid statements
      
      * Review feedback
      ac5ef00e
  10. 03 Feb, 2021 1 commit
    • Robert Klotzner's avatar
      Generic request/response infrastructure for Polkadot (#2352) · ecc3772d
      Robert Klotzner authored
      * Move NetworkBridgeEvent to subsystem::messages.
      
      It is not protocol related at all, it is in fact only part of the
      subsystem communication as it gets wrapped into messages of each
      subsystem.
      
      * Request/response infrastructure is taking shape.
      
      WIP: Does not compile.
      
      * Multiplexer variant not supported by Rusts type system.
      
      * request_response::request type checks.
      
      * Cleanup.
      
      * Minor fixes for request_response.
      
      * Implement request sending + move multiplexer.
      
      Request multiplexer is moved to bridge, as there the implementation is
      more straight forward as we can specialize on `AllMessages` for the
      multiplexing target.
      
      Sending of requests is mostly complete, apart from a few `From`
      instances. Receiving is also almost done, initializtion needs to be
      fixed and the multiplexer needs to be invoked.
      
      * Remove obsolete multiplexer.
      
      * Initialize bridge with multiplexer.
      
      * Finish generic request sending/receiving.
      
      Subsystems are now able to receive and send requests and responses via
      the overseer.
      
      * Doc update.
      
      * Fixes.
      
      * Link issue for not yet implemented code.
      
      * Fixes suggested by @ordian
      
       - thanks!
      
      - start encoding at 0
      - don't crash on zero protocols
      - don't panic on not yet implemented request handling
      
      * Update node/network/protocol/src/request_response/v1.rs
      
      Use index 0 instead of 1.
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      
      * Update node/network/protocol/src/request_response.rs
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      
      * Fix existing tests.
      
      * Better avoidance of division by zoro errors.
      
      * Doc fixes.
      
      * send_request -> start_request.
      
      * Fix missing renamings.
      
      * Update substrate.
      
      * Pass TryConnect instead of true.
      
      * Actually import `IfDisconnected`.
      
      * Fix wrong import.
      
      * Update node/network/bridge/src/lib.rs
      
      typo
      
      Co-authored-by: Pierre Krieger's avatarPierre Krieger <pierre.krieger1708@gmail.com>
      
      * Update node/network/bridge/src/multiplexer.rs
      
      Remove redundant import.
      
      Co-authored-by: Pierre Krieger's avatarPierre Krieger <pierre.krieger1708@gmail.com>
      
      * Stop doing tracing from within `From` instance.
      
      Thanks for the catch @tomaka
      
      !
      
      * Get rid of redundant import.
      
      * Formatting cleanup.
      
      * Fix tests.
      
      * Add link to issue.
      
      * Clarify comments some more.
      
      * Fix tests.
      
      * Formatting fix.
      
      * tabs
      
      * Fix link
      
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      
      * Use map_err.
      
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      
      * Improvements inspired by suggestions by @drahnr.
      
      - Channel size is now determined by function.
      - Explicitely scope NetworkService::start_request.
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      Co-authored-by: Pierre Krieger's avatarPierre Krieger <pierre.krieger1708@gmail.com>
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      ecc3772d
  11. 05 Jan, 2021 1 commit
  12. 04 Jan, 2021 1 commit
    • asynchronous rob's avatar
      Improve Network Spans (#2169) · a891884e
      asynchronous rob authored
      * utility functions for erasure-coding threshold
      
      * add candidate-hash tag to candidate jaeger spans
      
      * debug implementation for jaeger span
      
      * add a span to each live candidate in availability dist.
      
      * availability span covers only our piece
      
      * fix tests
      
      * keep span alive slightly longer
      
      * remove spammy bitfield-gossip-received log
      
      * Revert "remove spammy bitfield-gossip-received log"
      
      This reverts commit 831a2db5
      
      .
      
      * add claimed validator to bitfield-gossip span
      
      * add peer-id to handle-incoming span
      
      * add peer-id to availability distribution span
      
      * Update node/network/availability-distribution/src/lib.rs
      
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      
      * Update erasure-coding/src/lib.rs
      
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      
      * Update node/subsystem/src/jaeger.rs
      
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      a891884e
  13. 20 Dec, 2020 1 commit
  14. 17 Dec, 2020 1 commit
    • Andronik Ordian's avatar
      refactor View to include finalized_number (#2128) · c429e15c
      Andronik Ordian authored
      * refactor View to include finalized_number
      
      * guide: update the NetworkBridge on BlockFinalized
      
      * av-store: fix the tests
      
      * actually fix tests
      
      * grumbles
      
      * ignore macro doctest
      
      * use Hash::repeat_bytes more consistently
      
      * broadcast empty leaves updates as well
      
      * fix issuing view updates on empty leaves updates
      c429e15c
  15. 11 Dec, 2020 1 commit
  16. 26 Nov, 2020 1 commit
  17. 24 Nov, 2020 1 commit
  18. 23 Nov, 2020 1 commit
  19. 20 Nov, 2020 2 commits
    • Peter Goodspeed-Niklaus's avatar
      Add Prometheus timers to the subsystems (#1923) · e655654e
      Peter Goodspeed-Niklaus authored
      * reexport prometheus-super for ease of use of other subsystems
      
      * add some prometheus timers for collation generation subsystem
      
      * add timing metrics to av-store
      
      * add metrics to candidate backing
      
      * add timing metric to bitfield signing
      
      * add timing metrics to candidate selection
      
      * add timing metrics to candidate-validation
      
      * add timing metrics to chain-api
      
      * add timing metrics to provisioner
      
      * add timing metrics to runtime-api
      
      * add timing metrics to availability-distribution
      
      * add timing metrics to bitfield-distribution
      
      * add timing metrics to collator protocol: collator side
      
      * add timing metrics to collator protocol: validator side
      
      * fix candidate validation test failures
      
      * add timing metrics to pov distribution
      
      * add timing metrics to statement-distribution
      
      * use substrate_prometheus_endpoint prometheus reexport instead of prometheus_super
      
      * don't include JOB_DELAY in bitfield-signing metrics
      
      * give adder-collator ability to easily export its genesis-state and validation code
      
      * wip: adder-collator pushbutton script
      
      * don't attempt to register the adder-collator automatically
      
      Instead, get these values with
      
      ```sh
      target/release/adder-collator export-genesis-state
      target/release/adder-collator export-genesis-wasm
      ```
      
      And then register the parachain on https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944#/explorer
      
      To collect prometheus data, after running the script, create `prometheus.yml` per the instructions
      at https://www.notion.so/paritytechnologies/Setting-up-Prometheus-locally-835cb3a9df7541a781c381006252b5ff
      
      
      and then run:
      
      ```sh
      docker run -v `pwd`/prometheus.yml:/etc/prometheus/prometheus.yml:z --network host prom/prometheus
      ```
      
      Demonstrates that data makes it across to prometheus, though it is likely to be useful in the future
      to tweak the buckets.
      
      * Update parachain/test-parachains/adder/collator/src/cli.rs
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      
      * use the grandpa-pause parameter
      
      * skip metrics in tracing instrumentation
      
      * remove unnecessary grandpa_pause cli param
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      e655654e
    • Peter Goodspeed-Niklaus's avatar
      Add tracing support to node (#1940) · f7ea3d07
      Peter Goodspeed-Niklaus authored
      * drop in tracing to replace log
      
      * add structured logging to trace messages
      
      * add structured logging to debug messages
      
      * add structured logging to info messages
      
      * add structured logging to warn messages
      
      * add structured logging to error messages
      
      * normalize spacing and Display vs Debug
      
      * add instrumentation to the various 'fn run'
      
      * use explicit tracing module throughout
      
      * fix availability distribution test
      
      * don't double-print errors
      
      * remove further redundancy from logs
      
      * fix test errors
      
      * fix more test errors
      
      * remove unused kv_log_macro
      
      * fix unused variable
      
      * add tracing spans to collation generation
      
      * add tracing spans to av-store
      
      * add tracing spans to backing
      
      * add tracing spans to bitfield-signing
      
      * add tracing spans to candidate-selection
      
      * add tracing spans to candidate-validation
      
      * add tracing spans to chain-api
      
      * add tracing spans to provisioner
      
      * add tracing spans to runtime-api
      
      * add tracing spans to availability-distribution
      
      * add tracing spans to bitfield-distribution
      
      * add tracing spans to network-bridge
      
      * add tracing spans to collator-protocol
      
      * add tracing spans to pov-distribution
      
      * add tracing spans to statement-distribution
      
      * add tracing spans to overseer
      
      * cleanup
      f7ea3d07
  20. 05 Nov, 2020 1 commit
  21. 28 Oct, 2020 2 commits
    • Fedor Sakharov's avatar
      Change SpawnedSubsystem type to log subsystem errors (#1878) · d34585dd
      Fedor Sakharov authored
      * Change SpawnedSubsystem type to log subsystem errors
      
      * Remove clone
      d34585dd
    • Peter Goodspeed-Niklaus's avatar
      start working on building the real overseer (#1795) · 798f781f
      Peter Goodspeed-Niklaus authored
      
      
      * start working on building the real overseer
      
      Unfortunately, this fails to compile right now due to an upstream
      failure to compile which is probably brought on by a recent upgrade
      to rustc v1.47.
      
      * fill in AllSubsystems internal constructors
      
      * replace fn make_metrics with Metrics::attempt_to_register
      
      * update to account for #1740
      
      * remove Metrics::register, rename Metrics::attempt_to_register
      
      * add 'static bounds to real_overseer type params
      
      * pass authority_discovery and network_service to real_overseer
      
      It's not straightforwardly obvious that this is the best way to handle
      the case when there is no authority discovery service, but it seems
      to be the best option available at the moment.
      
      * select a proper database configuration for the availability store db
      
      * use subdirectory for av-store database path
      
      * apply Basti's patch which avoids needing to parameterize everything on Block
      
      * simplify path extraction
      
      * get all tests to compile
      
      * Fix Prometheus double-registry error
      
      for debugging purposes, added this to node/subsystem-util/src/lib.rs:472-476:
      
      ```rust
      Some(registry) => Self::try_register(registry).map_err(|err| {
      	eprintln!("PrometheusError calling {}::register: {:?}", std::any::type_name::<Self>(), err);
      	err
      }),
      ```
      
      That pointed out where the registration was failing, which led to
      this fix. The test still doesn't pass, but it now fails in a new
      and different way!
      
      * authorities must have authority discovery, but not necessarily overseer handlers
      
      * fix broken SpawnedSubsystem impls
      
      detailed logging determined that using the `Box::new` style of
      future generation, the `self.run` method was never being called,
      leading to dropped receivers / closed senders for those subsystems,
      causing the overseer to shut down immediately.
      
      This is not the final fix needed to get things working properly,
      but it's a good start.
      
      * use prometheus properly
      
      Prometheus lets us register simple counters, which aren't very
      interesting. It also allows us to register CounterVecs, which are.
      With a CounterVec, you can provide a set of labels, which can
      later be used to filter the counts.
      
      We were using them wrong, though. This pattern was repeated in a
      variety of places in the code:
      
      ```rust
      // panics with an cardinality mismatch
      let my_counter = register(CounterVec::new(opts, &["succeeded", "failed"])?, registry)?;
      my_counter.with_label_values(&["succeeded"]).inc()
      ```
      
      The problem is that the labels provided in the constructor are not
      the set of legal values which can be annotated, but a set of individual
      label names which can have individual, arbitrary values.
      
      This commit fixes that.
      
      * get av-store subsystem to actually run properly and not die on first signal
      
      * typo fix: incomming -> incoming
      
      * don't disable authority discovery in test nodes
      
      * Fix rococo-v1 missing session keys
      
      * Update node/core/av-store/Cargo.toml
      
      * try dummying out av-store on non-full-nodes
      
      * overseer and subsystems are required only for full nodes
      
      * Reduce the amount of warnings on browser target
      
      * Fix two more warnings
      
      * InclusionInherent should actually have an Inherent module on rococo
      
      * Ancestry: don't return genesis' parent hash
      
      * Update Cargo.lock
      
      * fix broken test
      
      * update test script: specify chainspec as script argument
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Update node/service/src/lib.rs
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * node/service/src/lib: Return error via ? operator
      
      * post-merge blues
      
      * add is_collator flag
      
      * prevent occasional av-store test panic
      
      * simplify fix; expand application
      
      * run authority_discovery in Role::Discover when collating
      
      * distinguish between proposer closed channel errors
      
      * add IsCollator enum, remove is_collator CLI flag
      
      * improve formatting
      
      * remove nop loop
      
      * Fix some stuff
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      Co-authored-by: Bastian Köcher's avatarBastian Köcher <git@kchr.de>
      Co-authored-by: Fedor Sakharov's avatarFedor Sakharov <fedor.sakharov@gmail.com>
      Co-authored-by: default avatarRobert Habermeier <robert@Roberts-MBP.lan1>
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      Co-authored-by: default avatarMax Inden <mail@max-inden.de>
      798f781f
  22. 27 Oct, 2020 1 commit
  23. 09 Oct, 2020 1 commit
    • Rakan Alhneiti's avatar
      Update to work with async keystore – Companion PR for #7000 (#1740) · bc7d1322
      Rakan Alhneiti authored
      
      
      * Fix keystore types
      
      * Use SyncCryptoStorePtr
      
      * Borrow keystore
      
      * Fix unused imports
      
      * Fix polkadot service
      
      * Fix bitfield-distribution tests
      
      * Fix indentation
      
      * Fix backing tests
      
      * Fix tests
      
      * Fix provisioner tests
      
      * Removed SyncCryptoStorePtr
      
      * Fix services
      
      * Address PR feedback
      
      * Address PR feedback - 2
      
      * Update CryptoStorePtr imports to be from sp_keystore
      
      * Typo
      
      * Fix CryptoStore import
      
      * Document the reason behind using filesystem keystore
      
      * Remove VALIDATORS
      
      * Fix duplicate dependency
      
      * Mark sp-keystore as optional
      
      * Fix availability distribution
      
      * Fix call to sign_with
      
      * Fix keystore usage
      
      * Remove tokio and fix parachains Cargo config
      
      * Typos
      
      * Fix keystore dereferencing
      
      * Fix CryptoStore import
      
      * Fix provisioner
      
      * Fix node backing
      
      * Update services
      
      * Cleanup dependencies
      
      * Use sync_keystore
      
      * Fix node service
      
      * Fix node service - 2
      
      * Fix node service - 3
      
      * Rename CryptoStorePtr to SyncCryptoStorePtr
      
      * "Update Substrate"
      
      * Apply suggestions from code review
      
      * Update node/core/backing/Cargo.toml
      
      * Update primitives/src/v0.rs
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Fix wasm build
      
      * Update Cargo.lock
      
      Co-authored-by: parity-processbot <>
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      bc7d1322
  24. 01 Oct, 2020 1 commit
    • Andronik Ordian's avatar
      implement remaining subsystem metrics (#1770) · 5bb296bf
      Andronik Ordian authored
      * overseer metrics: messages relayed
      
      * provisioner metrics: cosmetic changes
      
      * candidate selection metrics: cosmetic changes
      
      * availability bitfields metrics
      
      * availability distribution metrics
      
      * PoV distribution metrics
      
      * statement-distribution: small simplification
      
      * statement-distribution: extract log target into a const
      
      * statement-distribution: metrics
      
      * address review nits
      5bb296bf
  25. 29 Sep, 2020 1 commit
  26. 28 Sep, 2020 1 commit
  27. 18 Aug, 2020 1 commit
    • Andronik Ordian's avatar
      initial prometheus metrics (#1536) · 804958ab
      Andronik Ordian authored
      * service-new: cosmetic changes
      
      * overseer: draft of prometheus metrics
      
      * metrics: update active_leaves metrics
      
      * metrics: extract into functions
      
      * metrics: resolve XXX
      
      * metrics: it's ugly, but it works
      
      * Bump Substrate
      
      * metrics: move a bunch of code around
      
      * Bumb substrate again
      
      * metrics: fix a warning
      
      * fix a warning in runtime
      
      * metrics: statements signed
      
      * metrics: statements impl RegisterMetrics
      
      * metrics: refactor Metrics trait
      
      * metrics: add Metrics assoc type to JobTrait
      
      * metrics: move Metrics trait to util
      
      * metrics: fix overseer
      
      * metrics: fix backing
      
      * metrics: fix candidate validation
      
      * metrics: derive Default
      
      * metrics: docs
      
      * metrics: add stubs for other subsystems
      
      * metrics: add more stubs and fix compilation
      
      * metrics: fix doctest
      
      * metrics: move to subsystem
      
      * metrics: fix candidate validation
      
      * metrics: bitfield signing
      
      * metrics: av store
      
      * metrics: chain API
      
      * metrics: runtime API
      
      * metrics: stub for avad
      
      * metrics: candidates seconded
      
      * metrics: ok I gave up
      
      * metrics: provisioner
      
      * metrics: remove a clone by requiring Metrics: Sync
      
      * metrics: YAGNI
      
      * metrics: remove another TODO
      
      * metrics: for later
      
      * metrics: add parachain_ prefix
      
      * metrics: s/signed_statement/signed_statements
      
      * utils: add a comment for job metrics
      
      * metrics: address review comments
      
      * metrics: oops
      
      * metrics: make sure to save files before commit 😅
      
      
      
      * use _total suffix for requests metrics
      
      Co-authored-by: default avatarMax Inden <mail@max-inden.de>
      
      * metrics: add tests for overseer
      
      * update Cargo.lock
      
      * overseer: add a test for CollationGeneration
      
      * collation-generation: impl metrics
      
      * collation-generation: use kebab-case for name
      
      * collation-generation: add a constructor
      
      Co-authored-by: default avatarGav Wood <gavin@parity.io>
      Co-authored-by: default avatarAshley Ruglys <ashley.ruglys@gmail.com>
      Co-authored-by: default avatarMax Inden <mail@max-inden.de>
      804958ab
  28. 12 Aug, 2020 1 commit
    • asynchronous rob's avatar
      Network bridge refactoring impl (#1537) · 1e9c0540
      asynchronous rob authored
      * update networking types
      
      * port over overseer-protocol message types
      
      * Add the collation protocol to network bridge
      
      * message sending
      
      * stub for ConnectToValidators
      
      * add some helper traits and methods to protocol types
      
      * add collator protocol message
      
      * leaves-updating
      
      * peer connection and disconnection
      
      * add utilities for dispatching multiple events
      
      * implement message handling
      
      * add an observedrole enum with equality and no sentry nodes
      
      * derive partial-eq on network bridge event
      
      * add PartialEq impls for network message types
      
      * add Into implementation for observedrole
      
      * port over existing network bridge tests
      
      * add some more tests
      
      * port bitfield distribution
      
      * port over bitfield distribution tests
      
      * add codec indices
      
      * port PoV distribution
      
      * port over PoV distribution tests
      
      * port over statement distribution
      
      * port over statement distribution tests
      
      * update overseer and service-new
      
      * address review comments
      
      * port availability distribution
      
      * port over availability distribution tests
      1e9c0540
  29. 07 Aug, 2020 1 commit
  30. 28 Jul, 2020 1 commit
    • asynchronous rob's avatar
      Add new Runtime API messages and make runtime API request fallible (#1485) · 7ed4cb9c
      asynchronous rob authored
      * polkadot-subsystem: update runtime API message types
      
      * update all networking subsystems to use fallible runtime APIs
      
      * fix bitfield-signing and make it use new runtime APIs
      
      * port candidate-backing to handle runtime API errors and new types
      
      * remove old runtime API messages
      
      * remove unused imports
      
      * fix grumbles
      
      * fix backing tests
      7ed4cb9c
  31. 27 Jul, 2020 2 commits
    • Fedor Sakharov's avatar
      Availability store subsystem (#1404) · b838b386
      Fedor Sakharov authored
      * Initial commit
      
      * WIP
      
      * Make atomic transactions
      
      * Remove pruning code
      
      * Fix build and add a Nop to bridge
      
      * Fixes from review
      
      * Move config struct around for clarity
      
      * Rename constructor and warn on missing docs
      
      * Fix a test and rename a message
      
      * Fix some more reviews
      
      * Obviously failed to rebase cleanly
      b838b386
    • Peter Goodspeed-Niklaus's avatar
      add ActiveLeavesUpdate, remove StartWork, StopWork (#1458) · 12a02556
      Peter Goodspeed-Niklaus authored
      * add ActiveLeavesUpdate, remove StartWork, StopWork
      
      * replace StartWork, StopWork in subsystem crate tests
      
      * mechanically update OverseerSignal in other modules
      
      * convert overseer to take advantage of new multi-hash update abilities
      
      Note: this does not yet convert the tests; some of the tests now freeze:
      
      test tests::overseer_start_stop_works ... test tests::overseer_start_stop_works has been running for over 60 seconds
      test tests::overseer_finalize_works ... test tests::overseer_finalize_works has been running for over 60 seconds
      
      * fix broken overseer tests
      
      * manually impl PartialEq for ActiveLeavesUpdate, rm trait Equivalent
      
      This cleans up the code a bit and makes it easier in the future to
      do the right thing when comparing ALUs.
      
      * use target in all network bridge logging
      
      * reduce spamming of  and
      12a02556
  32. 26 Jul, 2020 1 commit
  33. 21 Jul, 2020 1 commit
    • Peter Goodspeed-Niklaus's avatar
      Add test suite and minor refinements to the utility subsystem (#1403) · cc2d7afd
      Peter Goodspeed-Niklaus authored
      * get conclude signal working properly; don't allocate a vector
      
      * wip: add test suite / example / explanation for using utility subsystem
      
      Unfortunately, the test fails right now for reasons which seem
      very odd. Just have to keep poking at it.
      
      * explicitly import everything
      
      * fix subsystem-util test
      
      The root problem here was two-fold:
      
      - there was a circular dependency from subsystem -> test-helpers/subsystem ->
        subsystem
      - cfg(test) doesn't propagate between crates
      
      The solution: move the subsystem test helpers into a sub-module
      within subsystem. Publicly export them from the previous location
      so no other code breaks.
      
      Doing this has an additional benefit: it ensures that no production
      code can ever accidentally use the subsystem helpers, as they are compile-
      gated on cfg(test).
      
      * fully commit to moving test helpers into a subsystem module
      
      * add some more tests
      
      * get rid of log tests in favor of real error forwarding
      
      It's not obvious whether we'll ever really want to chase down
      these errors outside a testing context, but having the capability
      won't hurt.
      
      * fix issue which caused test to hang on osx
      
      * only require that job errors are PartialEq when testing
      
      also fix polkadot-node-core-backing tests
      
      * get rid of any notion of partialeq
      
      * rethink testing
      
      Combine tests of starting and stopping job: leaving a test executor
      with a job running was pretty clearly the cause of the sometimes-hang.
      
      Also, add a timeout so tests _can't_ hang anymore; they just fail
      after a while.
      
      * rename fwd_errors -> forward_errors
      
      * warn on error propagation failure
      
      * fix unused import leftover from merge
      
      * derive eq for subsystemerror
      cc2d7afd
  34. 17 Jul, 2020 1 commit
  35. 10 Jul, 2020 1 commit
    • asynchronous rob's avatar
      Refactor primitives (#1383) · 96af6ead
      asynchronous rob authored
      * create a v1 primitives module
      
      * Improve guide on availability types
      
      * punctuate
      
      * new parachains runtime uses new primitives
      
      * tests of new runtime now use new primitives
      
      * add ErasureChunk to guide
      
      * export erasure chunk from v1 primitives
      
      * subsystem crate uses v1 primitives
      
      * node-primitives uses new v1 primitives
      
      * port overseer to new primitives
      
      * new-proposer uses v1 primitives (no ParachainHost anymore)
      
      * fix no-std compilation for primitives
      
      * service-new uses v1 primitives
      
      * network-bridge uses new primitives
      
      * statement distribution uses v1 primitives
      
      * PoV distribution uses v1 primitives; add PoV::hash fn
      
      * move parachain to v0
      
      * remove inclusion_inherent module and place into v1
      
      * remove everything from primitives crate root
      
      * remove some unused old types from v0 primitives
      
      * point everything else at primitives::v0
      
      * squanch some warns up
      
      * add RuntimeDebug import to no-std as well
      
      * port over statement-table and validation
      
      * fix final errors in validation and node-primitives
      
      * add dummy Ord impl to committed candidate receipt
      
      * guide: update CandidateValidationMessage
      
      * add primitive for validationoutputs
      
      * expand CandidateValidationMessage further
      
      * bikeshed
      
      * add some impls to omitted-validation-data and available-data
      
      * expand CandidateValidationMessage
      
      * make erasure-coding generic over v1/v0
      
      * update usages of erasure-coding
      
      * implement commitments.hash()
      
      * use Arc<Pov> for CandidateValidation
      
      * improve new erasure-coding method names
      
      * fix up candidate backing
      
      * update docs a bit
      
      * fix most tests and add short-circuiting to make_pov_available
      
      * fix remainder of candidate backing tests
      
      * squanching warns
      
      * squanch it up
      
      * some fallout
      
      * overseer fallout
      
      * free from polkadot-test-service hell
      96af6ead
  36. 06 Jul, 2020 1 commit
    • asynchronous rob's avatar
      Implement the Statement Distribution Subsystem (#1326) · 8348cc4c
      asynchronous rob authored
      * set up data types and control flow for statement distribution
      
      * add some set-like methods to View
      
      * implement sending to peers
      
      * start fixing equivocation handling
      
      * Add a section to the statement distribution subsystem on equivocations and flood protection
      
      * fix typo and amend wording
      
      * implement flood protection
      
      * have peer knowledge tracker follow when peer first learns about a candidate
      
      * send dependents after circulating
      
      * add another TODO
      
      * trigger send in one more place
      
      * refactors from review
      
      * send new statements to candidate backing
      
      * instantiate active head data with runtime API values
      
      * track our view changes and peer view changes
      
      * apply a benefit to peers who send us statements we want
      
      * remove unneeded TODO
      
      * add some comments and improve Hash implementation
      
      * start tests and fix `note_statement`
      
      * test active_head seconding logic
      
      * test that the per-peer tracking logic works
      
      * test per-peer knowledge tracker
      
      * test that peer view updates lead to messages being sent
      
      * test statement circulation
      
      * address review comments
      
      * have view set methods return references
      8348cc4c