1. 30 Aug, 2021 1 commit
  2. 26 Aug, 2021 1 commit
  3. 12 Aug, 2021 1 commit
    • Robert Klotzner's avatar
      Remove request multiplexer (#3624) · 117466aa
      Robert Klotzner authored
      * WIP: Get rid of request multiplexer.
      
      * WIP
      
      * Receiver for handling of incoming requests.
      
      * Get rid of useless `Fault` abstraction.
      
      The things the type system let us do are not worth getting abstracted in
      its own type. Instead error handling is going to be merely a pattern.
      
      * Make most things compile again.
      
      * Port availability distribution away from request multiplexer.
      
      * Formatting.
      
      * Port dispute distribution over.
      
      * Fixup statement distribution.
      
      * Handle request directly in collator protocol.
      
      + Only allow fatal errors at top level.
      
      * Use direct request channel for availability recovery.
      
      * Finally get rid of request multiplexer
      
      Fixes #2842 and paves the way for more back pressure possibilities.
      
      * Fix overseer and statement distribution tests.
      
      * Fix collator protocol and network bridge tests.
      
      * Fix tests in availability recovery.
      
      * Fix availability distribution tests.
      
      * Fix dispute distribution tests.
      
      * Add missing dependency
      
      * Typos.
      
      * Review remarks.
      
      * More remarks.
      117466aa
  4. 07 Aug, 2021 1 commit
  5. 06 Aug, 2021 2 commits
  6. 09 Jul, 2021 1 commit
    • Robert Klotzner's avatar
      Dispute distribution implementation (#3282) · f9d71f8c
      Robert Klotzner authored
      * Dispute protocol.
      
      * Dispute distribution protocol.
      
      * Get network requests routed.
      
      * WIP: Basic dispute sender logic.
      
      * Basic validator determination logic.
      
      * WIP: Getting things to typecheck.
      
      * Slightly larger timeout.
      
      * More typechecking stuff.
      
      * Cleanup.
      
      * Finished most of the sending logic.
      
      * Handle active leaves updates
      
      - Cleanup dead disputes
      - Update sends for new sessions
      - Retry on errors
      
      * Pass sessions in already.
      
      * Startup dispute sending.
      
      * Provide incoming decoding facilities
      
      and use them in statement-distribution.
      
      * Relaxed runtime util requirements.
      
      We only need a `SubsystemSender` not a full `SubsystemContext`.
      
      * Better usability of incoming requests.
      
      Make it possible to consume stuff without clones.
      
      * Add basic receiver functionality.
      
      * Cleanup + fixes for sender.
      
      * One more sender fix.
      
      * Start receiver.
      
      * Make sure to send responses back.
      
      * WIP: Exposed authority discovery
      
      * Make tests pass.
      
      * Fully featured receiver.
      
      * Decrease cost of `NotAValidator`.
      
      * Make `RuntimeInfo` LRU cache size configurable.
      
      * Cache more sessions.
      
      * Fix collator protocol.
      
      * Disable metrics for now.
      
      * Make dispute-distribution a proper subsystem.
      
      * Fix naming.
      
      * Code style fixes.
      
      * Factored out 4x copied mock function.
      
      * WIP: Tests.
      
      * Whitespace cleanup.
      
      * Accessor functions.
      
      * More testing.
      
      * More Debug instances.
      
      * Fix busy loop.
      
      * Working tests.
      
      * More tests.
      
      * Cleanup.
      
      * Fix build.
      
      * Basic receiving test.
      
      * Non validator message gets dropped.
      
      * More receiving tests.
      
      * Test nested and subsequent imports.
      
      * Fix spaces.
      
      * Better formatted imports.
      
      * Import cleanup.
      
      * Metrics.
      
      * Message -> MuxedMessage
      
      * Message -> MuxedMessage
      
      * More review remarks.
      
      * Add missing metrics.rs.
      
      * Fix flaky test.
      
      * Dispute coordinator - deliver confirmations.
      
      * Send out `DisputeMessage` on issue local statement.
      
      * Unwire dispute distribution.
      
      * Review remarks.
      
      * Review remarks.
      
      * Better docs.
      f9d71f8c
  7. 09 Jun, 2021 1 commit
  8. 27 May, 2021 1 commit
  9. 29 Mar, 2021 1 commit
  10. 11 Mar, 2021 1 commit
  11. 03 Mar, 2021 1 commit
  12. 26 Feb, 2021 1 commit
    • Robert Klotzner's avatar
      Request based availability distribution (#2423) · 950447e1
      Robert Klotzner authored
      * WIP
      
      * availability distribution, still very wip.
      
      Work on the requesting side of things.
      
      * Some docs on what I intend to do.
      
      * Checkpoint of session cache implementation
      
      as I will likely replace it with something smarter.
      
      * More work, mostly on cache
      
      and getting things to type check.
      
      * Only derive MallocSizeOf and Debug for std.
      
      * availability-distribution: Cache feature complete.
      
      * Sketch out logic in `FetchTask` for actual fetching.
      
      - Compile fixes.
      - Cleanup.
      
      * Format cleanup.
      
      * More format fixes.
      
      * Almost feature complete `fetch_task`.
      
      Missing:
      
      - Check for cancel
      - Actual querying of peer ids.
      
      * Finish FetchTask so far.
      
      * Directly use AuthorityDiscoveryId in protocol and cache.
      
      * Resolve `AuthorityDiscoveryId` on sending requests.
      
      * Rework fetch_task
      
      - also make it impossible to check the wrong chunk index.
      - Export needed function in validator_discovery.
      
      * From<u32> implementation for `ValidatorIndex`.
      
      * Fixes and more integration work.
      
      * Make session cache proper lru cache.
      
      * Use proper lru cache.
      
      * Requester finished.
      
      * ProtocolState -> Requester
      
      Also make sure to not fetch our own chunk.
      
      * Cleanup + fixes.
      
      * Remove unused functions
      
      - FetchTask::is_finished
      - SessionCache::fetch_session_info
      
      * availability-distribution responding side.
      
      * Cleanup + Fixes.
      
      * More fixes.
      
      * More fixes.
      
      adder-collator is running!
      
      * Some docs.
      
      * Docs.
      
      * Fix reporting of bad guys.
      
      * Fix tests
      
      * Make all tests compile.
      
      * Fix test.
      
      * Cleanup + get rid of some warnings.
      
      * state -> requester
      
      * Mostly doc fixes.
      
      * Fix test suite.
      
      * Get rid of now redundant message types.
      
      * WIP
      
      * Rob's review remarks.
      
      * Fix test suite.
      
      * core.relay_parent -> leaf for session request.
      
      * Style fix.
      
      * Decrease request timeout.
      
      * Cleanup obsolete errors.
      
      * Metrics + don't fail on non fatal errors.
      
      * requester.rs -> requester/mod.rs
      
      * Panic on invalid BadValidator report.
      
      * Fix indentation.
      
      * Use typed default timeout constant.
      
      * Make channel size 0, as each sender gets one slot anyways.
      
      * Fix incorrect metrics initialization.
      
      * Fix build after merge.
      
      * More fixes.
      
      * Hopefully valid metrics names.
      
      * Better metrics names.
      
      * Some tests that already work.
      
      * Slightly better docs.
      
      * Some more tests.
      
      * Fix network bridge test.
      950447e1
  13. 29 Jan, 2021 1 commit
  14. 18 Jan, 2021 1 commit
  15. 15 Jan, 2021 1 commit
  16. 11 Jan, 2021 1 commit
  17. 28 Dec, 2020 1 commit
  18. 17 Dec, 2020 1 commit
    • Bastian Köcher's avatar
      Fix bug and further optimizations in availability distribution (#2104) · affa668a
      Bastian Köcher authored
      
      
      * Fix bug and further optimizations in availability distribution
      
      - There was a bug that resulted in only getting one candidate per block
      as the candidates were put into the hashmap with the relay block hash as
      key. The solution for this is to use the candidate hash and the relay
      block hash as key.
      - We stored received/sent messages with the candidate hash and chunk
      index as key. The candidate hash wasn't required in this case, as the
      messages are already stored per candidate.
      
      * Update node/core/bitfield-signing/src/lib.rs
      Co-authored-by: asynchronous rob's avatarRobert Habermeier <rphmeier@gmail.com>
      
      * Remove the reverse map
      
      * major refactor of receipts & query_live
      
      * finish refactoring
      
      remove ancestory mapping,
      
      improve relay-parent cleanup & receipts-cache cleanup,
      add descriptor to `PerCandidate`
      
      * rename and rewrite query_pending_availability
      
      * add a bunch of consistency tests
      
      * Add some last changes
      
      * xy
      
      * fz
      
      * Make it compile again
      
      * Fix one test
      
      * Fix logging
      
      * Remove some buggy code
      
      * Make tests work again
      
      * Move stuff around
      
      * Remove dbg
      
      * Remove state from test_harness
      
      * More refactor and new test
      
      * New test and fixes
      
      * Move metric
      
      * Remove "duplicated code"
      
      * Fix tests
      
      * New test
      
      * Change break to continue
      
      * Update node/core/av-store/src/lib.rs
      
      * Update node/core/av-store/src/lib.rs
      
      * Update node/core/bitfield-signing/src/lib.rs
      Co-authored-by: Fedor Sakharov's avatarFedor Sakharov <fedor.sakharov@gmail.com>
      
      * update guide to match live_candidates changes
      
      * add comment
      
      * fix bitfield signing
      Co-authored-by: asynchronous rob's avatarRobert Habermeier <rphmeier@gmail.com>
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      Co-authored-by: Fedor Sakharov's avatarFedor Sakharov <fedor.sakharov@gmail.com>
      affa668a
  19. 07 Dec, 2020 1 commit
  20. 24 Nov, 2020 1 commit
  21. 20 Nov, 2020 1 commit
    • Peter Goodspeed-Niklaus's avatar
      Add tracing support to node (#1940) · f7ea3d07
      Peter Goodspeed-Niklaus authored
      * drop in tracing to replace log
      
      * add structured logging to trace messages
      
      * add structured logging to debug messages
      
      * add structured logging to info messages
      
      * add structured logging to warn messages
      
      * add structured logging to error messages
      
      * normalize spacing and Display vs Debug
      
      * add instrumentation to the various 'fn run'
      
      * use explicit tracing module throughout
      
      * fix availability distribution test
      
      * don't double-print errors
      
      * remove further redundancy from logs
      
      * fix test errors
      
      * fix more test errors
      
      * remove unused kv_log_macro
      
      * fix unused variable
      
      * add tracing spans to collation generation
      
      * add tracing spans to av-store
      
      * add tracing spans to backing
      
      * add tracing spans to bitfield-signing
      
      * add tracing spans to candidate-selection
      
      * add tracing spans to candidate-validation
      
      * add tracing spans to chain-api
      
      * add tracing spans to provisioner
      
      * add tracing spans to runtime-api
      
      * add tracing spans to availability-distribution
      
      * add tracing spans to bitfield-distribution
      
      * add tracing spans to network-bridge
      
      * add tracing spans to collator-protocol
      
      * add tracing spans to pov-distribution
      
      * add tracing spans to statement-distribution
      
      * add tracing spans to overseer
      
      * cleanup
      f7ea3d07
  22. 19 Nov, 2020 1 commit
  23. 18 Nov, 2020 1 commit
  24. 17 Nov, 2020 1 commit
    • Andronik Ordian's avatar
      update most of the dependencies (#1946) · 31fb8fed
      Andronik Ordian authored
      * update tiny-keccak to 0.2
      
      * update deps except bitvec and shared_memory
      
      * fix some warning after futures upgrade
      
      * remove useless package rename caused by bug in cargo-upgrade
      
      * revert parity-util-mem *
      
      * remove unused import
      
      * cargo update
      
      * remove all renames on parity-scale-codec
      
      * remove the leftovers
      
      * remove unused dep
      31fb8fed
  25. 27 Oct, 2020 1 commit
  26. 09 Oct, 2020 1 commit
    • Rakan Alhneiti's avatar
      Update to work with async keystore – Companion PR for #7000 (#1740) · bc7d1322
      Rakan Alhneiti authored
      
      
      * Fix keystore types
      
      * Use SyncCryptoStorePtr
      
      * Borrow keystore
      
      * Fix unused imports
      
      * Fix polkadot service
      
      * Fix bitfield-distribution tests
      
      * Fix indentation
      
      * Fix backing tests
      
      * Fix tests
      
      * Fix provisioner tests
      
      * Removed SyncCryptoStorePtr
      
      * Fix services
      
      * Address PR feedback
      
      * Address PR feedback - 2
      
      * Update CryptoStorePtr imports to be from sp_keystore
      
      * Typo
      
      * Fix CryptoStore import
      
      * Document the reason behind using filesystem keystore
      
      * Remove VALIDATORS
      
      * Fix duplicate dependency
      
      * Mark sp-keystore as optional
      
      * Fix availability distribution
      
      * Fix call to sign_with
      
      * Fix keystore usage
      
      * Remove tokio and fix parachains Cargo config
      
      * Typos
      
      * Fix keystore dereferencing
      
      * Fix CryptoStore import
      
      * Fix provisioner
      
      * Fix node backing
      
      * Update services
      
      * Cleanup dependencies
      
      * Use sync_keystore
      
      * Fix node service
      
      * Fix node service - 2
      
      * Fix node service - 3
      
      * Rename CryptoStorePtr to SyncCryptoStorePtr
      
      * "Update Substrate"
      
      * Apply suggestions from code review
      
      * Update node/core/backing/Cargo.toml
      
      * Update primitives/src/v0.rs
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Fix wasm build
      
      * Update Cargo.lock
      
      Co-authored-by: parity-processbot <>
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      bc7d1322
  27. 10 Sep, 2020 1 commit
  28. 20 Aug, 2020 1 commit
  29. 12 Aug, 2020 1 commit
    • asynchronous rob's avatar
      Network bridge refactoring impl (#1537) · 1e9c0540
      asynchronous rob authored
      * update networking types
      
      * port over overseer-protocol message types
      
      * Add the collation protocol to network bridge
      
      * message sending
      
      * stub for ConnectToValidators
      
      * add some helper traits and methods to protocol types
      
      * add collator protocol message
      
      * leaves-updating
      
      * peer connection and disconnection
      
      * add utilities for dispatching multiple events
      
      * implement message handling
      
      * add an observedrole enum with equality and no sentry nodes
      
      * derive partial-eq on network bridge event
      
      * add PartialEq impls for network message types
      
      * add Into implementation for observedrole
      
      * port over existing network bridge tests
      
      * add some more tests
      
      * port bitfield distribution
      
      * port over bitfield distribution tests
      
      * add codec indices
      
      * port PoV distribution
      
      * port over PoV distribution tests
      
      * port over statement distribution
      
      * port over statement distribution tests
      
      * update overseer and service-new
      
      * address review comments
      
      * port availability distribution
      
      * port over availability distribution tests
      1e9c0540
  30. 10 Aug, 2020 1 commit
  31. 07 Aug, 2020 1 commit
  32. 28 Jul, 2020 2 commits
  33. 23 Jul, 2020 1 commit
    • Bernhard Schuster's avatar
      implement bitfield distribution subsystem (#1368) · 08fcbda4
      Bernhard Schuster authored
      
      
      * feat bitfield distribution
      
      * feat bitfield distribution part 2
      
      * pair programming with rustc & cargo
      
      * lets go
      
      * move bitfield-distribution to the node/network folder
      
      * shape shifting
      
      * lunchtime
      
      * ignore the two fn recursion for now
      
      * step by step
      
      * triplesteps
      
      * bandaid commit
      
      * unordered futures magic
      
      * chore
      
      * reword markdown
      
      * clarify
      
      * lacks abortable processing impl details
      
      * slimify
      
      * fix: warnings and avoid ctx.clone() improve comments
      
      * review comments
      
      * fix details
      
      * make sure outgoing messages are tracked
      
      * fix name
      
      * fix subsystem
      
      * partial test impl
      
      * relax context bounds
      
      * test
      
      * X
      
      * X
      
      * initial test
      
      * fix relay_message not tracked when origin is self
      
      * fix/guide: grammar
      Co-authored-by: asynchronous rob's avatarRobert Habermeier <rphmeier@gmail.com>
      
      * work around missing Eq+PartialEq
      
      * fix: add missing message to provisioner
      
      * unify per_job to job_data
      
      * fix/review: part one
      
      * fix/review: more grumbles
      
      * fix/review: track incoming messages per peer
      
      * fix/review: extract fn, avoid nested matches
      
      * fix/review: more tests, simplify test
      
      * fix/review: extend tests to cover more cases
      
      * chore/rename: Tracker -> ProtocolState
      
      * chore check and comment rewording
      
      * feat test: invalid peer message
      
      * remove ignored test cases and unused macros
      
      * fix master merge fallout + warnings
      Co-authored-by: asynchronous rob's avatarRobert Habermeier <rphmeier@gmail.com>
      08fcbda4
  34. 21 Jul, 2020 1 commit
    • Peter Goodspeed-Niklaus's avatar
      Add test suite and minor refinements to the utility subsystem (#1403) · cc2d7afd
      Peter Goodspeed-Niklaus authored
      * get conclude signal working properly; don't allocate a vector
      
      * wip: add test suite / example / explanation for using utility subsystem
      
      Unfortunately, the test fails right now for reasons which seem
      very odd. Just have to keep poking at it.
      
      * explicitly import everything
      
      * fix subsystem-util test
      
      The root problem here was two-fold:
      
      - there was a circular dependency from subsystem -> test-helpers/subsystem ->
        subsystem
      - cfg(test) doesn't propagate between crates
      
      The solution: move the subsystem test helpers into a sub-module
      within subsystem. Publicly export them from the previous location
      so no other code breaks.
      
      Doing this has an additional benefit: it ensures that no production
      code can ever accidentally use the subsystem helpers, as they are compile-
      gated on cfg(test).
      
      * fully commit to moving test helpers into a subsystem module
      
      * add some more tests
      
      * get rid of log tests in favor of real error forwarding
      
      It's not obvious whether we'll ever really want to chase down
      these errors outside a testing context, but having the capability
      won't hurt.
      
      * fix issue which caused test to hang on osx
      
      * only require that job errors are PartialEq when testing
      
      also fix polkadot-node-core-backing tests
      
      * get rid of any notion of partialeq
      
      * rethink testing
      
      Combine tests of starting and stopping job: leaving a test executor
      with a job running was pretty clearly the cause of the sometimes-hang.
      
      Also, add a timeout so tests _can't_ hang anymore; they just fail
      after a while.
      
      * rename fwd_errors -> forward_errors
      
      * warn on error propagation failure
      
      * fix unused import leftover from merge
      
      * derive eq for subsystemerror
      cc2d7afd
  35. 17 Jul, 2020 1 commit
  36. 08 Jul, 2020 1 commit
    • asynchronous rob's avatar
      Implement PoV Distribution Subsystem (#1344) · b651e4af
      asynchronous rob authored
      * introduce candidatedescriptor type
      
      * add PoVDistribution message type
      
      * loosen bound on PoV Distribution to account for equivocations
      
      * re-export some types from the messages module
      
      * begin PoV Distribution subsystem
      
      * remove redundant index from PoV distribution
      
      * define state machine for pov distribution
      
      * handle overseer signals
      
      * set up control flow
      
      * remove `ValidatorStatement` section
      
      * implement PoV fetching
      
      * implement distribution logic
      
      * add missing `
      
      * implement some network bridge event handlers
      
      * stub for message processing, handle our view change
      
      * control flow for handling messages
      
      * handle `awaiting` message
      
      * handle any incoming PoVs and redistribute
      
      * actually provide a subsystem implementation
      
      * remove set-builder notation
      
      * begin testing PoV distribution
      
      * test that we send awaiting messages only to peers with same view
      
      * ensure we distribute awaited PoVs to peers on view changes
      
      * test that peers can complete fetch and are rewarded
      
      * test some reporting logic
      
      * ensure peer is reported for flooding
      
      * test punishing peers diverging from awaited protocol
      
      * test that we eagerly complete peers' awaited PoVs based on what we receive
      
      * test that we prune the awaited set after receiving
      
      * expand pov-distribution in guide to match a change I made
      
      * remove unneeded import
      b651e4af
  37. 30 Jun, 2020 1 commit
    • asynchronous rob's avatar
      Implement Network Bridge (#1280) · eaae8653
      asynchronous rob authored
      
      
      * network bridge skeleton
      
      * move some primitives around and add debug impls
      
      * protocol registration glue & abstract network interface
      
      * add send_msgs to subsystemctx
      
      * select logic
      
      * transform different events into actions and handle
      
      * implement remaining network bridge state machine
      
      * start test skeleton
      
      * make network methods asynchronous
      
      * extract subsystem out to subsystem crate
      
      * port over overseer to subsystem context trait
      
      * fix minimal example
      
      * fix overseer doc test
      
      * update network-bridge crate
      
      * write a subsystem test-helpers crate
      
      * write a network test helper for network-bridge
      
      * set up (broken) view test
      
      * Revamp network to be more async-friendly and not require Sync
      
      * fix spacing
      
      * fix test compilation
      
      * insert side-channel for actions
      
      * Add some more message types to AllMessages
      
      * introduce a test harness
      
      * add some tests
      
      * ensure service compiles and passes tests
      
      * fix typo
      
      * fix service-new compilation
      
      * Subsystem test helpers send messages synchronously
      
      * remove smelly action inspector
      
      * remove superfluous let binding
      
      * fix warnings
      
      * Update node/network/bridge/src/lib.rs
      Co-authored-by: default avatarPeter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
      
      * fix compilation
      Co-authored-by: default avatarPeter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
      eaae8653