1. 29 Jan, 2021 1 commit
  2. 25 Jan, 2021 1 commit
  3. 18 Jan, 2021 1 commit
  4. 15 Jan, 2021 1 commit
  5. 11 Jan, 2021 2 commits
  6. 17 Dec, 2020 1 commit
    • Andronik Ordian's avatar
      refactor View to include finalized_number (#2128) · c429e15c
      Andronik Ordian authored
      * refactor View to include finalized_number
      
      * guide: update the NetworkBridge on BlockFinalized
      
      * av-store: fix the tests
      
      * actually fix tests
      
      * grumbles
      
      * ignore macro doctest
      
      * use Hash::repeat_bytes more consistently
      
      * broadcast empty leaves updates as well
      
      * fix issuing view updates on empty leaves updates
      c429e15c
  7. 27 Nov, 2020 1 commit
    • asynchronous rob's avatar
      Move erasure root out of candidate commitments and into descriptor (#2010) · fff46359
      asynchronous rob authored
      
      
      * guide: move erasure-root to candidate descriptor
      
      * primitives: move erasure root to descriptor
      
      * guide: unify candidate commitments and validation outputs
      
      * primitives: unify validation outputs and candidate commitments
      
      * parachains-runtime: fix fallout
      
      * runtimes: fix fallout
      
      * collation generation: fix fallout
      
      * fix stray reference in primitives
      
      * fix fallout in node-primitives
      
      * fix remaining fallout in collation generation
      
      * fix fallout in candidate validation
      
      * fix fallout in runtime API subsystem
      
      * fix fallout in subsystem messages
      
      * fix fallout in candidate backing
      
      * fix fallout in availability distribution
      
      * don't clone
      
      * clone
      
      Co-authored-by: Sergey Pepyakin's avatarSergei Shulepov <sergei@parity.io>
      fff46359
  8. 24 Nov, 2020 1 commit
  9. 23 Nov, 2020 1 commit
  10. 20 Nov, 2020 2 commits
    • Peter Goodspeed-Niklaus's avatar
      Add Prometheus timers to the subsystems (#1923) · e655654e
      Peter Goodspeed-Niklaus authored
      * reexport prometheus-super for ease of use of other subsystems
      
      * add some prometheus timers for collation generation subsystem
      
      * add timing metrics to av-store
      
      * add metrics to candidate backing
      
      * add timing metric to bitfield signing
      
      * add timing metrics to candidate selection
      
      * add timing metrics to candidate-validation
      
      * add timing metrics to chain-api
      
      * add timing metrics to provisioner
      
      * add timing metrics to runtime-api
      
      * add timing metrics to availability-distribution
      
      * add timing metrics to bitfield-distribution
      
      * add timing metrics to collator protocol: collator side
      
      * add timing metrics to collator protocol: validator side
      
      * fix candidate validation test failures
      
      * add timing metrics to pov distribution
      
      * add timing metrics to statement-distribution
      
      * use substrate_prometheus_endpoint prometheus reexport instead of prometheus_super
      
      * don't include JOB_DELAY in bitfield-signing metrics
      
      * give adder-collator ability to easily export its genesis-state and validation code
      
      * wip: adder-collator pushbutton script
      
      * don't attempt to register the adder-collator automatically
      
      Instead, get these values with
      
      ```sh
      target/release/adder-collator export-genesis-state
      target/release/adder-collator export-genesis-wasm
      ```
      
      And then register the parachain on https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944#/explorer
      
      To collect prometheus data, after running the script, create `prometheus.yml` per the instructions
      at https://www.notion.so/paritytechnologies/Setting-up-Prometheus-locally-835cb3a9df7541a781c381006252b5ff
      
      
      and then run:
      
      ```sh
      docker run -v `pwd`/prometheus.yml:/etc/prometheus/prometheus.yml:z --network host prom/prometheus
      ```
      
      Demonstrates that data makes it across to prometheus, though it is likely to be useful in the future
      to tweak the buckets.
      
      * Update parachain/test-parachains/adder/collator/src/cli.rs
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      
      * use the grandpa-pause parameter
      
      * skip metrics in tracing instrumentation
      
      * remove unnecessary grandpa_pause cli param
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      e655654e
    • Peter Goodspeed-Niklaus's avatar
      Add tracing support to node (#1940) · f7ea3d07
      Peter Goodspeed-Niklaus authored
      * drop in tracing to replace log
      
      * add structured logging to trace messages
      
      * add structured logging to debug messages
      
      * add structured logging to info messages
      
      * add structured logging to warn messages
      
      * add structured logging to error messages
      
      * normalize spacing and Display vs Debug
      
      * add instrumentation to the various 'fn run'
      
      * use explicit tracing module throughout
      
      * fix availability distribution test
      
      * don't double-print errors
      
      * remove further redundancy from logs
      
      * fix test errors
      
      * fix more test errors
      
      * remove unused kv_log_macro
      
      * fix unused variable
      
      * add tracing spans to collation generation
      
      * add tracing spans to av-store
      
      * add tracing spans to backing
      
      * add tracing spans to bitfield-signing
      
      * add tracing spans to candidate-selection
      
      * add tracing spans to candidate-validation
      
      * add tracing spans to chain-api
      
      * add tracing spans to provisioner
      
      * add tracing spans to runtime-api
      
      * add tracing spans to availability-distribution
      
      * add tracing spans to bitfield-distribution
      
      * add tracing spans to network-bridge
      
      * add tracing spans to collator-protocol
      
      * add tracing spans to pov-distribution
      
      * add tracing spans to statement-distribution
      
      * add tracing spans to overseer
      
      * cleanup
      f7ea3d07
  11. 19 Nov, 2020 1 commit
  12. 17 Nov, 2020 1 commit
    • Andronik Ordian's avatar
      update most of the dependencies (#1946) · 31fb8fed
      Andronik Ordian authored
      * update tiny-keccak to 0.2
      
      * update deps except bitvec and shared_memory
      
      * fix some warning after futures upgrade
      
      * remove useless package rename caused by bug in cargo-upgrade
      
      * revert parity-util-mem *
      
      * remove unused import
      
      * cargo update
      
      * remove all renames on parity-scale-codec
      
      * remove the leftovers
      
      * remove unused dep
      31fb8fed
  13. 09 Nov, 2020 2 commits
  14. 06 Nov, 2020 1 commit
    • Sergey Pepyakin's avatar
      Implement HRMP (#1900) · 73553661
      Sergey Pepyakin authored
      * HRMP: Update the impl guide
      
      * HRMP: Incorporate the channel notifications into the guide
      
      * HRMP: Renaming in the impl guide
      
      * HRMP: Constrain the maximum number of HRMP messages per candidate
      
      This commit addresses the HRMP part of https://github.com/paritytech/polkadot/issues/1869
      
      
      
      * XCM: Introduce HRMP related message types
      
      * HRMP: Data structures and plumbing
      
      * HRMP: Configuration
      
      * HRMP: Data layout
      
      * HRMP: Acceptance & Enactment
      
      * HRMP: Test base logic
      
      * Update adder collator
      
      * HRMP: Runtime API for accessing inbound messages
      
      Also, removing some redundant fully-qualified names.
      
      * HRMP: Add diagnostic logging in acceptance criteria
      
      * HRMP: Additional tests
      
      * Self-review fixes
      
      * save test refactorings for the next time
      
      * Missed a return statement.
      
      * a formatting blip
      
      * Add missing logic for appending HRMP digests
      
      * Remove the channel contents vectors which became empty
      
      * Tighten HRMP channel digests invariants.
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarPeter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
      
      * Remove a note about sorting for channel id
      
      * Add missing rustdocs to the configuration
      
      * Clarify and update the invariant for HrmpChannelDigests
      
      * Make the onboarding invariant less sloppy
      
      Namely, introduce `Paras::is_valid_para` (in fact, it already is present
      in the implementation) and hook up the invariant to that.
      
      Note that this says "within a session" because I don't want to make it
      super strict on the session boundary. The logic on the session boundary
      should be extremely careful.
      
      * Make `CandidateCheckContext` use T::BlockNumber for hrmp_watermark
      
      Co-authored-by: default avatarPeter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
      73553661
  15. 03 Nov, 2020 1 commit
  16. 02 Nov, 2020 2 commits
  17. 29 Oct, 2020 1 commit
  18. 28 Oct, 2020 4 commits
    • Fedor Sakharov's avatar
      Change SpawnedSubsystem type to log subsystem errors (#1878) · d34585dd
      Fedor Sakharov authored
      * Change SpawnedSubsystem type to log subsystem errors
      
      * Remove clone
      d34585dd
    • Sergey Pepyakin's avatar
      let go fees (#1867) · 11df6f86
      Sergey Pepyakin authored
      11df6f86
    • Sergey Pepyakin's avatar
      Downward Message Processing implementation (#1859) · ed759c75
      Sergey Pepyakin authored
      * DMP: data structures and plumbing
      
      * DMP: Implement DMP logic in the router module
      
      DMP: Integrate DMP parts into the inclusion module
      
      * DMP: Introduce the max size limit for the size of a downward message
      
      * DMP: Runtime API for accessing inbound messages
      
      * OCD
      
      Small clean ups
      
      * DMP: fix the naming of the error
      
      * DMP: add caution about a non-existent recipient
      ed759c75
    • Peter Goodspeed-Niklaus's avatar
      start working on building the real overseer (#1795) · 798f781f
      Peter Goodspeed-Niklaus authored
      
      
      * start working on building the real overseer
      
      Unfortunately, this fails to compile right now due to an upstream
      failure to compile which is probably brought on by a recent upgrade
      to rustc v1.47.
      
      * fill in AllSubsystems internal constructors
      
      * replace fn make_metrics with Metrics::attempt_to_register
      
      * update to account for #1740
      
      * remove Metrics::register, rename Metrics::attempt_to_register
      
      * add 'static bounds to real_overseer type params
      
      * pass authority_discovery and network_service to real_overseer
      
      It's not straightforwardly obvious that this is the best way to handle
      the case when there is no authority discovery service, but it seems
      to be the best option available at the moment.
      
      * select a proper database configuration for the availability store db
      
      * use subdirectory for av-store database path
      
      * apply Basti's patch which avoids needing to parameterize everything on Block
      
      * simplify path extraction
      
      * get all tests to compile
      
      * Fix Prometheus double-registry error
      
      for debugging purposes, added this to node/subsystem-util/src/lib.rs:472-476:
      
      ```rust
      Some(registry) => Self::try_register(registry).map_err(|err| {
      	eprintln!("PrometheusError calling {}::register: {:?}", std::any::type_name::<Self>(), err);
      	err
      }),
      ```
      
      That pointed out where the registration was failing, which led to
      this fix. The test still doesn't pass, but it now fails in a new
      and different way!
      
      * authorities must have authority discovery, but not necessarily overseer handlers
      
      * fix broken SpawnedSubsystem impls
      
      detailed logging determined that using the `Box::new` style of
      future generation, the `self.run` method was never being called,
      leading to dropped receivers / closed senders for those subsystems,
      causing the overseer to shut down immediately.
      
      This is not the final fix needed to get things working properly,
      but it's a good start.
      
      * use prometheus properly
      
      Prometheus lets us register simple counters, which aren't very
      interesting. It also allows us to register CounterVecs, which are.
      With a CounterVec, you can provide a set of labels, which can
      later be used to filter the counts.
      
      We were using them wrong, though. This pattern was repeated in a
      variety of places in the code:
      
      ```rust
      // panics with an cardinality mismatch
      let my_counter = register(CounterVec::new(opts, &["succeeded", "failed"])?, registry)?;
      my_counter.with_label_values(&["succeeded"]).inc()
      ```
      
      The problem is that the labels provided in the constructor are not
      the set of legal values which can be annotated, but a set of individual
      label names which can have individual, arbitrary values.
      
      This commit fixes that.
      
      * get av-store subsystem to actually run properly and not die on first signal
      
      * typo fix: incomming -> incoming
      
      * don't disable authority discovery in test nodes
      
      * Fix rococo-v1 missing session keys
      
      * Update node/core/av-store/Cargo.toml
      
      * try dummying out av-store on non-full-nodes
      
      * overseer and subsystems are required only for full nodes
      
      * Reduce the amount of warnings on browser target
      
      * Fix two more warnings
      
      * InclusionInherent should actually have an Inherent module on rococo
      
      * Ancestry: don't return genesis' parent hash
      
      * Update Cargo.lock
      
      * fix broken test
      
      * update test script: specify chainspec as script argument
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Update node/service/src/lib.rs
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * node/service/src/lib: Return error via ? operator
      
      * post-merge blues
      
      * add is_collator flag
      
      * prevent occasional av-store test panic
      
      * simplify fix; expand application
      
      * run authority_discovery in Role::Discover when collating
      
      * distinguish between proposer closed channel errors
      
      * add IsCollator enum, remove is_collator CLI flag
      
      * improve formatting
      
      * remove nop loop
      
      * Fix some stuff
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      Co-authored-by: Bastian Köcher's avatarBastian Köcher <git@kchr.de>
      Co-authored-by: Fedor Sakharov's avatarFedor Sakharov <fedor.sakharov@gmail.com>
      Co-authored-by: default avatarRobert Habermeier <robert@Roberts-MBP.lan1>
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      Co-authored-by: default avatarMax Inden <mail@max-inden.de>
      798f781f
  19. 27 Oct, 2020 1 commit
  20. 24 Oct, 2020 1 commit
    • Sergey Pepyakin's avatar
      Runtime API for checking validation outputs (#1842) · 4a17a2bc
      Sergey Pepyakin authored
      * annoying whitespaces
      
      * update guide
      
      Add `CheckValidationOutputs` runtime api and also change the
      candidate-validation stuff
      
      * promote ValidationOutputs to global primitives
      
      i.e. move it from node specific primitives to global v1 primitives. This
      will be needed when we share it later in the runtime inclusion module
      
      * refactor acceptance checks in the inclusion module
      
      factor out the common code to share it during the block inclusion and
      for the forthcoming CheckValidationOutputs runtime api.
      
      Also note that the acceptance criteria was updated to incorporate checks
      that exist now in candidate-validation
      
      * plumb the runtime api outside
      
      * extract validation_data from ValidationOutputs
      
      * use runtime-api to check validation outputs
      
      apart from that refactor, update docs and tidy a bit
      
      * Update the maxium code size
      
      This is to fix a test that performs an upgrade.
      4a17a2bc
  21. 29 Sep, 2020 1 commit
  22. 11 Sep, 2020 1 commit
  23. 09 Sep, 2020 1 commit
  24. 18 Aug, 2020 2 commits
    • asynchronous rob's avatar
      Implement validation data refactor (#1585) · 8c881e45
      asynchronous rob authored
      
      
      * update primitives
      
      * correct parent_head field
      
      * make hrmp field pub
      
      * refactor validation data: runtime
      
      * refactor validation data: messages
      
      * add arguments to full_validation_data runtime API
      
      * port runtime API
      
      * mostly port over candidate validation
      
      * remove some parameters from ValidationParams
      
      * guide: update candidate validation
      
      * update candidate outputs
      
      * update ValidationOutputs in primitives
      
      * port over candidate validation
      
      * add a new test for no-transient behavior
      
      * update util runtime API wrappers
      
      * candidate backing
      
      * fix missing imports
      
      * change some fields of validation data around
      
      * runtime API impl
      
      * update candidate validation
      
      * fix backing tests
      
      * grumbles from review
      
      * fix av-store tests
      
      * fix some more crates
      
      * fix provisioner tests
      
      * fix availability distribution tests
      
      * port collation-generation to new validation data
      
      * fix overseer tests
      
      * Update roadmap/implementers-guide/src/node/utility/candidate-validation.md
      
      Co-authored-by: default avatarPeter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
      
      Co-authored-by: default avatarPeter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
      8c881e45
    • Andronik Ordian's avatar
      initial prometheus metrics (#1536) · 804958ab
      Andronik Ordian authored
      * service-new: cosmetic changes
      
      * overseer: draft of prometheus metrics
      
      * metrics: update active_leaves metrics
      
      * metrics: extract into functions
      
      * metrics: resolve XXX
      
      * metrics: it's ugly, but it works
      
      * Bump Substrate
      
      * metrics: move a bunch of code around
      
      * Bumb substrate again
      
      * metrics: fix a warning
      
      * fix a warning in runtime
      
      * metrics: statements signed
      
      * metrics: statements impl RegisterMetrics
      
      * metrics: refactor Metrics trait
      
      * metrics: add Metrics assoc type to JobTrait
      
      * metrics: move Metrics trait to util
      
      * metrics: fix overseer
      
      * metrics: fix backing
      
      * metrics: fix candidate validation
      
      * metrics: derive Default
      
      * metrics: docs
      
      * metrics: add stubs for other subsystems
      
      * metrics: add more stubs and fix compilation
      
      * metrics: fix doctest
      
      * metrics: move to subsystem
      
      * metrics: fix candidate validation
      
      * metrics: bitfield signing
      
      * metrics: av store
      
      * metrics: chain API
      
      * metrics: runtime API
      
      * metrics: stub for avad
      
      * metrics: candidates seconded
      
      * metrics: ok I gave up
      
      * metrics: provisioner
      
      * metrics: remove a clone by requiring Metrics: Sync
      
      * metrics: YAGNI
      
      * metrics: remove another TODO
      
      * metrics: for later
      
      * metrics: add parachain_ prefix
      
      * metrics: s/signed_statement/signed_statements
      
      * utils: add a comment for job metrics
      
      * metrics: address review comments
      
      * metrics: oops
      
      * metrics: make sure to save files before commit 😅
      
      
      
      * use _total suffix for requests metrics
      
      Co-authored-by: default avatarMax Inden <mail@max-inden.de>
      
      * metrics: add tests for overseer
      
      * update Cargo.lock
      
      * overseer: add a test for CollationGeneration
      
      * collation-generation: impl metrics
      
      * collation-generation: use kebab-case for name
      
      * collation-generation: add a constructor
      
      Co-authored-by: default avatarGav Wood <gavin@parity.io>
      Co-authored-by: default avatarAshley Ruglys <ashley.ruglys@gmail.com>
      Co-authored-by: default avatarMax Inden <mail@max-inden.de>
      804958ab
  25. 17 Aug, 2020 1 commit
  26. 07 Aug, 2020 1 commit
  27. 03 Aug, 2020 1 commit
  28. 31 Jul, 2020 1 commit
    • Andronik Ordian's avatar
      Chain API subsystem (#1498) · d4022633
      Andronik Ordian authored
      * chain-api subsystem skeleton
      
      * chain-api subsystem: draft impl
      
      * chain-api subsystem: mock testclient
      
      * chain-api subsystem: impl HeaderBacked for TestClient
      
      * chain-api subsystem: impl basic tests
      
      * chain-api subsystem: tiny guide
      
      * chain-api subsystem: rename ChainApiRequestMessage to ChainApiMessage
      
      * chain-api subsystem: add the page to the ToC
      
      * chain-api subsystem: proper error type
      
      * chain-api subsystem: impl ancestors request
      
      * chain-api subsystem: tests for ancestors request
      
      * guide: fix ancestor return type
      
      * runtime-api subsystem: remove unused dep
      
      * fix fmt
      
      * fix outdated comment
      
      * chain-api subsystem: s/format/to_string
      
      * lower-case subsystem names
      
      * chain-api subsystem: resolve Finalized todo
      
      * chain-api subsystem: remove TODO
      
      * extract request errors into a module
      
      * remove caching TODO
      
      * fix imports
      d4022633
  29. 30 Jul, 2020 1 commit
    • asynchronous rob's avatar
      Candidate Validation Subsystem (#1432) · cdb5c408
      asynchronous rob authored
      * skeleton for candidate-validation
      
      * add to workspace
      
      * implement candidate validation logic
      
      * guide: note occupied-core assumption for candidate validation
      
      * adjust message doc
      
      * wire together `run` asynchronously
      
      * add a Subsystem implementation
      
      * clean up a couple warnings
      
      * fix compilation errors due to merge
      
      * improve candidate-validation.md
      
      * remove old reference to subsystem-test helpers crate
      
      * update Cargo.lock
      
      * add a couple new Runtime API methods
      
      * add a candidate validation message
      
      * fetch validation data from the chain state
      
      * some tests for assumption checking
      
      * make spawn_validate_exhaustive mockable
      
      * more tests on the error handling side
      
      * fix all other grumbles except for wasm validation API change
      
      * wrap a SpawnNamed in candidate-validation
      
      * warn
      
      * amend guide
      
      * squanch warning
      
      * remove duplicate after merge
      cdb5c408