1. 10 Mar, 2021 1 commit
  2. 19 Feb, 2021 1 commit
    • Bernhard Schuster's avatar
      feat/jaeger: more spans, more stages (#2477) · f9186eb2
      Bernhard Schuster authored
      * feat/jaeger: more spans, more stages
      
      Stage numbers are still arbitrarily picked.
      
      * feat/jaeger: additional spans
      
      * chore/spellcheck: improve the dictionary
      
      * fix/jaeger JaegerSpan -> jaeger::Span
      f9186eb2
  3. 14 Feb, 2021 1 commit
    • Bastian Köcher's avatar
      Notify collators about seconded collation (#2430) · ac5ef00e
      Bastian Köcher authored
      * Notify collators about seconded collation
      
      This pr adds functionality to inform a collator that its collation was
      seconded by a parachain validator. Before this signed statement was only
      gossiped over the validation substream. Now, we explicitly send the
      seconded statement to the collator after it was validated successfully.
      
      Besides that it changes the `CollatorFn` to return an optional result
      sender that is informed when the build collation was seconded by a
      parachain validator.
      
      * Add test
      
      * Make sure we only send `Seconded` statements
      
      * Make sure we only receive valid statements
      
      * Review feedback
      ac5ef00e
  4. 05 Jan, 2021 1 commit
  5. 17 Dec, 2020 1 commit
  6. 11 Dec, 2020 1 commit
  7. 01 Dec, 2020 1 commit
  8. 30 Nov, 2020 1 commit
    • Bastian Köcher's avatar
      Simplify subsystem jobs (#2037) · 1fbf09ac
      Bastian Köcher authored
      * Simplify subsystem jobs
      
      This pr simplifies the subsystem jobs interface. Instead of requiring an
      extra message that is used to signal that a job should be ended, a job
      now ends when the receiver returns `None`. Besides that it changes the
      interface to enforce that messages to a job provide a relay parent.
      
      * Drop ToJobTrait
      
      * Remove FromJob
      
      We always convert this message to FromJobCommand anyway.
      1fbf09ac
  9. 28 Nov, 2020 1 commit
  10. 20 Nov, 2020 2 commits
    • Peter Goodspeed-Niklaus's avatar
      Add Prometheus timers to the subsystems (#1923) · e655654e
      Peter Goodspeed-Niklaus authored
      * reexport prometheus-super for ease of use of other subsystems
      
      * add some prometheus timers for collation generation subsystem
      
      * add timing metrics to av-store
      
      * add metrics to candidate backing
      
      * add timing metric to bitfield signing
      
      * add timing metrics to candidate selection
      
      * add timing metrics to candidate-validation
      
      * add timing metrics to chain-api
      
      * add timing metrics to provisioner
      
      * add timing metrics to runtime-api
      
      * add timing metrics to availability-distribution
      
      * add timing metrics to bitfield-distribution
      
      * add timing metrics to collator protocol: collator side
      
      * add timing metrics to collator protocol: validator side
      
      * fix candidate validation test failures
      
      * add timing metrics to pov distribution
      
      * add timing metrics to statement-distribution
      
      * use substrate_prometheus_endpoint prometheus reexport instead of prometheus_super
      
      * don't include JOB_DELAY in bitfield-signing metrics
      
      * give adder-collator ability to easily export its genesis-state and validation code
      
      * wip: adder-collator pushbutton script
      
      * don't attempt to register the adder-collator automatically
      
      Instead, get these values with
      
      ```sh
      target/release/adder-collator export-genesis-state
      target/release/adder-collator export-genesis-wasm
      ```
      
      And then register the parachain on https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944#/explorer
      
      To collect prometheus data, after running the script, create `prometheus.yml` per the instructions
      at https://www.notion.so/paritytechnologies/Setting-up-Prometheus-locally-835cb3a9df7541a781c381006252b5ff
      
      
      and then run:
      
      ```sh
      docker run -v `pwd`/prometheus.yml:/etc/prometheus/prometheus.yml:z --network host prom/prometheus
      ```
      
      Demonstrates that data makes it across to prometheus, though it is likely to be useful in the future
      to tweak the buckets.
      
      * Update parachain/test-parachains/adder/collator/src/cli.rs
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      
      * use the grandpa-pause parameter
      
      * skip metrics in tracing instrumentation
      
      * remove unnecessary grandpa_pause cli param
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      e655654e
    • Peter Goodspeed-Niklaus's avatar
      Add tracing support to node (#1940) · f7ea3d07
      Peter Goodspeed-Niklaus authored
      * drop in tracing to replace log
      
      * add structured logging to trace messages
      
      * add structured logging to debug messages
      
      * add structured logging to info messages
      
      * add structured logging to warn messages
      
      * add structured logging to error messages
      
      * normalize spacing and Display vs Debug
      
      * add instrumentation to the various 'fn run'
      
      * use explicit tracing module throughout
      
      * fix availability distribution test
      
      * don't double-print errors
      
      * remove further redundancy from logs
      
      * fix test errors
      
      * fix more test errors
      
      * remove unused kv_log_macro
      
      * fix unused variable
      
      * add tracing spans to collation generation
      
      * add tracing spans to av-store
      
      * add tracing spans to backing
      
      * add tracing spans to bitfield-signing
      
      * add tracing spans to candidate-selection
      
      * add tracing spans to candidate-validation
      
      * add tracing spans to chain-api
      
      * add tracing spans to provisioner
      
      * add tracing spans to runtime-api
      
      * add tracing spans to availability-distribution
      
      * add tracing spans to bitfield-distribution
      
      * add tracing spans to network-bridge
      
      * add tracing spans to collator-protocol
      
      * add tracing spans to pov-distribution
      
      * add tracing spans to statement-distribution
      
      * add tracing spans to overseer
      
      * cleanup
      f7ea3d07
  11. 05 Nov, 2020 2 commits
    • Andronik Ordian's avatar
      more resilient subsystems (#1908) · 9758a180
      Andronik Ordian authored
      * backing: extract log target
      
      * bitfield-signing: extract log target
      
      * utils: fix a typo
      
      * provisioner: extract log target
      
      * candidate selection: remove unused error variant
      
      * bitfield-distribution: change the return type of run
      
      * pov-distribution: extract log target
      
      * collator-protocol: simplify runtime request
      
      * collation-generation: do not exit early on error
      
      * collation-generation: do not exit on double init
      
      * collator-protocol: do not exit on errors and rename LOG_TARGET
      
      * collator-protocol: a workaround for ununused imports warning
      
      * Update node/network/bitfield-distribution/src/lib.rs
      
      * collation-generation: elevate warn! to error!
      
      * collator-protocol: fix imports
      
      * post merge fix
      
      * fix compilation
      9758a180
    • Bastian Köcher's avatar
      Do not validate a candidate in candidate selection (#1912) · c360efa4
      Bastian Köcher authored
      * Do not validate a candidate in candidate selection
      
      The candidate selection subsystem should not validate a candidate, as
      this is done by the backing subsystem on a `Second` request. Otherwise
      we validate one candidate twice.
      
      * Update candidate-selection.md
      c360efa4
  12. 03 Nov, 2020 1 commit
  13. 01 Nov, 2020 1 commit
  14. 28 Oct, 2020 3 commits
    • Sergey Pepyakin's avatar
      let go fees (#1867) · 11df6f86
      Sergey Pepyakin authored
      11df6f86
    • Sergey Pepyakin's avatar
      Downward Message Processing implementation (#1859) · ed759c75
      Sergey Pepyakin authored
      * DMP: data structures and plumbing
      
      * DMP: Implement DMP logic in the router module
      
      DMP: Integrate DMP parts into the inclusion module
      
      * DMP: Introduce the max size limit for the size of a downward message
      
      * DMP: Runtime API for accessing inbound messages
      
      * OCD
      
      Small clean ups
      
      * DMP: fix the naming of the error
      
      * DMP: add caution about a non-existent recipient
      ed759c75
    • Peter Goodspeed-Niklaus's avatar
      start working on building the real overseer (#1795) · 798f781f
      Peter Goodspeed-Niklaus authored
      
      
      * start working on building the real overseer
      
      Unfortunately, this fails to compile right now due to an upstream
      failure to compile which is probably brought on by a recent upgrade
      to rustc v1.47.
      
      * fill in AllSubsystems internal constructors
      
      * replace fn make_metrics with Metrics::attempt_to_register
      
      * update to account for #1740
      
      * remove Metrics::register, rename Metrics::attempt_to_register
      
      * add 'static bounds to real_overseer type params
      
      * pass authority_discovery and network_service to real_overseer
      
      It's not straightforwardly obvious that this is the best way to handle
      the case when there is no authority discovery service, but it seems
      to be the best option available at the moment.
      
      * select a proper database configuration for the availability store db
      
      * use subdirectory for av-store database path
      
      * apply Basti's patch which avoids needing to parameterize everything on Block
      
      * simplify path extraction
      
      * get all tests to compile
      
      * Fix Prometheus double-registry error
      
      for debugging purposes, added this to node/subsystem-util/src/lib.rs:472-476:
      
      ```rust
      Some(registry) => Self::try_register(registry).map_err(|err| {
      	eprintln!("PrometheusError calling {}::register: {:?}", std::any::type_name::<Self>(), err);
      	err
      }),
      ```
      
      That pointed out where the registration was failing, which led to
      this fix. The test still doesn't pass, but it now fails in a new
      and different way!
      
      * authorities must have authority discovery, but not necessarily overseer handlers
      
      * fix broken SpawnedSubsystem impls
      
      detailed logging determined that using the `Box::new` style of
      future generation, the `self.run` method was never being called,
      leading to dropped receivers / closed senders for those subsystems,
      causing the overseer to shut down immediately.
      
      This is not the final fix needed to get things working properly,
      but it's a good start.
      
      * use prometheus properly
      
      Prometheus lets us register simple counters, which aren't very
      interesting. It also allows us to register CounterVecs, which are.
      With a CounterVec, you can provide a set of labels, which can
      later be used to filter the counts.
      
      We were using them wrong, though. This pattern was repeated in a
      variety of places in the code:
      
      ```rust
      // panics with an cardinality mismatch
      let my_counter = register(CounterVec::new(opts, &["succeeded", "failed"])?, registry)?;
      my_counter.with_label_values(&["succeeded"]).inc()
      ```
      
      The problem is that the labels provided in the constructor are not
      the set of legal values which can be annotated, but a set of individual
      label names which can have individual, arbitrary values.
      
      This commit fixes that.
      
      * get av-store subsystem to actually run properly and not die on first signal
      
      * typo fix: incomming -> incoming
      
      * don't disable authority discovery in test nodes
      
      * Fix rococo-v1 missing session keys
      
      * Update node/core/av-store/Cargo.toml
      
      * try dummying out av-store on non-full-nodes
      
      * overseer and subsystems are required only for full nodes
      
      * Reduce the amount of warnings on browser target
      
      * Fix two more warnings
      
      * InclusionInherent should actually have an Inherent module on rococo
      
      * Ancestry: don't return genesis' parent hash
      
      * Update Cargo.lock
      
      * fix broken test
      
      * update test script: specify chainspec as script argument
      
      * Apply suggestions from code review
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * Update node/service/src/lib.rs
      
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      
      * node/service/src/lib: Return error via ? operator
      
      * post-merge blues
      
      * add is_collator flag
      
      * prevent occasional av-store test panic
      
      * simplify fix; expand application
      
      * run authority_discovery in Role::Discover when collating
      
      * distinguish between proposer closed channel errors
      
      * add IsCollator enum, remove is_collator CLI flag
      
      * improve formatting
      
      * remove nop loop
      
      * Fix some stuff
      
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      Co-authored-by: Bastian Köcher's avatarBastian Köcher <git@kchr.de>
      Co-authored-by: Fedor Sakharov's avatarFedor Sakharov <fedor.sakharov@gmail.com>
      Co-authored-by: default avatarRobert Habermeier <robert@Roberts-MBP.lan1>
      Co-authored-by: default avatarBastian Köcher <bkchr@users.noreply.github.com>
      Co-authored-by: default avatarMax Inden <mail@max-inden.de>
      798f781f
  15. 27 Oct, 2020 1 commit
  16. 24 Oct, 2020 1 commit
    • Sergey Pepyakin's avatar
      Runtime API for checking validation outputs (#1842) · 4a17a2bc
      Sergey Pepyakin authored
      * annoying whitespaces
      
      * update guide
      
      Add `CheckValidationOutputs` runtime api and also change the
      candidate-validation stuff
      
      * promote ValidationOutputs to global primitives
      
      i.e. move it from node specific primitives to global v1 primitives. This
      will be needed when we share it later in the runtime inclusion module
      
      * refactor acceptance checks in the inclusion module
      
      factor out the common code to share it during the block inclusion and
      for the forthcoming CheckValidationOutputs runtime api.
      
      Also note that the acceptance criteria was updated to incorporate checks
      that exist now in candidate-validation
      
      * plumb the runtime api outside
      
      * extract validation_data from ValidationOutputs
      
      * use runtime-api to check validation outputs
      
      apart from that refactor, update docs and tidy a bit
      
      * Update the maxium code size
      
      This is to fix a test that performs an upgrade.
      4a17a2bc
  17. 01 Oct, 2020 1 commit
    • Andronik Ordian's avatar
      implement remaining subsystem metrics (#1770) · 5bb296bf
      Andronik Ordian authored
      * overseer metrics: messages relayed
      
      * provisioner metrics: cosmetic changes
      
      * candidate selection metrics: cosmetic changes
      
      * availability bitfields metrics
      
      * availability distribution metrics
      
      * PoV distribution metrics
      
      * statement-distribution: small simplification
      
      * statement-distribution: extract log target into a const
      
      * statement-distribution: metrics
      
      * address review nits
      5bb296bf
  18. 29 Sep, 2020 1 commit
  19. 10 Sep, 2020 1 commit
  20. 08 Sep, 2020 1 commit
    • Peter Goodspeed-Niklaus's avatar
      implement candidate selection subsystem (#1645) · 6c874ed4
      Peter Goodspeed-Niklaus authored
      
      
      * choose the straightforward candidate selection algorithm for now
      
      * add draft implementation of candidate selection
      
      * fix typo in summary
      
      * more properly report misbehaving collators
      
      * describe how CandidateSelection subsystem becomes aware of candidates
      
      * revise candidate selection / collator protocol interaction pattern
      
      * implement rest of candidate selection per the guide
      
      * review: resolve nits
      
      * start writing test suite, harness
      
      * implement first test
      
      * add second test
      
      * implement third test
      
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      6c874ed4