- 18 Mar, 2021 2 commits
-
-
Bastian Köcher authored
-
Robert Klotzner authored
* Introduce collation fetching protocol also move to mod.rs * Allow `PeerId`s in requests to network bridge. * Fix availability distribution tests. * Move CompressedPoV to primitives. * Request based collator protocol: validator side - Missing: tests - Collator side - don't connect, if not connected * Fixes. * Basic request based collator side. * Minor fix on collator side. * Don't connect in requests in collation protocol. Also some cleanup. * Fix PoV distribution * Bump substrate * Add back metrics + whitespace fixes. * Add back missing spans. * More cleanup. * Guide update. * Fix tests * Handle results in tests. * Fix weird compilation issue. * Add missing ) * Get rid of dead code. * Get rid of redundant import. * Fix runtime build. * Cleanup. * Fix wasm build. * Format fixes. Thanks @andronik !
-
- 14 Mar, 2021 1 commit
-
-
asynchronous rob authored
* improve backing/provisioner spans * span for collation requests * add para_id to unbacked candidate spans * differentiate validation-construction and find-assignment in selection * better find-assignment spans * organize unbacked-candidate spans directly under job root * Update node/core/provisioner/src/lib.rs Co-authored-by:
Andronik Ordian <write@reusable.software> Co-authored-by:
-
- 19 Feb, 2021 1 commit
-
-
Bernhard Schuster authored
* feat/jaeger: more spans, more stages Stage numbers are still arbitrarily picked. * feat/jaeger: additional spans * chore/spellcheck: improve the dictionary * fix/jaeger JaegerSpan -> jaeger::Span
-
- 17 Feb, 2021 1 commit
-
-
Bernhard Schuster authored
* refactor/reputation: unify the values used * chore/rep: rename Annoy* to Cost*, make duplicate message Cost*Repeated * fix/reputation: lost and found, convert at the boundary to substrate * refactor/rep: move conversion to base reputation one level down, left conversions * fix/rep: order of magnitude adjustments Thanks pierre! * remove spaces * chore/rep: give rationale for order of magnitude * refactor/rep: move UnifiedReputationChange to separate file * fix/rep: order of magnitudes correction
-
- 14 Feb, 2021 1 commit
-
-
Bastian Köcher authored
* Notify collators about seconded collation This pr adds functionality to inform a collator that its collation was seconded by a parachain validator. Before this signed statement was only gossiped over the validation substream. Now, we explicitly send the seconded statement to the collator after it was validated successfully. Besides that it changes the `CollatorFn` to return an optional result sender that is informed when the build collation was seconded by a parachain validator. * Add test * Make sure we only send `Seconded` statements * Make sure we only receive valid statements * Review feedback
-
- 04 Feb, 2021 1 commit
-
-
Sergey Pepyakin authored
* Implement `Debug` manually for CandidateHash This will make candidate hashes printed consistently without the `CandidateHash(` and `)` decorations. * Do not print CompressedPov's guts It can be overwhelming. Better just use the size. * Log when candidate is generated * Print para_id and candidate_hash upon receiving a collation
-
- 03 Feb, 2021 1 commit
-
-
Robert Klotzner authored
* Move NetworkBridgeEvent to subsystem::messages. It is not protocol related at all, it is in fact only part of the subsystem communication as it gets wrapped into messages of each subsystem. * Request/response infrastructure is taking shape. WIP: Does not compile. * Multiplexer variant not supported by Rusts type system. * request_response::request type checks. * Cleanup. * Minor fixes for request_response. * Implement request sending + move multiplexer. Request multiplexer is moved to bridge, as there the implementation is more straight forward as we can specialize on `AllMessages` for the multiplexing target. Sending of requests is mostly complete, apart from a few `From` instances. Receiving is also almost done, initializtion needs to be fixed and the multiplexer needs to be invoked. * Remove obsolete multiplexer. * Initialize bridge with multiplexer. * Finish generic request sending/receiving. Subsystems are now able to receive and send requests and responses via the overseer. * Doc update. * Fixes. * Link issue for not yet implemented code. * Fixes suggested by @ordian - thanks! - start encoding at 0 - don't crash on zero protocols - don't panic on not yet implemented request handling * Update node/network/protocol/src/request_response/v1.rs Use index 0 instead of 1. Co-authored-by:
Pierre Krieger <pierre.krieger1708@gmail.com>
* Update node/network/bridge/src/multiplexer.rs
Remove redundant import.
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
* Stop doing tracing from within `From` instance.
Thanks for the catch @tomaka
!
* Get rid of redundant import.
* Formatting cleanup.
* Fix tests.
* Add link to issue.
* Clarify comments some more.
* Fix tests.
* Formatting fix.
* tabs
* Fix link
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> * Use map_err. Co-authored-by:
Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by:
-
- 02 Feb, 2021 2 commits
-
-
Sergey Pepyakin authored
-
Bastian Köcher authored
-
- 21 Jan, 2021 1 commit
-
-
Bastian Köcher authored
* Compress the PoV block before sending it over the network This pr changes the way we send PoV blocks over the network. We now compress the PoV block before it is send over the network. This should reduce the size significant for PoVs which contain the runtime WASM for example. * Preallocate 1KB * Try something.. * Switch to zstd and some renamings * Make compression/decompression fail in browsers * Use some sane maximum value * Update roadmap/implementers-guide/src/types/network.md Co-authored-by:
Robert Habermeier <rphmeier@gmail.com>
-
- 17 Jan, 2021 1 commit
-
-
Bastian Köcher authored
* Improve logging to make debugging parachains easier This pr should make debugging parachains easier, by printing more information about the validation process. *
🤦 * moare * Convert to debug
-
- 05 Jan, 2021 1 commit
-
-
Bastian Köcher authored
* Add one Jaeger span per relay parent This adds one Jaeger span per relay parent, instead of always creating new spans per relay parent. This should improve the UI view, because subsystems are now grouped below one common span. * Fix doc tests * Replace `PerLeaveSpan` to `PerLeafSpan` * More renaming * Moare * Update node/subsystem/src/lib.rs Co-authored-by:
-
- 17 Dec, 2020 1 commit
-
-
Andronik Ordian authored
* refactor View to include finalized_number * guide: update the NetworkBridge on BlockFinalized * av-store: fix the tests * actually fix tests * grumbles * ignore macro doctest * use Hash::repeat_bytes more consistently * broadcast empty leaves updates as well * fix issuing view updates on empty leaves updates
-
- 11 Dec, 2020 1 commit
-
-
Bernhard Schuster authored
Co-authored-by:
Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com> Co-authored-by:
Bastian Köcher <bkchr@users.noreply.github.com>
-
- 23 Nov, 2020 1 commit
-
-
Andronik Ordian authored
* send_message should not return an error * Apply suggestions from code review Co-authored-by:
-
- 20 Nov, 2020 2 commits
-
-
Peter Goodspeed-Niklaus authored
* reexport prometheus-super for ease of use of other subsystems * add some prometheus timers for collation generation subsystem * add timing metrics to av-store * add metrics to candidate backing * add timing metric to bitfield signing * add timing metrics to candidate selection * add timing metrics to candidate-validation * add timing metrics to chain-api * add timing metrics to provisioner * add timing metrics to runtime-api * add timing metrics to availability-distribution * add timing metrics to bitfield-distribution * add timing metrics to collator protocol: collator side * add timing metrics to collator protocol: validator side * fix candidate validation test failures * add timing metrics to pov distribution * add timing metrics to statement-distribution * use substrate_prometheus_endpoint prometheus reexport instead of prometheus_super * don't include JOB_DELAY in bitfield-signing metrics * give adder-collator ability to easily export its genesis-state and validation code * wip: adder-collator pushbutton script * don't attempt to register the adder-collator automatically Instead, get these values with ```sh target/release/adder-collator export-genesis-state target/release/adder-collator export-genesis-wasm ``` And then register the parachain on https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944#/explorer To collect prometheus data, after running the script, create `prometheus.yml` per the instructions at https://www.notion.so/paritytechnologies/Setting-up-Prometheus-locally-835cb3a9df7541a781c381006252b5ff and then run: ```sh docker run -v `pwd`/prometheus.yml:/etc/prometheus/prometheus.yml:z --network host prom/prometheus ``` Demonstrates that data makes it across to prometheus, though it is likely to be useful in the future to tweak the buckets. * Update parachain/test-parachains/adder/collator/src/cli.rs Co-authored-by:
-
Peter Goodspeed-Niklaus authored
* drop in tracing to replace log * add structured logging to trace messages * add structured logging to debug messages * add structured logging to info messages * add structured logging to warn messages * add structured logging to error messages * normalize spacing and Display vs Debug * add instrumentation to the various 'fn run' * use explicit tracing module throughout * fix availability distribution test * don't double-print errors * remove further redundancy from logs * fix test errors * fix more test errors * remove unused kv_log_macro * fix unused variable * add tracing spans to collation generation * add tracing spans to av-store * add tracing spans to backing * add tracing spans to bitfield-signing * add tracing spans to candidate-selection * add tracing spans to candidate-validation * add tracing spans to chain-api * add tracing spans to provisioner * add tracing spans to runtime-api * add tracing spans to availability-distribution * add tracing spans to bitfield-distribution * add tracing spans to network-bridge * add tracing spans to collator-protocol * add tracing spans to pov-distribution * add tracing spans to statement-distribution * add tracing spans to overseer * cleanup
-
- 05 Nov, 2020 1 commit
-
-
Andronik Ordian authored
* backing: extract log target * bitfield-signing: extract log target * utils: fix a typo * provisioner: extract log target * candidate selection: remove unused error variant * bitfield-distribution: change the return type of run * pov-distribution: extract log target * collator-protocol: simplify runtime request * collation-generation: do not exit early on error * collation-generation: do not exit on double init * collator-protocol: do not exit on errors and rename LOG_TARGET * collator-protocol: a workaround for ununused imports warning * Update node/network/bitfield-distribution/src/lib.rs * collation-generation: elevate warn! to error! * collator-protocol: fix imports * post merge fix * fix compilation
-
- 03 Nov, 2020 1 commit
-
-
Bastian Köcher authored
* Parachain improvements - Set the parachains configuration in Rococo genesis - Don't stop the overseer when a subsystem job is stopped - Several small code changes * Remove unused functionality * Return error from the runtime instead of printing it * Apply suggestions from code review Co-authored-by:
Sergei Shulepov <sergei@parity.io> Co-authored-by:
-
- 28 Oct, 2020 1 commit
-
-
Peter Goodspeed-Niklaus authored
* start working on building the real overseer Unfortunately, this fails to compile right now due to an upstream failure to compile which is probably brought on by a recent upgrade to rustc v1.47. * fill in AllSubsystems internal constructors * replace fn make_metrics with Metrics::attempt_to_register * update to account for #1740 * remove Metrics::register, rename Metrics::attempt_to_register * add 'static bounds to real_overseer type params * pass authority_discovery and network_service to real_overseer It's not straightforwardly obvious that this is the best way to handle the case when there is no authority discovery service, but it seems to be the best option available at the moment. * select a proper database configuration for the availability store db * use subdirectory for av-store database path * apply Basti's patch which avoids needing to parameterize everything on Block * simplify path extraction * get all tests to compile * Fix Prometheus double-registry error for debugging purposes, added this to node/subsystem-util/src/lib.rs:472-476: ```rust Some(registry) => Self::try_register(registry).map_err(|err| { eprintln!("PrometheusError calling {}::register: {:?}", std::any::type_name::<Self>(), err); err }), ``` That pointed out where the registration was failing, which led to this fix. The test still doesn't pass, but it now fails in a new and different way! * authorities must have authority discovery, but not necessarily overseer handlers * fix broken SpawnedSubsystem impls detailed logging determined that using the `Box::new` style of future generation, the `self.run` method was never being called, leading to dropped receivers / closed senders for those subsystems, causing the overseer to shut down immediately. This is not the final fix needed to get things working properly, but it's a good start. * use prometheus properly Prometheus lets us register simple counters, which aren't very interesting. It also allows us to register CounterVecs, which are. With a CounterVec, you can provide a set of labels, which can later be used to filter the counts. We were using them wrong, though. This pattern was repeated in a variety of places in the code: ```rust // panics with an cardinality mismatch let my_counter = register(CounterVec::new(opts, &["succeeded", "failed"])?, registry)?; my_counter.with_label_values(&["succeeded"]).inc() ``` The problem is that the labels provided in the constructor are not the set of legal values which can be annotated, but a set of individual label names which can have individual, arbitrary values. This commit fixes that. * get av-store subsystem to actually run properly and not die on first signal * typo fix: incomming -> incoming * don't disable authority discovery in test nodes * Fix rococo-v1 missing session keys * Update node/core/av-store/Cargo.toml * try dummying out av-store on non-full-nodes * overseer and subsystems are required only for full nodes * Reduce the amount of warnings on browser target * Fix two more warnings * InclusionInherent should actually have an Inherent module on rococo * Ancestry: don't return genesis' parent hash * Update Cargo.lock * fix broken test * update test script: specify chainspec as script argument * Apply suggestions from code review Co-authored-by:Bastian Köcher <git@kchr.de> Co-authored-by:
Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Robert Habermeier <robert@Roberts-MBP.lan1> Co-authored-by:
Max Inden <mail@max-inden.de>
-
- 06 Oct, 2020 2 commits
-
-
Andronik Ordian authored
* collator: fix a typo * collator: fix more typos * collator: fix even more typos
-
Bastian Köcher authored
This pr changes the collator interface function to return an optional collation instead of a collation. This is required as the parachain itself can fail to generate a valid collation for various reason. Now if the collation fails it will return `None`. Besides that the pr adds some `RuntimeDebug` derive for `ValidationData` and removes some whitespaces.
-
- 29 Sep, 2020 1 commit
-
-
Andronik Ordian authored
-
- 28 Sep, 2020 1 commit
-
-
Fedor Sakharov authored* Metrics * Dont punish late collations * Fix metrics * Update node/network/collator-protocol/src/lib.rs Co-authored-by:
-
- 10 Sep, 2020 1 commit
-
-
Fedor Sakharov authored
* WIP * The initial implementation of the collator side. * Improve comments * Multiple collation requests * Add more tests and comments to validator side * Add comments, remove dead code * Apply suggestions from code review Co-authored-by:
-
