- Aug 18, 2023
-
-
asynchronous rob authored
* inclusion emulator logic for asynchronous backing (#4790) * initial stab at candidate_context * fmt * docs & more TODOs * some cleanups * reframe as inclusion_emulator * documentations yes * update types * add constraint modifications * watermark * produce modifications * v2 primitives: re-export all v1 for consistency * vstaging primitives * emulator constraints: handle code upgrades * produce outbound HRMP modifications * stack. * method for applying modifications * method just for sanity-checking modifications * fragments produce modifications, not prospectives * make linear * add some TODOs * remove stacking; handle code upgrades * take `fragment` private * reintroduce stacking. * fragment constructor * add TODO * allow validating fragments against future constraints * docs * relay-parent number and min code size checks * check code upgrade restriction * check max hrmp per candidate * fmt * remove GoAhead logic because it wasn't helpful * docs on code upgrade failure * test stacking * test modifications against constraints * fmt * test fragments * descending or duplicate test * fmt * remove unused imports in vstaging * wrong primitives * spellcheck * Runtime changes for Asynchronous Backing (#4786) * inclusion: utility for allowed relay-parents * inclusion: use prev number instead of prev hash * track most recent context of paras * inclusion: accept previous relay-parents * update dmp advancement rule for async backing * fmt * add a comment about validation outputs * clean up a couple of TODOs * weights * fix weights * fmt * Resolve dmp todo * Restore inclusion tests * Restore paras_inherent tests * MostRecentContext test * Benchmark for new paras dispatchable * Prepare check_validation_outputs for upgrade * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs * cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs * Implementers guide changes * More tests for allowed relay parents * Add a github issue link * Compute group index based on relay parent * Storage migration * Move allowed parents tracker to shared * Compile error * Get group assigned to core at the next block * Test group assignment * fmt * Error instead of panic * Update guide * Extend doc-comment * Update runtime/parachains/src/shared.rs Co-authored-by:
Robert Habermeier <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]> Co-authored-by:
Parity Bot <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]> * Prospective Parachains Subsystem (#4913) * docs and skeleton * subsystem skeleton * main loop * fragment tree basics & fmt * begin fragment trees & view * flesh out more of view update logic * further flesh out update logic * some refcount functions for fragment trees * add fatal/non-fatal errors * use non-fatal results * clear up some TODOs * ideal format for scheduling info * add a bunch of TODOs * some more fluff * extract fragment graph to submodule * begin fragment graph API * trees, not graphs * improve docs * scope and constructor for trees * add some test TODOs * limit max ancestors and store constraints * constructor * constraints: fix bug in HRMP watermarks * fragment tree population logic * set::retain * extract population logic * implement add_and_populate * fmt * add some TODOs in tests * implement child-selection * strip out old stuff based on wrong assumptions * use fatality * implement pruning * remove unused ancestor constraints * fragment tree instantiation * remove outdated comment * add message/request types and skeleton for handling * fmt * implement handle_candidate_seconded * candidate storage: handle backed * implement handle_candidate_backed * implement answer_get_backable_candidate * remove async where not needed * implement fetch_ancestry * add logic for run_iteration * add some docs * remove global allow(unused), fix warnings * make spellcheck happy (despite English) * fmt * bump Cargo.lock * replace tracing with gum * introduce PopulateFrom trait * implement GetHypotheticalDepths * revise docs slightly * first fragment tree scope test * more scope tests * test add_candidate * fmt * test retain * refactor test code * test populate is recursive * test contiguity of depth 0 is maintained * add_and_populate tests * cycle tests * remove PopulateFrom trait * fmt * test hypothetical depths (non-recursive) * have CandidateSeconded return membership * tree membership requests * Add a ProspectiveParachainsSubsystem struct * add a staging API for base constraints * add a `From` impl * add runtime API for staging_validity_constraints * implement fetch_base_constraints * implement `fetch_upcoming_paras` * remove reconstruction of candidate receipt; no obvious usecase * fmt * export message to broader module * remove last TODO * correctly export * fix compilation and add GetMinimumRelayParent request * make provisioner into a real subsystem with proper mesage bounds * fmt * fix ChannelsOut in overseer test * fix overseer tests * fix again * fmt * Integrate prospective parachains subsystem into backing: Part 1 (#5557) * BEGIN ASYNC candidate-backing CHANGES * rename & document modes * answer prospective validation data requests * GetMinimumRelayParents request is now plural * implement an implicit view utility for backing subsystems * implicit-view: get allowed relay parents * refactorings and improvements to implicit view * add some TODOs for tests * split implicit view updates into 2 functions * backing: define State to prepare for functional refactor * add some docs * backing: implement bones of new leaf activation logic * backing: create per-relay-parent-states * use new handle_active_leaves_update * begin extracting logic from CandidateBackingJob * mostly extract statement import from job logic * handle statement imports outside of job logic * do some TODO planning for prospective parachains integration * finish rewriting backing subsystem in functional style * add prospective parachains mode to relay parent entries * fmt * add a RejectedByProspectiveParachains error * notify prospective parachains of seconded and backed candidates * always validate candidates exhaustively in backing. * return persisted_validation_data from validation * handle rejections by prospective parachains * implement seconding sanity check * invoke validate_and_second * Alter statement table to allow multiple seconded messages per validator * refactor backing to have statements carry PVD * clean up all warnings * Add tests for implicit view * Improve doc comments * Prospective parachains mode based on Runtime API version * Add a TODO * Rework seconding_sanity_check * Iterate over responses * Update backing tests * collator-protocol: load PVD from runtime * Fix validator side tests * Update statement-distribution to fetch PVD * Fix statement-distribution tests * Backing tests with prospective paras #1 * fix per_relay_parent pruning in backing * Test multiple leaves * Test seconding sanity check * Import statement order Before creating an entry in `PerCandidateState` map wait for the approval from the prospective parachains * Add a test for correct state updates * Second multiple candidates per relay parent test * Add backing tests with prospective paras * Second more than one test without prospective paras * Add a test for prospective para blocks * Update malus * typos Co-authored-by:
Chris Sosnin <[email protected]> * Track occupied depth in backing per parachain (#5778) * provisioner: async backing changes (#5711) * Provisioner changes for async backing * Select candidates based on prospective paras mode * Revert naming * Update tests * Update TODO comment * review * provisioner: async backing changes (#5711) * Provisioner changes for async backing * Select candidates based on prospective paras mode * Revert naming * Update tests * Update TODO comment * review * fmt * Network bridge changes for asynchronous backing + update subsystems to handle versioned packets (#5991) * BEGIN STATEMENT DISTRIBUTION WORK create a vstaging network protocol which is the same as v1 * mostly make network bridge amenable to vstaging * network-bridge: fully adapt to vstaging * add some TODOs for tests * fix fallout in bitfield-distribution * bitfield distribution tests + TODOs * fix fallout in gossip-support * collator-protocol: fix message fallout * collator-protocol: load PVD from runtime * add TODO for vstaging tests * make things compile * set used network protocol version using a feature * fmt * get approval-distribution building * fix approval-distribution tests * spellcheck * nits * approval distribution net protocol test * bitfield distribution net protocol test * Revert "collator-protocol: fix message fallout" This reverts commit 07cc887303e16c6b3843ecb25cdc7cc2080e2ed1. * Network bridge tests Co-authored-by:
Chris Sosnin <[email protected]> * remove max_pov_size requirement from prospective pvd request (#6014) * remove max_pov_size requirement from prospective pvd request * fmt * Extract legacy statement distribution to its own module (#6026) * add compatibility type to v2 statement distribution message * warning cleanup * handle compatibility layer for v2 * clean up an unimplemented!() block * circulate statements based on version * extract legacy v1 code into separate module * remove unimplemented * clean up naming of from_requester/responder * remove TODOs * have backing share seconded statements with PVD * fmt * fix warning * Quick fix unused warning for not yet implemented/used staging messages. * Fix network bridge test * Fix wrong merge. We now have 23 subsystems (network bridge split + prospective parachains) Co-authored-by:
Robert Klotzner <[email protected]> * Version 3 is already live. * Fix tests (#6055) * Fix backing tests * Fix warnings. * fmt * collator-protocol: asynchronous backing changes (#5740) * Draft collator side changes * Start working on collations management * Handle peer's view change * Versioning on advertising * Versioned collation fetching request * Handle versioned messages * Improve docs for collation requests * Add spans * Add request receiver to overseer * Fix collator side tests * Extract relay parent mode to lib * Validator side draft * Add more checks for advertisement * Request pvd based on async backing mode * review * Validator side improvements * Make old tests green * More fixes * Collator side tests draft * Send collation test * fmt * Collator side network protocol versioning * cleanup * merge artifacts * Validator side net protocol versioning * Remove fragment tree membership request * Resolve todo * Collator side core state test * Improve net protocol compatibility * Validator side tests * more improvements * style fixes * downgrade log * Track implicit assignments * Limit the number of seconded candidates per para * Add a sanity check * Handle fetched candidate * fix tests * Retry fetch * Guard against dequeueing while already fetching * Reintegrate connection management * Timeout on advertisements * fmt * spellcheck * update tests after merge * validator assignment fixes for backing and collator protocol (#6158) * Rename depth->ancestry len in tests * Refactor group assignments * Remove implicit assignments * backing: consider occupied core assignments * Track a single para on validator side * Refactor prospective parachains mode request (#6179) * Extract prospective parachains mode into util * Skip activations depending on the mode * backing: don't send backed candidate to provisioner (#6185) * backing: introduce `CanSecond` request for advertisements filtering (#6225) * Drop BoundToRelayParent * draft changes * fix backing tests * Fix genesis ancestry * Fix validator side tests * more tests * cargo generate-lockfile * Implement `StagingValidityConstraints` Runtime API method (#6258) * Implement StagingValidityConstraints * spellcheck * fix ump params * Update hrmp comment * Introduce ump per candidate limit * hypothetical earliest block * refactor primitives usage * hypothetical earliest block number test * fix build * Prepare the Runtime for asynchronous backing upgrade (#6287) * Introduce async backing params to runtime config * fix cumulus config * use config * finish runtimes * Introduce new staging API * Update collator protocol * Update provisioner * Update prospective parachains * Update backing * Move async backing params lower in the config * make naming consistent * misc * Use real prospective parachains subsystem (#6407) * Backport `HypotheticalFrontier` into the feature branch (#6605) * implement more general HypotheticalFrontier * fmt * drop unneeded request Co-authored-by:
Robert Habermeier <[email protected]> * Resolve todo about legacy leaf activation (#6447) * fix bug/warning in handling membership answers * Remove `HypotheticalDepthRequest` in favor of `HypotheticalFrontierRequest` (#6521) * Remove `HypotheticalDepthRequest` for `HypotheticalFrontierRequest` * Update tests * Fix (removed wrong docstring) * Fix can_second request * Patch some dead_code errors --------- Co-authored-by:
Chris Sosnin <[email protected]> * Async Backing: Send Statement Distribution "Backed" messages (#6634) * Backing: Send Statement Distribution "Backed" messages Closes #6590. **TODO:** - [ ] Adjust tests * Fix compile errors * (Mostly) fix tests * Fix comment * Fix test and compile error * Test that `StatementDistributionMessage::Backed` is sent * Fix compile error * Fix some clippy errors * Add prospective parachains subsystem tests (#6454) * Add prospective parachains subsystem test * Add `should_do_no_work_if_async_backing_disabled_for_leaf` test * Implement `activate_leaf` helper, up to getting ancestry * Finish implementing `activate_leaf` * Small refactor in `activate_leaf` * Get `CandidateSeconded` working * Finish `send_candidate_and_check_if_found` test * Refactor; send more leaves & candidates * Refactor test * Implement `check_candidate_parent_leaving_view` test * Start work on `check_candidate_on_multiple_forks` test * Don’t associate specific parachains with leaf * Finish `correctly_updates_leaves` test * Fix cycle due to reused head data * Fix `check_backable_query` test * Fix `check_candidate_on_multiple_forks` test * Add `check_depth_and_pvd_queries` test * Address review comments * Remove TODO * add a new index for output head data to candidate storage * Resolve test TODOs * Fix compile errors * test candidate storage pruning, make sure new index is cleaned up --------- Co-authored-by:
Robert Habermeier <[email protected]> * Node-side metrics for asynchronous backing (#6549) * Add metrics for `prune_view_candidate_storage` * Add metrics for `request_unblocked_collations` * Fix docstring * Couple fixes from review comments * Fix `check_depth_query` test * inclusion-emulator: mirror advancement rule check (#6361) * inclusion-emulator: mirror advancement rule check * fix build * prospective-parachains: introduce `backed_in_path_only` flag for advertisements (#6649) * Introduce `backed_in_path_only` flag for depth request * fmt * update doc comment * fmt * Add async-backing zombienet tests (#6314) * Async backing: impl guide for statement distribution (#6738) Co-authored-by:
Bradley Olson <[email protected]> Co-authored-by:
alexgparity <[email protected]> * Asynchronous backing statement distribution: Take III (#5999) * add notification types for v2 statement-distribution * improve protocol docs * add empty vstaging module * fmt * add backed candidate packet request types * start putting down structure of new logic * handle activated leaf * some sanity-checking on outbound statements * fmt * update vstaging share to use statements with PVD * tiny refactor, candidate_hash location * import local statements * refactor statement import * first stab at broadcast logic * fmt * fill out some TODOs * start on handling incoming * split off session info into separate map * start in on a knowledge tracker * address some grumbles * format * missed comment * some docs for direct * add note on slashing * amend * simplify 'direct' code * finish up the 'direct' logic * add a bunch of tests for the direct-in-group logic * rename 'direct' to 'cluster', begin a candidate_entry module * distill candidate_entry * start in on a statement-store module * some utilities for the statement store * rewrite 'send_statement_direct' using new tools * filter sending logic on peers which have the relay-parent in their view. * some more logic for handling incoming statements * req/res: BackedCandidatePacket -> AttestedCandidate + tweaks * add a `validated_in_group` bitfield to BackedCandidateInventory * BackedCandidateInventory -> Manifest * start in on requester module * add outgoing request for attested candidate * add a priority mechanism for requester * some request dispatch logic * add seconded mask to tagged-request * amend manifest to hold group index * handle errors and set up scaffold for response validation * validate attested candidate responses * requester -> requests * add some utilities for manipulating requests * begin integrating requester * start grid module * tiny * refactor grid topology to expose more info to subsystems * fix grid_topology test * fix overseer test * implement topology group-based view construction logic * fmt * flesh out grid slightly more * add indexed groups utility * integrate Groups into per-session info * refactor statement store to borrow Groups * implement manifest knowledge utility * add a test for topology setup * don't send to group members * test for conflicting manifests * manifest knowledge tests * fmt * rename field * garbage collection for grid tracker * routines for finding correct/incorrect advertisers * add manifest import logic * tweak naming * more tests for manifest import * add comment * rework candidates into a view-wide tracker * fmt * start writing boilerplate for grid sending * fmt * some more group boilerplate * refactor handling of topology and authority IDs * fmt * send statements directly to grid peers where possible * send to cluster only if statement belongs to cluster * improve handling of cluster statements * handle incoming statements along the grid * API for introduction of candidates into the tree * backing: use new prospective parachains API * fmt prospective parachains changes * fmt statement-dist * fix condition * get ready for tracking importable candidates * prospective parachains: add Cow logic * incomplete and complete hypothetical candidates * remove keep_if_unneeded * fmt * implement more general HypotheticalFrontier * fmt, cleanup * add a by_parent_hash index to candidate tracker * more framework for future code * utilities for getting all hypothetical candidates for frontier * track origin in statement store * fmt * requests should return peer * apply post-confirmation reckoning * flesh out import/announce/circulate logic on new statements * adjust * adjust TODO comment * fix backing tests * update statement-distribution to use new indexedvec * fmt * query hypothetical candidates * implement `note_importable_under` * extract common utility of fragment tree updates * add a helper function for getting statements unknown by backing * import fresh statements to backing * send announcements and acknowledgements over grid * provide freshly importable statements also avoid tracking backed candidates in statement distribution * do not issue requests on newly importable candidates * add TODO for later when confirming candidate * write a routine for handling backed candidate notifications * simplify grid substantially * add some test TODOs * handle confirmed candidates & grid announcements * finish implementing manifest handling, including follow up statements * send follow-up statements when acknowledging freshly backed * fmt * handle incoming acknowledgements * a little DRYing * wire up network messages to handlers * fmt * some skeleton code for peer view update handling * more peer view skeleton stuff * Fix async backing statement distribution tests (#6621) * Fix compile errors in tests * Cargo fmt * Resolve some todos in async backing statement-distribution branch (#6482) * Implement `remove_by_relay_parent` * Extract `minimum_votes` to shared primitives. * Add `can_send_statements_received_with_prejudice` test * Fix test * Update docstrings * Cargo fmt * Fix compile error * Fix compile errors in tests * Cargo fmt * Add module docs; write `test_priority_ordering` (first draft) * Fix `test_priority_ordering` * Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority` * Address review comments * Remove `Entry::get_mut` * fix test compilation * add a TODO for a test * clean up a couple of TODOs * implement sending pending cluster statements * refactor utility function for sending acknowledgement and statements * mostly implement catching peers up via grid * Fix clippy error * alter grid to track all pending statements * fix more TODOs and format * tweak a TODO in requests * some logic for dispatching requests * fmt * skeleton for response receiving * Async backing statement distribution: cluster tests (#6678) * Add `pending_statements_set_when_receiving_fresh_statements` * Add `pending_statements_updated_when_sending_statements` test * fix up * fmt * update TODO * rework seconded mask in requests * change doc * change unhandledresponse not to borrow request manager * only accept responses sufficient to back * finish implementing response handling * extract statement filter to protocol crate * rework requests: use statement filter in network protocol * dispatch cluster requests correctly * rework cluster statement sending * implement request answering * fmt * only send confirmed candidate statement messages on unified relay-parent * Fix Tests In Statement Distribution Branch * Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * clean up some review comments * clean up warnings * Async backing statement distribution: grid tests (#6673) * Add `manifest_import_returns_ok_true` test * cargo fmt * Add pending_communication_receiving_manifest_on_confirmed_candidate * Add `senders_can_provide_manifests_in_acknowledgement` test * Add a couple of tests for pending statements * Add `pending_statements_cleared_when_sending` test * Add `pending_statements_respect_remote_knowledge` test * Refactor group creation in tests * Clarify docs * Address some review comments * Make some clarifications * Fix post-merge errors * Clarify test `senders_can_provide_manifests_in_acknowledgement` * Try writing `pending_statements_are_updated_after_manifest_exchange` * Document "seconding limit" and `reject_overflowing_manifests` test * Test that seconding counts are not updated for validators on error * Fix tests * Fix manifest exchange test * Add more tests in `requests.rs` (#6707) This resolves remaining TODOs in this file. * remove outdated inventory terminology * Async backing statement distribution: `Candidates` tests (#6658) * Async Backing: Fix clippy errors in statement distribution branch (#6720) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * Async Backing: Fix clippy errors in statement distribution branch * Fix some more clippy lints * add tests module * fix warnings in existing tests * create basic test harness * create a test state struct * fmt * create empty cluster & grid modules for tests * some TODOs for cluster test suite * describe test-suite for grid logic * describe request test suite * fix seconding-limit bug * Remove extraneous `pub` This somehow made it into my clippy PR. * Fix some test compile warnings * Remove some unneeded `allow`s * adapt some new test helpers from Marcin * add helper for activating a gossip topology * add utility for signing statements * helpers for connecting/disconnecting peers * round out network utilities * fmt * fix bug in initializing validator-meta * fix compilation * implement first cluster test * TODOs for incoming request tests * Remove unneeded `make_committed_candidate` helper * fmt * some more tests for cluster * add a TODO about grid senders * integrate inbound req/res into test harness * polish off initial cluster test suite * keep introduce candidate request * fix tests after introduce candidate request * fmt * Add grid protocol to module docs * Fix comments * Test `backed_in_path_only: true` * Update node/network/protocol/src/lib.rs Co-authored-by:
Chris Sosnin <[email protected]> * Update node/network/protocol/src/request_response/mod.rs Co-authored-by:
Chris Sosnin <[email protected]> * Mark receiver with `vstaging` * validate grid senders based on manifest kind * fix mask_seconded/valid * fix unwanted-mask check * fix build * resolve todo on leaf mode * Unify protocol naming to vstaging * fmt, fix grid test after topology change * typo Co-authored-by:
Chris Sosnin <[email protected]> * address review * adjust comment, make easier to understand * Fix typo --------- Co-authored-by:
Marcin S <[email protected]> Co-authored-by:
Marcin S <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]> * miscellaneous fixes to make asynchronous backing work (#6791) * propagate network-protocol-staging feature * add feature to adder-collator as well * allow collation-generation of occupied cores * prospective parachains: special treatment for pending availability candidates * runtime: fetch candidates pending availability * lazily construct PVD for pending candidates * fix fallout in prospective parachains hypothetical/select_child * runtime: enact candidates when creating paras-inherent * make tests compile * test pending availability in the scope * add prospective parachains test * fix validity constraints leftovers * drop prints * Fix typos --------- Co-authored-by:
Chris Sosnin <[email protected]> Co-authored-by:
Marcin S <[email protected]> * Remove restart from test (#6840) * Async Backing: Statement Distribution Tests (#6755) * start on handling incoming * split off session info into separate map * start in on a knowledge tracker * address some grumbles * format * missed comment * some docs for direct * add note on slashing * amend * simplify 'direct' code * finish up the 'direct' logic * add a bunch of tests for the direct-in-group logic * rename 'direct' to 'cluster', begin a candidate_entry module * distill candidate_entry * start in on a statement-store module * some utilities for the statement store * rewrite 'send_statement_direct' using new tools * filter sending logic on peers which have the relay-parent in their view. * some more logic for handling incoming statements * req/res: BackedCandidatePacket -> AttestedCandidate + tweaks * add a `validated_in_group` bitfield to BackedCandidateInventory * BackedCandidateInventory -> Manifest * start in on requester module * add outgoing request for attested candidate * add a priority mechanism for requester * some request dispatch logic * add seconded mask to tagged-request * amend manifest to hold group index * handle errors and set up scaffold for response validation * validate attested candidate responses * requester -> requests * add some utilities for manipulating requests * begin integrating requester * start grid module * tiny * refactor grid topology to expose more info to subsystems * fix grid_topology test * fix overseer test * implement topology group-based view construction logic * fmt * flesh out grid slightly more * add indexed groups utility * integrate Groups into per-session info * refactor statement store to borrow Groups * implement manifest knowledge utility * add a test for topology setup * don't send to group members * test for conflicting manifests * manifest knowledge tests * fmt * rename field * garbage collection for grid tracker * routines for finding correct/incorrect advertisers * add manifest import logic * tweak naming * more tests for manifest import * add comment * rework candidates into a view-wide tracker * fmt * start writing boilerplate for grid sending * fmt * some more group boilerplate * refactor handling of topology and authority IDs * fmt * send statements directly to grid peers where possible * send to cluster only if statement belongs to cluster * improve handling of cluster statements * handle incoming statements along the grid * API for introduction of candidates into the tree * backing: use new prospective parachains API * fmt prospective parachains changes * fmt statement-dist * fix condition * get ready for tracking importable candidates * prospective parachains: add Cow logic * incomplete and complete hypothetical candidates * remove keep_if_unneeded * fmt * implement more general HypotheticalFrontier * fmt, cleanup * add a by_parent_hash index to candidate tracker * more framework for future code * utilities for getting all hypothetical candidates for frontier * track origin in statement store * fmt * requests should return peer * apply post-confirmation reckoning * flesh out import/announce/circulate logic on new statements * adjust * adjust TODO comment * fix backing tests * update statement-distribution to use new indexedvec * fmt * query hypothetical candidates * implement `note_importable_under` * extract common utility of fragment tree updates * add a helper function for getting statements unknown by backing * import fresh statements to backing * send announcements and acknowledgements over grid * provide freshly importable statements also avoid tracking backed candidates in statement distribution * do not issue requests on newly importable candidates * add TODO for later when confirming candidate * write a routine for handling backed candidate notifications * simplify grid substantially * add some test TODOs * handle confirmed candidates & grid announcements * finish implementing manifest handling, including follow up statements * send follow-up statements when acknowledging freshly backed * fmt * handle incoming acknowledgements * a little DRYing * wire up network messages to handlers * fmt * some skeleton code for peer view update handling * more peer view skeleton stuff * Fix async backing statement distribution tests (#6621) * Fix compile errors in tests * Cargo fmt * Resolve some todos in async backing statement-distribution branch (#6482) * Implement `remove_by_relay_parent` * Extract `minimum_votes` to shared primitives. * Add `can_send_statements_received_with_prejudice` test * Fix test * Update docstrings * Cargo fmt * Fix compile error * Fix compile errors in tests * Cargo fmt * Add module docs; write `test_priority_ordering` (first draft) * Fix `test_priority_ordering` * Move `insert_or_update_priority`: `Drop` -> `set_cluster_priority` * Address review comments * Remove `Entry::get_mut` * fix test compilation * add a TODO for a test * clean up a couple of TODOs * implement sending pending cluster statements * refactor utility function for sending acknowledgement and statements * mostly implement catching peers up via grid * Fix clippy error * alter grid to track all pending statements * fix more TODOs and format * tweak a TODO in requests * some logic for dispatching requests * fmt * skeleton for response receiving * Async backing statement distribution: cluster tests (#6678) * Add `pending_statements_set_when_receiving_fresh_statements` * Add `pending_statements_updated_when_sending_statements` test * fix up * fmt * update TODO * rework seconded mask in requests * change doc * change unhandledresponse not to borrow request manager * only accept responses sufficient to back * finish implementing response handling * extract statement filter to protocol crate * rework requests: use statement filter in network protocol * dispatch cluster requests correctly * rework cluster statement sending * implement request answering * fmt * only send confirmed candidate statement messages on unified relay-parent * Fix Tests In Statement Distribution Branch * Async Backing: Integrate `vstaging` of statement distribution into `lib.rs` (#6715) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * clean up some review comments * clean up warnings * Async backing statement distribution: grid tests (#6673) * Add `manifest_import_returns_ok_true` test * cargo fmt * Add pending_communication_receiving_manifest_on_confirmed_candidate * Add `senders_can_provide_manifests_in_acknowledgement` test * Add a couple of tests for pending statements * Add `pending_statements_cleared_when_sending` test * Add `pending_statements_respect_remote_knowledge` test * Refactor group creation in tests * Clarify docs * Address some review comments * Make some clarifications * Fix post-merge errors * Clarify test `senders_can_provide_manifests_in_acknowledgement` * Try writing `pending_statements_are_updated_after_manifest_exchange` * Document "seconding limit" and `reject_overflowing_manifests` test * Test that seconding counts are not updated for validators on error * Fix tests * Fix manifest exchange test * Add more tests in `requests.rs` (#6707) This resolves remaining TODOs in this file. * remove outdated inventory terminology * Async backing statement distribution: `Candidates` tests (#6658) * Async Backing: Fix clippy errors in statement distribution branch (#6720) * Integrate `handle_active_leaves_update` * Integrate `share_local_statement`/`handle_backed_candidate_message` * Start hooking up request/response flow * Finish hooking up request/response flow * Limit number of parallel requests in responder * Fix test compilation errors * Fix missing check for prospective parachains mode * Fix some more compile errors * Async Backing: Fix clippy errors in statement distribution branch * Fix some more clippy lints * add tests module * fix warnings in existing tests * create basic test harness * create a test state struct * fmt * create empty cluster & grid modules for tests * some TODOs for cluster test suite * describe test-suite for grid logic * describe request test suite * fix seconding-limit bug * Remove extraneous `pub` This somehow made it into my clippy PR. * Fix some test compile warnings * Remove some unneeded `allow`s * adapt some new test helpers from Marcin * add helper for activating a gossip topology * add utility for signing statements * helpers for connecting/disconnecting peers * round out network utilities * fmt * fix bug in initializing validator-meta * fix compilation * implement first cluster test * TODOs for incoming request tests * Remove unneeded `make_committed_candidate` helper * fmt * Hook up request sender * Add `valid_statement_without_prior_seconded_is_ignored` test * Fix `valid_statement_without_prior_seconded_is_ignored` test * some more tests for cluster * add a TODO about grid senders * integrate inbound req/res into test harness * polish off initial cluster test suite * keep introduce candidate request * fix tests after introduce candidate request * fmt * Add grid protocol to module docs * Remove obsolete test * Fix comments * Test `backed_in_path_only: true` * Update node/network/protocol/src/lib.rs Co-authored-by:
Chris Sosnin <[email protected]> * Update node/network/protocol/src/request_response/mod.rs Co-authored-by:
Chris Sosnin <[email protected]> * Mark receiver with `vstaging` * First draft of `ensure_seconding_limit_is_respected` test * validate grid senders based on manifest kind * fix mask_seconded/valid * fix unwanted-mask check * fix build * resolve todo on leaf mode * Unify protocol naming to vstaging * Fix `ensure_seconding_limit_is_respected` test * Start `backed_candidate_leads_to_advertisement` test * fmt, fix grid test after topology change * Send Backed notification * Finish `backed_candidate_leads_to_advertisement` test * Finish `peer_reported_for_duplicate_statements` test * Finish `received_advertisement_before_confirmation_leads_to_request` * Add `advertisements_rejected_from_incorrect_peers` test * Add `manifest_rejected_*` tests * Add `manifest_rejected_when_group_does_not_match_para` test * Add `local_node_sanity_checks_incoming_requests` test * Add `local_node_respects_statement_mask` test * Add tests where peer is reported for providing invalid signatures * Add `cluster_peer_allowed_to_send_incomplete_statements` test * Add `received_advertisement_after_backing_leads_to_acknowledgement` * Add `received_advertisement_after_confirmation_before_backing` test * peer_reported_for_advertisement_conflicting_with_confirmed_candidate * Add `peer_reported_for_not_enough_statements` test * Add `peer_reported_for_providing_statements_meant_to_be_masked_out` * Add `additional_statements_are_shared_after_manifest_exchange` * Add `grid_statements_imported_to_backing` test * Add `relay_parent_entering_peer_view_leads_to_advertisement` test * Add `advertisement_not_re_sent_when_peer_re_enters_view` test * Update node/network/statement-distribution/src/vstaging/tests/grid.rs Co-authored-by:
asynchronous rob <[email protected]> * Resolve TODOs, update test * Address unused code * Add check after every test for unhandled requests * Refactor (`make_dummy_leaf` and `handle_sent_request`) * Refactor (`make_dummy_topology`) * Minor refactor --------- Co-authored-by:
Robert Habermeier <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]> * Fix some clippy lints in tests * Async backing: minor fixes (#6920) * bitfield-distribution test * implicit view tests * Refactor parameters -> params * scheduler: update storage migration (#6963) * update scheduler migration * Adjust weight to account for storage read * Statement Distribution Guide Edits (#7025) * Statement distribution guide edits * Addressed Marcin's comments * Add attested candidate request retry timeouts (#6833) Co-authored-by:
Chris Sosnin <[email protected]> Co-authored-by:
asynchronous rob <[email protected]> Co-authored-by:
Robert Habermeier <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]> Fix async backing statement distribution tests (#6621) Resolve some todos in async backing statement-distribution branch (#6482) Fix clippy errors in statement distribution branch (#6720) * Async backing: add Prospective Parachains impl guide (#6933) Co-authored-by:
Bradley Olson <[email protected]> * Updates to Provisioner Guide for Async Backing (#7106) * Initial corrections and clarifications * Partial first draft * Finished first draft * Adding back wrongly removed test bit * fmt * Update roadmap/implementers-guide/src/node/utility/provisioner.md Co-authored-by:
Marcin S. <[email protected]> * Addressing comments * Reorganization * fmt --------- Co-authored-by:
Marcin S. <[email protected]> * fmt * Renaming Parathread Mentions (#7287) * Renaming parathreads * Renaming module to pallet * More updates * PVF: Refactor workers into separate crates, remove host dependency (#7253) * PVF: Refactor workers into separate crates, remove host dependency * Fix compile error * Remove some leftover code * Fix compile errors * Update Cargo.lock * Remove worker main.rs files I accidentally copied these from the other PR. This PR isn't intended to introduce standalone workers yet. * Address review comments * cargo fmt * Update a couple of comments * Update log targets * Update quote to 1.0.27 (#7280) Signed-off-by:
Oliver Tale-Yazdi <[email protected]> Co-authored-by: parity-processbot <> * pallets: implement `Default` for `GenesisConfig` in `no_std` (#7271) * pallets: implement Default for GenesisConfig in no_std This change is follow-up of: https://github.com/paritytech/substrate/pull/14108 It is a step towards: https://github.com/paritytech/substrate/issues/13334 * Cargo.lock updated * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * cli: enable BEEFY by default on test networks (#7293) We consider BEEFY mature enough to run by default on all nodes for test networks (Rococo/Wococo/Versi). Right now, most nodes are not running it since it's opt-in using --beefy flag. Switch to an opt-out model for test networks. Replace --beefy flag from CLI with --no-beefy and have BEEFY client start by default on test networks. Signed-off-by:
acatangiu <[email protected]> * runtime: past session slashing runtime API (#6667) * runtime/vstaging: unapplied_slashes runtime API * runtime/vstaging: key_ownership_proof runtime API * runtime/ParachainHost: submit_report_dispute_lost * fix key_ownership_proof API * runtime: submit_report_dispute_lost runtime API * nits * Update node/subsystem-types/src/messages.rs Co-authored-by:
Marcin S. <[email protected]> * revert unrelated fmt changes * post merge fixes * fix compilation --------- Co-authored-by:
Marcin S. <[email protected]> * Correcting git mishap * Document usage of `gum` crate (#7294) * Document usage of gum crate * Small fix * Add some more basic info * Update node/gum/src/lib.rs Co-authored-by:
Andrei Sandu <[email protected]> * Update target docs --------- Co-authored-by:
Andrei Sandu <[email protected]> * XCM: Fix issue with RequestUnlock (#7278) * XCM: Fix issue with RequestUnlock * Leave API changes for v4 * Fix clippy errors * Fix tests --------- Co-authored-by: parity-processbot <> * Companion for Substrate#14228 (#7295) * Companion for Substrate#14228 https://github.com/paritytech/substrate/pull/14228 * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * Companion for #14237: Use latest sp-crates (#7300) * To revert: Update substrate branch to "lexnv/bump_sp_crates" Signed-off-by:
Alexandru Vasile <[email protected]> * Revert "To revert: Update substrate branch to "lexnv/bump_sp_crates"" This reverts commit 5f1db84eac4a226c37b7f6ce6ee19b49dc7e2008. * Update cargo lock Signed-off-by:
Alexandru Vasile <[email protected]> * Update cargo.lock Signed-off-by:
Alexandru Vasile <[email protected]> * Update cargo.lock Signed-off-by:
Alexandru Vasile <[email protected]> --------- Signed-off-by:
Alexandru Vasile <[email protected]> * bounded-collections bump to 0.1.7 (#7305) * bounded-collections bump to 0.1.7 Companion for: paritytech/substrate#14225 * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * bump to quote 1.0.28 (#7306) * `RollingSessionWindow` cleanup (#7204) * Replace `RollingSessionWindow` with `RuntimeInfo` - initial commit * Fix tests in import * Fix the rest of the tests * Remove dead code * Fix todos * Simplify session caching * Comments for `SessionInfoProvider` * Separate `SessionInfoProvider` from `State` * `cache_session_info_for_head` becomes freestanding function * Remove unneeded `mut` usage * fn session_info -> fn get_session_info() to avoid name clashes. The function also tries to initialize `SessionInfoProvider` * Fix SessionInfo retrieval * Code cleanup * Don't wrap `SessionInfoProvider` in an `Option` * Remove `earliest_session()` * Remove pre-caching -> wip * Fix some tests and code cleanup * Fix all tests * Fixes in tests * Fix comments, variable names and small style changes * Fix a warning * impl From<SessionWindowSize> for NonZeroUsize * Fix logging for `get_session_info` - remove redundant logs and decrease log level to DEBUG * Code review feedback * Storage migration removing `COL_SESSION_WINDOW_DATA` from parachains db * Remove `col_session_data` usages * Storage migration clearing columns w/o removing them * Remove session data column usages from `approval-voting` and `dispute-coordinator` tests * Add some test cases from `RollingSessionWindow` to `dispute-coordinator` tests * Fix formatting in initialized.rs * Fix a corner case in `SessionInfo` caching for `dispute-coordinator` * Remove `RollingSessionWindow` ;( * Revert "Fix formatting in initialized.rs" This reverts commit 0f94664ec9f3a7e3737a30291195990e1e7065fc. * v2 to v3 migration drops `COL_DISPUTE_COORDINATOR_DATA` instead of clearing it * Fix `NUM_COLUMNS` in `approval-voting` * Use `columns::v3::NUM_COLUMNS` when opening db * Update node/service/src/parachains_db/upgrade.rs Co-authored-by:
Andrei Sandu <[email protected]> * Don't write in `COL_DISPUTE_COORDINATOR_DATA` for `test_rocksdb_migrate_2_to_3` * Fix `NUM+COLUMNS` in approval_voting * Fix formatting * Fix columns usage * Clarification comments about the different db versions --------- Co-authored-by:
Andrei Sandu <[email protected]> * pallet-para-config: Remove remnant WeightInfo functions (#7308) * pallet-para-config: Remove remnant WeightInfo functions Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * set_config_with_weight begone Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::configuration --------- Signed-off-by:
Oliver Tale-Yazdi <[email protected]> Co-authored-by: command-bot <> * XCM: PayOverXcm config (#6900) * Move XCM query functionality to trait * Fix tests * Add PayOverXcm implementation * fix the PayOverXcm trait to compile * moved doc comment out of trait implmeentation and to the trait * PayOverXCM documentation * Change documentation a bit * Added empty benchmark methods implementation and changed docs * update PayOverXCM to convert AccountIds to MultiLocations * Implement benchmarking method * Change v3 to latest * Descend origin to an asset sender (#6970) * descend origin to an asset sender * sender as tuple of dest and sender * Add more variants to the QueryResponseStatus enum * Change Beneficiary to Into<[u8; 32]> * update PayOverXcm to return concrete errors and use AccountId as sender * use polkadot-primitives for AccountId * fix dependency to use polkadot-core-primitives * force Unpaid instruction to the top of the instructions list * modify report_outcome to accept interior argument * use new_query directly for building final xcm query, instead of report_outcome * fix usage of new_query to use the XcmQueryHandler * fix usage of new_query to use the XcmQueryHandler * tiny method calling fix * xcm query handler (#7198) * drop redundant query status * rename ReportQueryStatus to OuterQueryStatus * revert rename of QueryResponseStatus * update mapping * Update xcm/xcm-builder/src/pay.rs Co-authored-by:
Gavin Wood <[email protected]> * Updates * Docs * Fix benchmarking stuff * Destination can be determined based on asset_kind * Tweaking API to minimise clones * Some repotting and docs --------- Co-authored-by:
Anthony Alaribe <[email protected]> Co-authored-by:
Muharem Ismailov <[email protected]> Co-authored-by:
Anthony Alaribe <[email protected]> Co-authored-by:
Gavin Wood <[email protected]> * Companion for #14265 (#7307) * Update Cargo.lock Signed-off-by:
Alexandru Vasile <[email protected]> * Update Cargo.lock Signed-off-by:
Alexandru Vasile <[email protected]> --------- Signed-off-by:
Alexandru Vasile <[email protected]> Co-authored-by: parity-processbot <> * bump serde to 1.0.163 (#7315) * bump serde to 1.0.163 * bump ci * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <> * fmt * Updated fmt * Removing changes accidentally pulled from master * fix another master pull issue * Another master pull fix * fmt * Fixing implementers guide build * Revert "Merge branch 'rh-async-backing-feature-while-frozen' of https://github.com/paritytech/polkadot into brad-rename-parathread" This reverts commit bebc24af52ab61155e3fe02cb3ce66a592bce49c, reversing changes made to 1b2de662dfb11173679d6da5bd0da9d149c85547. --------- Signed-off-by:
Oliver Tale-Yazdi <[email protected]> Signed-off-by:
acatangiu <[email protected]> Signed-off-by:
Alexandru Vasile <[email protected]> Co-authored-by:
Marcin S <[email protected]> Co-authored-by:
Oliver Tale-Yazdi <[email protected]> Co-authored-by:
Michal Kucharczyk <[email protected]> Co-authored-by:
Adrian Catangiu <[email protected]> Co-authored-by:
ordian <[email protected]> Co-authored-by:
Marcin S. <[email protected]> Co-authored-by:
Andrei Sandu <[email protected]> Co-authored-by:
Francisco Aguirre <[email protected]> Co-authored-by:
Bastian Köcher <[email protected]> Co-authored-by:
Alexandru Vasile <[email protected]> Co-authored-by:
Sam Johnson <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]> Co-authored-by:
Anthony Alaribe <[email protected]> Co-authored-by:
Muharem Ismailov <[email protected]> Co-authored-by:
Anthony Alaribe <[email protected]> Co-authored-by:
Gavin Wood <[email protected]> * fix bitfield distribution test * approval distribution tests * fix bridge tests * update Cargo.lock * [async-backing-branch] Optimize collator-protocol validator-side request fetching (#7457) * Optimize collator-protocol validator-side request fetching * address feedback: replace tuples with structs * feedback: add doc comments * move collation types to subfolder --------- Signed-off-by:
alindima <[email protected]> * Update collation generation for asynchronous backing (#7405) * break candidate receipt construction and distribution into own function * update implementers' guide to include SubmitCollation * implement SubmitCollation for collation-generation * fmt * fix test compilation & remove unnecessary submodule * add some TODOs for a test suite. * Update roadmap/implementers-guide/src/types/overseer-protocol.md Co-authored-by:
Andrei Sandu <[email protected]> * add new test harness and first test * refactor to avoid requiring background sender * ensure collation gets packaged and distributed * tests for the fallback case with no hint * add parent rp-number hint tests * fmt * update uses of CollationGenerationConfig * fix remaining test * address review comments * use subsystemsender for background tasks * fmt * remove ValidationCodeHashHint and related tests --------- Co-authored-by:
Andrei Sandu <[email protected]> * fix some more fallout from merge * fmt * remove staging APIs from Rococo & Westend (#7513) * send network messages on main protocol name (#7515) * misc async backing improvements for allowed ancestry blocks (#7532) * shared: fix acquire_info * backwards-compat test for prospective parachains * same relay parent is allowed * provisioner: request candidate receipt by relay parent (#7527) * return candidates hash from prospective parachains * update provisioner * update tests * guide changes * send a single message to backing * fix test * revert to old `handle_new_activations` logic in some cases (#7514) * revert to old `handle_new_activations` logic * gate sending messages on scheduled cores to max_depth >= 2 * fmt * 2->1 * Omnibus asynchronous backing bugfix PR (#7529) * fix a bug in backing * add some more logs * prospective parachains: take ancestry only up to session bounds * add test * fix zombienet tests (#7614) Signed-off-by:
Andrei Sandu <[email protected]> * fix runtime compilation * make bitfield distribution tests compile * attempt to fix zombienet disputes (#7618) * update metric name * update some metric names * avoid cycles when creating fake candidates * make undying collator more friendly to malformed parents * fix a bug in malus * fmt * clippy * add RUN_IN_CONTAINER to new ZombieNet tests (#7631) * remove duplicated migration happened because of master-merge --------- Signed-off-by:
Oliver Tale-Yazdi <[email protected]> Signed-off-by:
acatangiu <[email protected]> Signed-off-by:
Alexandru Vasile <[email protected]> Signed-off-by:
alindima <[email protected]> Signed-off-by:
Andrei Sandu <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]> Co-authored-by:
Parity Bot <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]> Co-authored-by:
Robert Klotzner <[email protected]> Co-authored-by:
Robert Klotzner <[email protected]> Co-authored-by:
Marcin S <[email protected]> Co-authored-by:
Marcin S <[email protected]> Co-authored-by:
Mattia L.V. Bradascio <[email protected]> Co-authored-by:
Bradley Olson <[email protected]> Co-authored-by:
alexgparity <[email protected]> Co-authored-by:
BradleyOlson64 <[email protected]> Co-authored-by:
Oliver Tale-Yazdi <[email protected]> Co-authored-by:
Michal Kucharczyk <[email protected]> Co-authored-by:
Adrian Catangiu <[email protected]> Co-authored-by:
ordian <[email protected]> Co-authored-by:
Andrei Sandu <[email protected]> Co-authored-by:
Francisco Aguirre <[email protected]> Co-authored-by:
Bastian Köcher <[email protected]> Co-authored-by:
Alexandru Vasile <[email protected]> Co-authored-by:
Sam Johnson <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]> Co-authored-by:
Anthony Alaribe <[email protected]> Co-authored-by:
Muharem Ismailov <[email protected]> Co-authored-by:
Anthony Alaribe <[email protected]> Co-authored-by:
Gavin Wood <[email protected]> Co-authored-by:
Alin Dima <[email protected]>
-
- Aug 17, 2023
-
-
eskimor authored
* First baby steps * Split scheduler into several modules * Towards a more modular approach for scheduling * move free_cores; IntoInterator -> BTreeMap * Move clear() * Move more functions out of scheduler * Change weight composition * More abstraction * Further refactor * clippy * fmt * fix test-runtime * Add parathreads pallet to construct_runtime! * Make all runtimes use (Parachains, Parathreads) scheduling * Delete commented out code * Remove parathreads scheduler from westend, rococo, and kusama * fix rococo, westend, and kusama config * Revert "fix rococo, westend, and kusama config" This reverts commit 59e4de380d5c7d17eaaba5e2c2b81405de3465e3. * Revert "Remove parathreads scheduler from westend, rococo, and kusama" This reverts commit 4c44255296083ac5670560790ed77104917890a4. * Remove CoreIndex from free_cores * Remove unnecessary struct for parathreads * parathreads provider take 1 * Comment out parathread tests * Pop into lookahead * fmt * Fill lookahead with two entries for parachains * fmt * Current stage * Towards ab parathreads * no AB use * Make tests typecheck * quick hack to set scheduling lookahead to 1 * Fix scheduler tests * fix paras_inherent tests * misc * Update more of a test * cfg(test) * some cleanup * Undo paras_inherent changes * Adjust paras inherent tests * Undo changes to v2 primitives * Undo v2 mod changes to tests * minor * Remove parathreads assigner and pallet * minor * minor * more cleanup * fmt * minor * minor * minor * Remove on_new_session from assignment provider * Make adder collator integration test pass * disable failing unit tests * minor * minor * re-enable one unit test * minor * handle retries, add concluded para to pop interface * comment out unused code * Remove core_para from interface * Remove first claimqueue element on clear if None instead removing all Nones * Move claimqueue get out of loop * Use VecDeque instead of Ved in ClaimQueue * Make occupied() AB ready(?) * handle freed disputed in clear_and_fill_claimqueue * clear_and_fill_claimqueue returns scheduled Vec * Rename and minor refactor * return position of assignment taken from claimqueue * minor * Fix session boundary parachains number change + extended test * Fix runtimes * Fix polkadot runtime * Remove polkadot pallet from benchmarks * fix test runtime * Add storage migration * Minor refactor * Minor * migratin typechecks * Add migration to runtimes * Towards modular scheduling II (#6568) * Add post migration check * pebkac * Disable migrations but mine * Revert "Disable migrations but mine" This reverts commit 4fa5c5a370c199944a7e0926f50b08626bfbad4c. * Move scheduler migration * Revert "Move scheduler migration" This reverts commit a16b1659a907950bae048a9f7010f2aa76e02b6d. * Fix migration * cleanup * Don't lose retries value anymore * comment out test function * Remove retries value from Assignment again * minor * Make collator for parathreads optional * data type refactor * update scheduler tests * Change test function cfg * comment out test function * Try cfg(test) only * fix cfg flags * Add get_max_retries function to provider interface (#7047) * Fix merge commit * pebkac * fix merge * update cargo.lock * fix merge * fix merge * Use btreemap instead of vec, fix scheduler calls. * Use imported `ScheduledCore` * Remove unused import in inclusion tests * Use keys() instead of mapping over a BTreeMap * Fix migrations for parachains scheduler * Use BlockNumberFor<T> everywhere in scheduler * Add on demand assignment provider pallet (#7110) * Address some PR comments * minor * more cleanup * find_map and timeout availability fixes * Change default scheduling_lookahead to 1 * Add on demand assignment provider pallet * Move test-runtime to new assignment provider * Run cargo format on scheduler tests * minor * Mutate cores in single loop * timeout predicate simplification * claimqueue desired size fix * Replace expect by ok_or * More improvements * Fix push back order and next_up_on_timeout * minor * session change docs * Add pre_new_session call to hand pre session updates * Remove sc_network dependency and PeerId from unnecessary data structures * Remove unnecessary peer_ids * Add OnDemandOrdering proxy (#7156) * Add OnDemandBidding proxy * Fix names * OnDemandAssigner for rococo only * Check PeerId in collator protocol before fetching collation * On occupied, remove non occupied cores from the claimqueue front and refill * Add missing docs * Comment out unused field * fix ScheduledCore in tests * Fix the fix * pebkac * fmt * Fix occupied dropping * Remove double import * ScheduledCore fixes * Readd sc-network dep * pebkac * OpaquePeerId -> PeerId in can_collate interface * Cargo.lock update for interface change * Remove checks not needed anymore? * Drop occupied core on session change if it would time out after the new session * Add on demand assignment provider pallet * Move test-runtime to new assignment provider * Run cargo format on scheduler tests * Add OnDemandOrdering proxy (#7156) * Add OnDemandBidding proxy * Fix names * OnDemandAssigner for rococo only * Remove unneeded config values * Update comments * Use and_then for queue position * Return the max size of the spot queue on error * Add comments to add_parathread_entry * Add module comments * Add log for when can_collate fails * Change assigner queue type to `Assignment` * Update assignment provider tests * More logs * Remove unused keyring import * disable can_collate * comment out can_collate * Can collate first checks set if empty * Move can_collate call to collation advertisement * Fix backing test * map to loop * Remove obsolete check * Move invalid collation test from backing to collator-protocol * fix unused imports * fix test * fix Debug derivation * Increase time limit on zombienet predicates * Increase zombienet timeout * Minor * Address some PR comments * Address PR comments * Comment out failing assert due to on-demand assigner missing * remove collator_restrictions info from backing * Move can_collate to ActiveParas * minor * minor * Update weight information for on demand config * Add ttl to parasentry * Fix tests missing parasentry ttl * Adjust scheduler tests to use ttl default values * Use match instead of if let for ttl drop * Use RuntimeDebug trait for `ParasEntry` fields * Add comments to on demand assignment pallet * Fix spot traffic calculation * Revert runtimedebug changes to primitives * Remove runtimedebug derivation from `ParasEntry` * Mention affinity in pallet level docs * Use RuntimeDebug trait for ParasEntry child types * Remove collator restrictions * Fix primitive versioning and other merge issues * Fix tests post merge * Fix node side tests * Edit parascheduler migration for clarity * Move parascheduler migration up to next release * Remove vestiges from merge * Fix tests * Refactor ttl handling * Remove unused things from scheduler tests * Move on demand assigner to own directory * Update documentation * Remove unused sc-network dependency in primitives Was used for collator restrictions * Remove unused import * Reenable scheduler test * Remove unused storage value * Enable timeout predicate test and fix fn Turns out that the issue with the compiler is fixed and we can now use impl Trait in the manner used here. * Remove unused imports * Add benchmarking entry for perbill in config * Correct typo * Address review comments * Log out errors when calculating spot traffic. * Change parascheduler's log target name * Update scheduler_common documentation * Use mutate for affinity fns, add tests * Add another on demand affinity test * Unify parathreads and parachains in HostConfig (take 2) (#7452) * Unify parathreads and parachains in HostConfig * Fixed missed occurences * Remove commented out lines * `HostConfiguration v7` * Fix version check * Add `MigrateToV7` to `Unreleased` * fmt * fmt * Fix compilation errors after the rebase * Update runtime/parachains/src/scheduler/tests.rs Co-authored-by:
Anton Vilhelm Ásgeirsson <[email protected]> * Update runtime/parachains/src/scheduler/tests.rs Co-authored-by:
Anton Vilhelm Ásgeirsson <[email protected]> * fmt * Fix migration test * Fix tests * Remove unneeded assert from tests * parathread_cores -> on_demand_cores; parathread_retries -> on_demand_retries * Fix a compilation error in tests * Remove unused `use` * update colander image version --------- Co-authored-by:
alexgparity <[email protected]> Co-authored-by:
Anton Vilhelm Ásgeirsson <[email protected]> Co-authored-by:
Javier Viola <[email protected]> * Fix branch after merge with master * Refactor out duplicate checks into a helper fn * Fix tests post merge * Rename add_parathread_assignment, add test * Update docs * Remove unused on_finalize function * Add weight info to on demand pallet * Update runtime/parachains/src/configuration.rs Co-authored-by:
Tsvetomir Dimitrov <[email protected]> * Update runtime/parachains/src/scheduler_common/mod.rs Co-authored-by:
Tsvetomir Dimitrov <[email protected]> * Update runtime/parachains/src/assigner_on_demand/mod.rs Co-authored-by:
Tsvetomir Dimitrov <[email protected]> * Add benchmarking to on demand pallet * Make place_order test check for success * Add on demand benchmarks * Add local test weights to rococo runtime * Modify TTL drop behaviour to not skip claims Previous behaviour would jump a new claim from the assignment provider ahead in the claimqueue, assuming lookahead is larger than 1. * Refactor ttl test to test claimqueue order * Disable place_order ext. when no on_demand cores * Use default genesis config for benchmark tests * Refactor config builder param * Move lifecycle test from scheduler to on demand * Remove unneeded lifecycle test Paras module via the parachain assignment provider doesn't provide new assignments if a parachain loses it's lease. The on demand assignment provider doesn't provide an assignment that is not a parathread. * Re enable validator shuffle test * More realistic weights for place_order * Remove redundant import * Fix backwards compatibility (hopefully) * ".git/.scripts/commands/bench/bench.sh" --subcommand=runtime --runtime=rococo --target_dir=polkadot --pallet=runtime_parachains::assigner_on_demand * Fix tests. * Fix off-by-one. * Re enable claimqueue fills test * Re enable schedule_rotates_groups test * Fix fill_claimqueue_fills test * Re enable next_up_on_timeout test, move fn * Do not pop from assignment provider when retrying * Fix tests missing collator in scheduledcore * Add comment about timeout predicate. * Rename parasentry retries to availability timeouts * Re enable schedule_schedules... test * Refactor prune retried test to new scheduler * Have all scheduler tests use genesis_cfg fn * Update docs * Update copyright notices on new files * Rename is_parachain_core to is_bulk_core * Remove erroneous TODO * Simplify import * ".git/.scripts/commands/bench/bench.sh" --subcommand=runtime --runtime=rococo --target_dir=polkadot --pallet=runtime_parachains::configuration * Revert AdvertiseCollation order shuffle * Refactor place_order into keepalive and allowdeath * Revert rename of hrmp max inbound channels parachain encompasses both on demand and slot auction / bulk. * Restore availability_timeout_predicate function * Clean up leftover comments * Update runtime/parachains/src/scheduler/tests.rs Co-authored-by:
Tsvetomir Dimitrov <[email protected]> * ".git/.scripts/commands/bench/bench.sh" --subcommand=runtime --runtime=westend --target_dir=polkadot --pallet=runtime_parachains::configuration --------- Co-authored-by:
alexgparity <[email protected]> Co-authored-by:
alexgparity <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]> Co-authored-by:
Javier Viola <[email protected]> Co-authored-by:
eskimor <[email protected]> Co-authored-by: command-bot <> * On Demand - update weights and small nits (#7605) * Remove collator restriction test in inclusion On demand parachains won't have collator restrictions implemented in this way but will instead use a preferred collator registered to a `ParaId` in `paras_registrar`. * Remove redundant config guard for test fns * Update weights * Update WeightInfo for on_demand assigner * Unify assignment provider parameters into one call (#7606) * Combine assignmentprovider params into one fn call * Move scheduler_common to a module under scheduler * Fix ttl handling in benchmark builder * Run cargo format * Remove obsolete test. * Small improvement. * Use same migration pattern as config module * Remove old TODO * Change log target name for assigner on demand * Fix migration * Fix clippy warnings * Add HostConfiguration storage migration to V8 * Add `MigrateToV8` to unreleased migrations for all runtimes * Fix storage version check for config v8 * Set `StorageVersion` to 8 in `MigrateToV8` * Remove dups. * Update primitives/src/v5/mod.rs Co-authored-by:
Bastian Köcher <[email protected]> --------- Co-authored-by:
alexgparity <[email protected]> Co-authored-by:
alexgparity <[email protected]> Co-authored-by:
antonva <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]> Co-authored-by:
Anton Vilhelm Ásgeirsson <[email protected]> Co-authored-by:
Javier Viola <[email protected]> Co-authored-by:
eskimor <[email protected]> Co-authored-by:
Bastian Köcher <[email protected]>
-
- Aug 14, 2023
-
-
Lulu authored
-
Oliver Tale-Yazdi authored
* Use same rustfmt.toml as Substrate Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * format format file Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Format with new config Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add Substrate Clippy config Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Print Clippy version in CI Otherwise its difficult to reproduce locally. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Make fmt happy Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update node/core/pvf/src/error.rs Co-authored-by:
Tsvetomir Dimitrov <[email protected]> * Update node/core/pvf/src/error.rs Co-authored-by:
Tsvetomir Dimitrov <[email protected]> --------- Signed-off-by:
Oliver Tale-Yazdi <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]>
-
- Aug 04, 2023
-
-
Lulu authored
* Add license to crates This is required to publish to crates.io * Add more licenses
-
- Aug 01, 2023
-
-
eskimor authored
* Take into account size as well in weight limiting. * Fix logging. * More logs. * Remove randomized selection in provisioner No longer supported by runtime. * Fix and simplify weight calculation. Random filtering of remote disputes got dropped. * Make existing tests pass. * Tests for size limiting. * Fix provisioner. * Remove rand dependency. * Better default block length for tests. * ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent * Update runtime/parachains/src/paras_inherent/mod.rs Co-authored-by:
Tsvetomir Dimitrov <[email protected]> * Update runtime/parachains/src/paras_inherent/mod.rs Co-authored-by:
Chris Sosnin <[email protected]> * Add back missing line. * Fix test. * fmt fix. * Add missing test annotation --------- Co-authored-by:
eskimor <[email protected]> Co-authored-by: command-bot <> Co-authored-by:
Tsvetomir Dimitrov <[email protected]> Co-authored-by:
Chris Sosnin <[email protected]>
-
- Jun 27, 2023
-
-
Sebastian Kunert authored
* Remove unneeded feature flag * More serde flags * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <>
-
- Jun 21, 2023
-
-
Koute authored
-
eskimor authored
* Clarifications on vstaging. * Typos. --------- Co-authored-by:
eskimor <[email protected]>
-
- Jun 14, 2023
-
-
eskimor authored
* Remove redundant enter call. * Remove optionality in dispute signature checking * Make enter_inner and create_inherent the same. * Remove redundant metric. * Unification: enter and create_inherent. * Remove `enter_inner` function. * Remove dead code. * Remove redundant import. * Remove dead code in disputes. * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent * Merge fix. * Fix tests. * Remove obsolete comment. * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent * ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent * Review remarks, fixes. * Guide updates. * Fmt. * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent --------- Co-authored-by:
eskimor <[email protected]> Co-authored-by: command-bot <>
-
- Jun 12, 2023
-
-
eskimor authored
* Move vstaging to production (and thus past session slashing). WIP: test-runtime still needs to be fixed. * Fix test-runtime. --------- Co-authored-by:
eskimor <[email protected]>
-
- Jun 05, 2023
-
-
ordian authored
* runtime/vstaging: unapplied_slashes runtime API * runtime/vstaging: key_ownership_proof runtime API * runtime/ParachainHost: submit_report_dispute_lost * fix key_ownership_proof API * runtime: submit_report_dispute_lost runtime API * nits * Update node/subsystem-types/src/messages.rs Co-authored-by:
Marcin S. <[email protected]> * revert unrelated fmt changes * dispute-coordinator: past session dispute slashing * encapsule runtime api call for submitting report * prettify: extract into a function * do not exit on runtime api error * fix tests * try initial zombienet test * try something * fix a typo * try cumulus-based collator * fix clippy * build polkadot-debug images with fast-runtime enabled * wip * runtime/inclusion: fix availability_threshold * fix wip * fix wip II * revert native provider * propagate tx submission error * DEBUG: sync=trace * print key ownership proof len * panic repro * log validator index in panic message * post merge fixes * replace debug assertion with a log * fix compilation * Let's log the dispatch info in validate block. * fix double encoding * Revert "Let's log the dispatch info in validate block." This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4. * Revert "Let's log the dispatch info in validate block." This reverts commit a70fbc51b464d7f4355dbada5e16cd83cf71eab4. * fix compilation * update to latest zombienet and fix test * lower finality lag to 11 * bump zombienet again * add a workaround, but still does not work * Update .gitlab-ci.yml bump zombienet. * add a comment and search logs on all nodes --------- Co-authored-by:
Marcin S. <[email protected]> Co-authored-by:
Bastian Köcher <[email protected]> Co-authored-by:
Javier Viola <[email protected]>
-
- Jun 01, 2023
-
-
Michal Kucharczyk authored
* ParaId: Serialize/Deserialize trait implemented in no-std * Serialize/Deserialize trait implemented in no-std for more types * serde in deps enabled * fix * missing types added * update lockfile for {"substrate"} * ".git/.scripts/commands/fmt/fmt.sh" --------- Co-authored-by: parity-processbot <>
-
- May 31, 2023
-
-
Sam Johnson authored
* bump serde to 1.0.163 * bump ci * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <>
-
- May 26, 2023
-
-
ordian authored
* runtime/vstaging: unapplied_slashes runtime API * runtime/vstaging: key_ownership_proof runtime API * runtime/ParachainHost: submit_report_dispute_lost * fix key_ownership_proof API * runtime: submit_report_dispute_lost runtime API * nits * Update node/subsystem-types/src/messages.rs Co-authored-by:
Marcin S. <[email protected]> * revert unrelated fmt changes * post merge fixes * fix compilation --------- Co-authored-by:
Marcin S. <[email protected]>
-
- May 19, 2023
-
-
Gavin Wood authored
* Add ProcessXcmMessage struct * Migrate away from weights in host config * New well-known key to report UMPQ capacity * Add missing file * Fixes * Remove original UMP files * Docs * Update runtime/parachains/src/inclusion/mod.rs Co-authored-by:
asynchronous rob <[email protected]> * Add benchmarking * Benchmarks * Mock example of using the QueueChangeHandler to update the WKK * Use master Cargo.lock Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Merge remote-tracking branch 'origin/master' into gav-message-queue * Update Cargo.lock Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update remove-weight migration The migration got touched on master; just resolving conflicts here. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add message- to dispatch-origin conversion for XCM processing Just using the `impl Into<MultiLocation>` was a bit inflexible. Like this, the Relaychain can convert `UMP(para)` to a MultiLocation `para`. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * DNM: Temporarily comment code since XCMv3 is not merged yet Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use u64 for queue-wide limits on UmpAcceptanceCheckErr Using u32 here was one audit finding for the queue pallet. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Define one sub-queue per *MP queue Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Harden check_upward_messages Using safe math and casts. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add type-safe well_known_keys Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add message-queue weights Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Deploy MessageQueue to Polkadot Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update Cargo.toml Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Migrate to parachain config V5 Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update UMP tests Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Cleanup Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Revert messed up merge
🤦 Signed-off-by:Oliver Tale-Yazdi <[email protected]> * Update remove-weight migration The migration got touched on master; just resolving conflicts here. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add message- to dispatch-origin conversion for XCM processing Just using the `impl Into<MultiLocation>` was a bit inflexible. Like this, the Relaychain can convert `UMP(para)` to a MultiLocation `para`. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * DNM: Temporarily comment code since XCMv3 is not merged yet Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use u64 for queue-wide limits on UmpAcceptanceCheckErr Using u32 here was one audit finding for the queue pallet. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Define one sub-queue per *MP queue Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Harden check_upward_messages Using safe math and casts. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add type-safe well_known_keys Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add message-queue weights Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Deploy MessageQueue to Polkadot Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update Cargo.toml Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Migrate to parachain config V5 Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update UMP tests Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Cleanup Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Make stuff compile Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Move DMP and HRMP messages to the MessageQueue It currently does not compile in the CIbecause of some local tweaks to Substrate. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Diener for CI Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * diener update cargo.lock Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Cleanup Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Revert wrong changes Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Revert "DNM: Temporarily comment code since XCMv3 is not merged yet" This reverts commit 820aa235cb21dd1d2621843607f7682bf035434e. * Make compile Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fixup runtimes Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Define benchmarks Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Cleanup Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix migration Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use master Cargo.lock Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Lockfile Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix test Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add AggregateMessageOrigin This enum currently only holds one value, but having it will make it easier in the future to extend. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Forbid UMP for off-boarding paras - Reject candidates with UMP messages for off-boarding paras - Forbid scheduling off-boarding when a para has unprocess UMPs Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Delete stupid test Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use BoundedVec for upward messages Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add weights and fix MessageProcessor Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Bound receive_upward_messages and check bound in configuration pallet Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Bound Debug impl Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * clippy Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix test runtime Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix xcm-simulator Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Properly fix xcm-simulator and fuzzer Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix tests Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * cargo update -p sp-io Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Adapt to upstream Substrate changes Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix ProcesseMessage impls Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Some tests Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use master Cargo.lock Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * cargo update -p sp-io Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use new MQ API Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix test Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix migration Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update Cargo.lock Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add UMP while Para offboarding tests Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use Mocked message processor for benchmarking Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use variables for constants Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add MQ pallet weights Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use MQ pallet weights Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Configure QueueChangeHandler Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add config test Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix MQ serive weight Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Cleanup Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Cleanup outgoing UMP dispatch queues Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use Master Cargo.lock Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update Cargo.lock Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Weight mul is not const Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Clippy Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Remove merge marker Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update runtime/parachains/src/inclusion/mod.rs Co-authored-by:
Gavin Wood <[email protected]> * Update runtime/kusama/src/lib.rs Co-authored-by:
Gavin Wood <[email protected]> * Use lowercase UMP Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Clarify comment Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use Weight::from_parts Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix test Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix doc Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Emit event after the fact Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add defensive_proof to receive_upward_messages Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Reapply "Remove original UMP files" Looks like they came back from the dead. Re-apply commit cf6d316f0 Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Remove old files Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Rename MaxUmpMessageLen -> MaxUmpMessageLenOf Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Test defensive message dropping of receive_upward_messages Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fixup imports Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Update implementors guide Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix tests Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Remove FAIL-CI mark Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Delete unused code Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add another test for MQ change hook Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Imports Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Keep Kusama runtime formatting Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Revert "Delete unused code" This reverts commit dd76bca5025b7e1ef846a9539c3607eed185f16a. * Feature gate mock functions Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Review: Use saturating_add Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Test RelayDispatchQueueSize storage key Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Move migration to own file to avoid merge conflicts Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Migration in own file Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fixup migration Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Rococo: configure MQ pallet Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fixup tests Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Revert "Rococo: configure MQ pallet" Going to do this as follow up, since it needs Substrate changes and i dont want to stall this MR any longer. This reverts commit b9c15e8a8339c4e877d654ee3f09903af4210736. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Revert "Fixup tests" This reverts commit 88f1cbe20774d20e5e1e554e798960ae39437af1. * Fixup migration Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix migration Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix CI Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix migration Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix other migration Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Bump MAX_CODE_SIZE to 10MiB Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add ForceUpdateUmpLimits migration Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * clippy Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * clippy Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * imports Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use defensive instead of defensive_proof 'defensive_proof' also prints the 'self', which spams the console too much when running the tests. Just the length is enough. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Rename to ScheduleConfigUpdate Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fixup migration checks Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add MAX_CODE_SIZE to ScheduleConfigUpdate Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Set MAX_CODE_SIZE to 4MiB Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix benchmark Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix formatting Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Revert "Add MAX_CODE_SIZE to ScheduleConfigUpdate" This reverts commit 7caffb09e83083b57affd548215e45b25c3d64dc. * Revert "Set MAX_CODE_SIZE to 4MiB" This reverts commit 103ffbaf686487d2fbe0082a16826af17cacc1a1. * Revert "Bump MAX_CODE_SIZE to 10MiB" This reverts commit 530734b7b0da5b7680054e0242348fcc79a666fe. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Remove consistency check from migration Re-addig these checks is blocked on https://github.com/paritytech/polkadot/issues/7108 Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix constants Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Bump MAX_UPWARD_MESSAGE_SIZE_BOUND for Westend Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix migrations Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Use old nightly for fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fixes * cargo fmt * Fix tests Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix tests Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fixes * Add MQ pallet to fuzzer Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix XMC simulator example Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Remove runtime-benchmarks from fuzzers Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Revert "Remove runtime-benchmarks from fuzzers" This reverts commit e1f2bb01b6dea2dd465539d3658719895b58b557. * Fix example simulator Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Add V6 migration and remove old ones Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Actually make old migrations reusable Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Dont delete old migrations Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Future proof AggregateMessageOrigin and review fixes There are indications that Loopback and Bridged will be needed soon. Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * More cleanup Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * fix benchmarks Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix fuzzer build Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Review Co-authored-by:
muharem <[email protected]> Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Remove old migration Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Set MQ service weight to 20% Signed-off-by:
Oliver Tale-Yazdi <[email protected]> * Fix tabs in Markdown Signed-off-by:
Oliver Tale-Yazdi <[email protected]> --------- Signed-off-by:
Oliver Tale-Yazdi <[email protected]> Co-authored-by:
asynchronous rob <[email protected]> Co-authored-by:
Oliver Tale-Yazdi <[email protected]> Co-authored-by:
Keith Yeung <[email protected]>
-
- Apr 17, 2023
-
-
dependabot[bot] authored
Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.3.4 to 0.4.1. - [Release notes](https://github.com/RustCrypto/utils/releases) - [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.3.4...hex-literal-v0.4.1 ) --- updated-dependencies: - dependency-name: hex-literal dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by:
dependabot[bot] <[email protected]> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
- Apr 14, 2023
-
-
s0me0ne-unkn0wn authored
* Add a pallet call to change executor params * Use `OptionQuery`; Avoid runtime panic * Move pending executor params to `configuration` * Move `ExecutorParams` to `HostConfiguration` structure * Add executor params to the v5 migration * Add an `ExecutorParams` benchmark * ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::configuration * Add to `WeightInfo` * Add dummy weights to other networks * ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::configuration * ".git/.scripts/commands/bench/bench.sh" runtime rococo runtime_parachains::configuration * ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::configuration * Use real weight * Fix comment --------- Co-authored-by: command-bot <>
-
- Apr 12, 2023
-
-
Roman Useinov authored
* bump scale-info to be in line with cumulus * remove irrelevant lines * fix Cargo.lock
-
- Apr 08, 2023
-
-
s0me0ne-unkn0wn authored
* Happy New Year! * Remove year entierly Co-authored-by:
Oliver Tale-Yazdi <[email protected]> * Remove years from copyright notice in the entire repo --------- Co-authored-by:
Oliver Tale-Yazdi <[email protected]>
-
- Apr 05, 2023
-
-
s0me0ne-unkn0wn authored
-
- Mar 28, 2023
-
-
Chris Sosnin authored
* Backport async backing params primitive * migration follow-up * link pr * parameters -> params * rustfmt::skip block ident
-
Tsvetomir Dimitrov authored
* Histogram support in runtime metrics Add support for histograms to the runtime metrics. Additionally add `polkadot_parachain_verify_dispute_signature` histogram which tracks the time needed from the runtime to verify a single validator signature of a dispute statement. * Add noops * u64 instead of f64 * Update buckets * Wrap `get_current_time()` in runtime metrics * Change the dimension of the Histogram from usec to sec * Fix a compilation error * Update buckets * Fix `on_signature_check_complete` calculation * Update buckets * Update buckets * formatting * Another weights update * Adjust buckets again * Final buckets adjustment * Revert "Fix a compilation error" This reverts commit 06290b40a39eeb78de2602d8916a39edf7a8b714. * Update primitives/src/v4/metrics.rs Co-authored-by:
Andrei Sandu <[email protected]> * Use `saturating_sub` for time difference calculation * Pass nanoseconds to client instead of seconds (using f64 in runtime is dangerous) --------- Co-authored-by:
Andrei Sandu <[email protected]>
-
- Mar 24, 2023
-
-
Davide Galassi authored
* Companion for #13683 * Wraps trait is not required * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <>
-
- Mar 23, 2023
-
-
Tsvetomir Dimitrov authored
-
- Mar 21, 2023
-
-
ordian authored
-
Chris Sosnin authored
-
- Mar 17, 2023
-
-
Davide Galassi authored
* Remove not required async calls * Fixed missing renaming * make_keystore can be sync * More fixes * Trivial nitpicks * Cherry pick test fix from master * Fixes after master merge * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <>
-
- Mar 15, 2023
-
-
s0me0ne-unkn0wn authored
-
- Mar 13, 2023
-
-
Vivek Pandya authored
* Remove use of Store trait from runtime directory * Remove Store trait usage from xcm directory * Run cargo fmt * update lockfile for {"substrate"} --------- Co-authored-by: parity-processbot <>
-
- Mar 08, 2023
-
-
s0me0ne-unkn0wn authored
* Move PVF timeouts to executor environment parameters * Typo Co-authored-by:
Marcin S. <[email protected]> * Fix comments * Change handle_import_statements to FatalResult (#6820) * Changing dispute db errors to fatal * fmt * Change node-key for bootnodes (#6772) * Additional tracing in `provisioner`, `vote_selection` and `dispute-coordinator` (#6775) * Additional tracing in `provisioner`, `vote_selection` * Add `fetched_onchain_disputes` metric to provisioner * Some tracelines in dispute-coordinator TODO: cherry pick this in the initial branch!!! * Remove spammy logs * Remove some trace lines * Rename and fix things * Fix comments * Typo * Minor fixes * Add codec indexes; Remove macro --------- Co-authored-by:
Marcin S. <[email protected]> Co-authored-by:
Bradley Olson <[email protected]> Co-authored-by:
Petr Mensik <[email protected]> Co-authored-by:
Tsvetomir Dimitrov <[email protected]>
-
- Mar 03, 2023
-
-
eskimor authored
* Add note about frozen `SessionInfo`. * Better comment. --------- Co-authored-by:
eskimor <[email protected]>
-
- Feb 20, 2023
-
-
s0me0ne-unkn0wn authored
-
- Feb 16, 2023
-
-
Bastian Köcher authored
* Change `MaxMemorySize` to `MaxMemoryPages` We should set the max memory for the executor in pages (64KiB) and not in bytes. The wasm memory is always a multiple of a page and we should use the same terminology. * FMT * Fix warning
-
Marcin S. authored
* Use a `BoundedVec` in `ValidationResult` > Use a `BoundedVec` for `upward_messages` and `horizontal_messages` in order to > limit the number of individual messages/memory allocations right at decoding > time. The reason for this is that the `ValidationResult` may contain a code > upgrade (including a full PVF binary), so the total size limit can't be set > too low and this limit will still allow several millions of upward messages, > which will (due to the memory allocator overhead) already have a > non-negligible memory footprint in decoded form. * List all fields when hashing so we don't miss one * Define types for `BoundedVec`s of messages * Fix test compile errors * Depend on `bounded-collections` 0.1.4 (fixes allocation issue) * Fix compilation issue * Derive `Hash` instead of manual `impl` * Avoid use of unwrap
-
- Feb 15, 2023
-
-
s0me0ne-unkn0wn authored
* Re-apply changes without Diener, rebase to the lastest master * Cache pruning * Bit-pack InstantiationStrategy * Move ExecutorParams version inside the structure itself * Rework runtime API and executor parameters storage * Pass executor parameters through backing subsystem * Update Cargo.lock * Introduce `ExecutorParams` to approval voting subsys * Introduce `ExecutorParams` to dispute coordinator * `cargo fmt` * Simplify requests from backing subsys * Fix tests * Replace manual config cloning with `.clone()` * Move constants to module * Parametrize executor performing PVF pre-check * Fix Malus * Fix test runtime * Introduce session executor params as a constant defined by session info pallet * Use Parity SCALE codec instead of hand-crafted binary encoding * Get rid of constants; Add docs * Get rid of constants * Minor typo * Fix Malus after rebase * `cargo fmt` * Use transparent SCALE encoding instead of explicit * Clean up * Get rid of relay parent to session index mapping * Join environment type and version in a single enum element * Use default execution parameters if running an old runtime * `unwrap()` -> `expect()` * Correct API version * Constants are back in town * Use constants for execution environment types * Artifact separation, first try * Get rid of explicit version * PVF execution queue worker separation * Worker handshake * Global renaming * Minor fixes resolving discussions * Two-stage requesting of executor params to make use of runtime API cache * Proper error handling in pvf-checker * Executor params storage bootstrapping * Propagate migration to v3 network runtimes * Fix storage versioning * Ensure `ExecutorParams` serialization determinism; Add comments * Rename constants to make things a bit more deterministic Get rid of stale code * Tidy up a structure of active PVFs * Minor formatting * Fix comment * Add try-runtime hooks * Add storage version write on upgrade Co-authored-by:
Andronik <[email protected]> * Add pre- and post-upgrade assertions * Require to specify environment type; Remove redundant `impl`s * Add `ExecutorParamHash` creation from `H256` * Fix candidate validation subsys tests * Return splittable error from executor params request fn * Revert "Return splittable error from executor params request fn" This reverts commit a0b274177d8bb2f6e13c066741892ecd2e72a456. * Decompose approval voting metrics * Use more relevant errors * Minor formatting fix * Assert a valid environment type instead of checking * Fix `try-runtime` hooks * After-merge fixes * Add migration logs * Remove dead code * Fix tests * Fix tests * Back to the strongly typed implementation * Promote strong types to executor interface * Remove stale comment * Move executor params to `SessionInfo`: primitives and runtime * Move executor params to `SessionInfo`: node * Try to bump primitives and API version * Get rid of `MallocSizeOf` * Bump target API version to v4 * Make use of session index already in place * Back to v3 * Fix all the tests * Add migrations to all the runtimes * Make use of existing `SessionInfo` in approval voting subsys * Rename `TARGET` -> `LOG_TARGET` * Bump all the primitives to v3 * Fix Rococo ParachainHost API version * Use `RollingSessionWindow` to acquire `ExecutorParams` in disputes * Fix nits from discussions; add comments * Re-evaluate queue logic * Rework job assignment in execution queue * Add documentation * Use `RuntimeInfo` to obtain `SessionInfo` (with blackjack and caching) * Couple `Pvf` with `ExecutorParams` wherever possible * Put members of `PvfWithExecutorParams` under `Arc` for cheap cloning * Fix comment * Fix CI tests * Fix clippy warnings * Address nits from discussions * Add a placeholder for raw data * Fix non exhaustive match * Remove redundant reexports and fix imports * Keep only necessary semantic features, as discussed * Rework `RuntimeInfo` to support mock implementation for tests * Remove unneeded bound * `cargo fmt` * Revert "Remove unneeded bound" This reverts commit 932463f26b00ce290e1e61848eb9328632ef8a61. * Fix PVF host tests * Fix PVF checker tests * Fix overseer declarations * Simplify tests * `MAX_KEEP_WAITING` timeout based on `BACKGING_EXECUTION_TIMEOUT` * Add a unit test for varying executor parameters * Minor fixes from discussions * Add prechecking max. memory parameter (see paritytech/srlabs_findings#110) * Fix and improve a test * Remove `ExecutionEnvironment` and `RawData` * New primitives versioning in parachain host API * `disputes()` implementation for Kusama and Polkadot * Move `ExecutorParams` from `vstaging` to stable primitives * Move disputes from `vstaging` to stable implementation * Fix `try-runtime` * Fixes after merge * Move `ExecutorParams` to the bottom of `SessionInfo` * Revert "Move executor params to `SessionInfo`: primitives and runtime" This reverts commit dfcfb85fefd1c5be6c8a8f72dc09fd1809cfa9ce. * Always use fresh activated live hash in pvf precheck (re-apply 34b09a4c20de17e7926ed942cd0d657d18f743fa) * Fixing tests (broken commit) * Fix candidate validation tests * Fix PVF host test * Minor fixes * Address discussions * Restore migration * Fix `use` to only include what is needed instead of `*` * Add comment to never touch `DEFAULT_CONFIG` * Update migration to set default `ExecutorParams` for `dispute_period` sessions back * Use `earliest_stored_session` instead of calculations * Nit * Add logs * Treat any runtime error as `NotSupported` again * Always return default executor params if not available * Revert "Always return default executor params if not available" This reverts commit b58ac4482ef444c67a9852d5776550d08e312f30. * Add paritytech/substrate#9997 workaround * `cargo fmt` * Remove migration (again!) * Bump executor params to API v4 (backport from #6698) --------- Co-authored-by:
Andronik <[email protected]>
-
- Feb 03, 2023
-
-
Keith Yeung authored
-
- Jan 11, 2023
-
-
s0me0ne-unkn0wn authored
* Re-export current primitives in crate root * Add missing exports * restart CI
-
- Dec 06, 2022
-
-
Marcin S. authored
* Begin removing `parity-util-mem`; remove `collect_memory_stats` * Update some dependencies that were using `parity-util-mem` * Remove `trie-memory-tracker` feature * Update Cargo.lock * Update `kvdb-shared-tests` * Add back jemalloc * Add missing license header * update lockfile for {"substrate"} Co-authored-by: parity-processbot <> Co-authored-by:
Andronik <[email protected]>
-
- Dec 05, 2022
-
-
Squirrel authored
* rust 1.64 enables workspace properties * add edition, repository and authors. * of course, update the version in one place. Co-authored-by:
Andronik <[email protected]>
-