1. 27 Nov, 2021 1 commit
  2. 26 Nov, 2021 4 commits
    • Sergey Pepyakin's avatar
      Do not log PVF prunning every hour (#4372) · e83aebaf
      Sergey Pepyakin authored
      This lowers the level of the PVF pruning.
      
      Closes https://github.com/paritytech/polkadot/issues/4361
      e83aebaf
    • Sergey Pepyakin's avatar
      Impose new restrictions on paras init and cleanup (#4360) · 62915a68
      Sergey Pepyakin authored
      * Impose new restrictions on paras init and cleanup
      
      For upcoming PVF pre-checking feature we will need to impose a couple of
      new restrictions for:
      
      - `schedule_para_initialize`.
      - `schedule_para_cleanup`.
      
      Specifically, for the former we do not want to allow registration of
      wasm blob that is empty, i.e. 0 bytes. While that currently already
      does not make a lot of sense, it allows us to simplify the PVF
      pre-checking logic: if this PR is deployed before the following changes
      for PVF prechecking then we can be sure that no paras onboarding have to
      have to go through the PVF pre-checking. In case, we deploy it
      altogether this property will allow us to distingush paras that came in
      before PVF pre-checking.
      
      For `schedule_para_cleanup` we do not want to allow offboarding of paras
      that are undergoing the upgrade process. While this is not a harsh
      restriction this change allows us to avoid making the PVF prechecking
      more complicated than it has to be.
      
      * Add a test for schedule_para_initialize
      
      * Link to `ParaLifecycle::is_stable` in docs.
      
      * `schedule_para_{init,cleanup}` docs
      
      Now they link to their original declarations in the pallet for more
      details.
      62915a68
    • sandreim's avatar
      Fix Provisioner dispute metrics naming (#4374) · 8d6c8826
      sandreim authored
      
      
      * update dispute metric names
      Signed-off-by: default avatarAndrei Sandu <andrei-mihail@parity.io>
      
      * update
      Signed-off-by: default avatarAndrei Sandu <andrei-mihail@parity.io>
      8d6c8826
    • Keith Yeung's avatar
      Update W3F URL links (#4376) · e90ee611
      Keith Yeung authored
      e90ee611
  3. 25 Nov, 2021 2 commits
  4. 24 Nov, 2021 8 commits
  5. 23 Nov, 2021 1 commit
    • Zeke Mostov's avatar
      Inherent filtering follow up (#4305) · 89529740
      Zeke Mostov authored
      
      
      * Add feature more feature gating for benchmarking + tests
      
      * New line
      
      * cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
      
      * cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs
      
      * Do not assume we use max validators per core
      
      * Use kusama weights for rococo (hopefully temp)
      
      * cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras_inherent.rs
      
      * Add more validity votes when neccesary
      
      * Some fixes for the last commit
      
      * Restore westend weights
      
      * cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras_inherent.rs
      
      * Revert bad westend weights write
      
      * Make sure to update val idx before skipping
      
      * Fix validity vote range to max at group size'
      
      * Temp setup for rococo
      
      * cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_paras_inherent.rs --header=./file_header.txt
      
      * Augment generated Rococo weights
      
      * Make it compile
      
      * Revert range for enter_backed_candidates_variable
      
      * Delete runtime/kusama/src/weights/runtime_paras_paras_inherent.rs
      Co-authored-by: default avatarParity Bot <admin@parity.io>
      89529740
  6. 22 Nov, 2021 3 commits
  7. 20 Nov, 2021 2 commits
    • Javier Viola's avatar
      introduce malus + zombienet based integration tests (#4131) · 2ffb293e
      Javier Viola authored
      * test/malus: craft the first maliciously disputing actor
      
      * initial draft
      
      * Add Dockerfile and instructions how to use it to build malus image locally
      
      * Forgot one flag for the build cmd
      
      * we are not docker specific, we are happy to use any containerruntime
      
      * shuffle things around
      
      * add initial tera based integration test
      
      * chores
      
      * fixins
      
      * simple setup to start
      
      * other samples (WIP)
      
      * add Docker version with cargo-chef
      
      * update substarte and small change of orders in commands in the container file
      
      * metrics one
      
      * fmt
      
      * minor
      
      * fixin
      
      * fix metric names
      
      * -d
      
      * add open gauge
      
      * fmt
      
      * spellcheck
      
      * fix test
      
      * adjust to changed error messages
      
      * refactor, more malus impls
      
      * more malus changes
      
      * foo
      
      * minor cleanup
      
      * suggest garbage candidate
      
      * chore
      
      * fix suggest garabge malus
      
      * malus: back garbage candidate
      
      * cargo lock
      
      * re-introduce metrics
      
      * chore: cargo fmt
      
      * undoe 1.54.0 output, CI uses 1.53.0 rustc
      
      * update location of js types
      
      * Fix trybuild
      
      * add tag to image name also; this will be replaced in the prod version
      
      * Tests fixed
      
      * add some fix me
      
      * add dockerfile for ci
      
      * Add docker file for malus for ci
      
      * use variables in .toml file
      
      * add chnages for malus test
      
      * some fixes
      
      * some more fixes
      
      * Update .gitlab-ci.yml
      
      * add local build for polkadot and malus
      
      * some fixes
      
      * enable disputes feature in CI
      
      * ok, ok
      
      * rename: MsgFilter -> MessageInterceptor
      
      * remove TODO that would not have worked
      
      * intercept
      
      * refactor
      
      * fix README and containers
      
      * fix
      
      * chore: cargo fmt
      
      * avoid some more malus-$VARIANT references
      
      * fix argument order
      
      * chore: add about
      
      * Update sanity check in relay chain selection
      
      * fix order, add dispute-unavailable-block malus
      
      * fixup: avoid underflow issue
      
      * it's all u32
      
      * fix conditional use
      
      * Revert "it's all u32"
      
      This reverts commit 6b3ae25b.
      
      * Revert "fixup: avoid underflow issue"
      
      This reverts commit 336cbe29.
      
      * Revert "Update sanity check in relay chain selection"
      
      This reverts commit 970647f3
      
      .
      
      * update the malus bin
      
      * Update node/malus/integrationtests/0003-dispute-unavailable-block.feature
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      
      * add some FIXME reminders
      
      * update path to index.js
      
      * Update .gitlab-ci.yml
      
      * Update node/malus/integrationtests/0001-dispute-valid-block.toml
      
      * try 1: make malus test run
      
      * chore: cargo fmt
      
      * temporary fix
      
      * use subcommand syntax from latest gurke
      
      * cargo +nightly fmt
      
      * add collator to a a test
      
      * docs: add env vars to README
      
      * update ci to run dispute-valid-block test
      
      * needs the polkadot image
      
      * Fix path for nodejs container
      
      * post merge fix
      
      * download proper dir  containg configs for malus test
      
      * update the malus ci job
      
      * rm a whitespace
      
      * temp build for malus
      
      * use correct build command for temp malus
      
      * remove subcommands for now
      
      * set max validators per core in the default HostConfig
      
      * tabs
      
      * update beefy
      
      * fixup
      
      * fixup II
      
      * make one variant compile
      
      * make other variants compile
      
      * revert changes to chain_spec
      
      * fmt
      
      * build malus image from polkadot-test-malus again
      
      * revert unrelated changes
      
      * try fixing build-malus job
      
      * Revert "remove subcommands for now"
      
      This reverts commit 5d8292bc
      
      .
      
      * try fixing build-malus job II
      
      * MVP working dispute-ancestor
      
      * renames
      
      * fix PVF execution on malus
      
      * fix test
      
      * fix typo
      
      * fmt
      
      * checkmate
      
      * try something
      
      * make it actually work
      
      * some tweaks to 01 feature test
      
      * fmt
      
      * sleep a bit more
      
      * complete wococoization
      
      * some tweaks to 01 feature test
      
      * typo fix
      
      * use correct metric names
      
      * fix
      
      * ffs
      
      * .
      
      * try some rearrangement
      
      * Attempt to wait till initial node bootstrap in test
      
      * Fix test syntax
      
      * Run malus tests with v2 script
      
      * Proper symlink created
      
      * simnet v14
      
      * add zombienet tests
      
      * add zombie net test - draft
      
      * add more tests to dispute suite
      
      * add within to fix tests
      
      * replace test directory and start test migration
      
      * migrate all the tests
      
      * add timeout to tests
      
      * reduce debug
      
      * make easy to test in dev
      
      * set appropriated debug
      
      * use image from ci
      
      * fix config for test
      
      * set images from ci
      
      * fix config
      
      * add COLIMAGE env
      
      * tweek tests
      
      * reduce debug
      
      * typo
      
      * wip, migrate old test to zombie-net
      
      * adjunt test config for zombie-net
      
      * run mauls 0001 test only
      
      * clean and setup smoke-test in zombie-net
      
      * add extra time to assertinons
      
      * clean code to merge and improve README
      
      * add info to access logs
      
      * improved readme
      
      * merge master and resolve conflicts
      
      * Update zombienet_tests/README.md
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      
      * clean and consolidate zombienet name
      
      * change runner in gitlab
      
      * add comment explain why we use wococo
      
      * change tag for runner
      
      * remove unused tests
      
      * remove dup Dockerfile and update description
      
      * fmt
      
      * fix compilation post-merge
      
      * fmt
      
      * cut me Some slack
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      Co-authored-by: Radu Popa's avatarradupopa2010 <radupopa2010@yahoo.com>
      Co-authored-by: default avatarBastian Köcher <info@kchr.de>
      Co-authored-by: default avatarAnton Gavrilov <AntonE.Gavrilov@gmail.com>
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      Co-authored-by: Lldenaurois's avatarLldenaurois <Ljdenaurois@gmail.com>
      2ffb293e
    • Lldenaurois's avatar
      Replicate Rob's PR (#4337) · cd9929fb
      Lldenaurois authored
      cd9929fb
  8. 19 Nov, 2021 7 commits
    • Bastian Köcher's avatar
      Companion for Taskmanager: Remove `clean_shutdown` (#4336) · 58e4ef8e
      Bastian Köcher authored
      * Companion for Taskmanager: Remove `clean_shutdown`
      
      https://github.com/paritytech/substrate/pull/10314
      
      * Update Substrate
      
      * Remove warning
      58e4ef8e
    • Bernhard Schuster's avatar
      prefer code upgrades in inherent filtering (#4334) · 16fb4698
      Bernhard Schuster authored
      * impl prefered items
      
      Closes #4330
      
      * do not stop attempting to select, just because one did not fit
      
      * doc
      
      * prefered -> preferred
      
      * missing usage of the preferred indices
      
      * sigh
      
      * shuffle is not available for chacha
      
      * remove duplicate weight addition
      
      * ref vs no ref
      16fb4698
    • Bernhard Schuster's avatar
      remove provisioner checks (#4254) · 71e76705
      Bernhard Schuster authored
      
      
      * chore/provisioner: move metrics to a separate module
      
      * avoid the duplicate names
      
      * reduce all checks
      
      * fixup tests
      
      * Update node/core/provisioner/src/lib.rs
      Co-authored-by: default avatarZeke Mostov <z.mostov@gmail.com>
      
      * chore: fmt
      
      * chore: spellcheck
      
      * doc
      
      * remove the enum anti-pattern
      
      * guide update - remove all the responsibilities
      
      * add another trivial check
      
      * Update node/core/provisioner/src/metrics.rs
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      
      * Update roadmap/implementers-guide/src/node/utility/provisioner.md
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      
      * Update node/core/provisioner/src/metrics.rs
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      Co-authored-by: default avatarZeke Mostov <z.mostov@gmail.com>
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      71e76705
    • Zeke Mostov's avatar
      Log para inherent inputs (#4331) · 1ec4819c
      Zeke Mostov authored
      1ec4819c
    • Robert Klotzner's avatar
      Dispute spam protection (#4134) · d488955d
      Robert Klotzner authored
      
      
      * Mostly notes.
      
      * Better error messages.
      
      * Introduce Fatal/NonFatal + drop back channel participation
      
      - Fatal/NonFatal - in order to make it easier to use utility functions.
      - We drop the back channel in dispute participation as it won't be
      needed any more.
      
      * Better error messages.
      
      * Utility function for receiving `CandidateEvent`s.
      
      * Ordering module typechecks.
      
      * cargo fmt
      
      * Prepare spam slots module.
      
      * Implement SpamSlots mechanism.
      
      * Implement queues.
      
      * cargo fmt
      
      * Participation.
      
      * Participation taking shape.
      
      * Finish participation.
      
      * cargo fmt
      
      * Cleanup.
      
      * WIP: Cleanup + Integration.
      
      * Make `RollingSessionWindow` initialized by default.
      
      * Make approval voting typecheck.
      
      * Get rid of lazy_static & fix approval voting tests
      
      * Move `SessionWindowSize` to node primitives.
      
      * Implement dispute coordinator initialization.
      
      * cargo fmt
      
      * Make queues return error instead of boolean.
      
      * Initialized: WIP
      
      * Introduce chain api for getting finalized block.
      
      * Fix ordering to only prune candidates on finalized events.
      
      * Pruning of old sessions in spam slots.
      
      * New import logic.
      
      * Make everything typecheck.
      
      * Fix warnings.
      
      * Get rid of obsolete dispute-participation.
      
      * Fixes.
      
      * Add back accidentelly deleted Cargo.lock
      
      * Deliver disputes in an ordered fashion.
      
      * Add module docs for errors
      
      * Use type synonym.
      
      * hidden docs.
      
      * Fix overseer tests.
      
      * Ordering provider taking `CandidateReceipt`.
      
      ... To be kicked on one next commit.
      
      * Fix ordering to use relay_parent
      
      as included block is not unique per candidate.
      
      * Add comment in ordering.rs.
      
      * Take care of duplicate entries in queues.
      
      * Better spam slots.
      
      * Review remarks + docs.
      
      * Fix db tests.
      
      * Participation tests.
      
      * Also scrape votes on first leaf for good measure.
      
      * Make tests typecheck.
      
      * Spelling.
      
      * Only participate in actual disputes, not on every import.
      
      * Don't account backing votes to spam slots.
      
      * Fix more tests.
      
      * Don't participate if we don't have keys.
      
      * Fix tests, typos and warnings.
      
      * Fix merge error.
      
      * Spelling fixes.
      
      * Add missing docs.
      
      * Queue tests.
      
      * More tests.
      
      * Add metrics + don't short circuit import.
      
      * Basic test for ordering provider.
      
      * Import fix.
      
      * Remove dead link.
      
      * One more dead link.
      Co-authored-by: Lldenaurois's avatarLldenaurois <Ljdenaurois@gmail.com>
      d488955d
    • Bastian Köcher's avatar
      Dependabot: Ignore sub-tokens (#4328) · 9f059fb1
      Bastian Köcher authored
      9f059fb1
    • Cheng JIANG's avatar
      export hrmp config (#4324) · cceb775a
      Cheng JIANG authored
      * export hrmp config
      
      * fix build
      cceb775a
  9. 18 Nov, 2021 6 commits
  10. 17 Nov, 2021 5 commits
  11. 16 Nov, 2021 1 commit
    • Bernhard Schuster's avatar
      move paras inherent filtering to runtime (#4028) · c198d26d
      Bernhard Schuster authored
      
      
      * move things around, add filter methods
      
      * validator keys, modify availability bitfields according to disputes
      
      * simplify, keep the filter -> sanitize generic for both usecases
      
      * minor
      
      * assure tests still work, reduce changeset
      
      * integration
      
      * start entropy passing
      
      * fixins
      
      * compile, 1 failing test
      
      * filter with coverage
      
      * fixins
      
      * Update runtime/parachains/src/paras_inherent.rs
      Co-authored-by: asynchronous rob's avatarRobert Habermeier <rphmeier@gmail.com>
      
      * slip of the pen
      
      * improve test cases
      
      * misc
      
      * fix
      
      * fixins
      
      * test avoid extra into() calls in assert_noop!
      
      * chores
      
      * ff
      
      * test fixup superfluous into call
      
      * chore: pfmt
      
      * improve apply_block_weight_limit to try to maximize the number of sufficiently backed
      
      blocks and add extra bitfields in a round-robin fashion
      
      * new code treats the lack of backed candidates as ok
      
      * Use vrf based entropy
      
      * fixup vrf random
      
      * add warn
      
      * slip of the pen
      
      * fixup
      
      * assure ordering
      
      * rethink apply_weights
      
      * mock
      
      * use a closure as predicate check
      
      * extract and use DisputedBitfield
      
      * chore: simplify
      
      * remove stray dbg
      
      * chore: fmt
      
      * address feedback
      
      * fix test, halfway there
      
      * stage1
      
      * dbg stuff
      
      * make group selection align
      
      * fix session index
      
      * fix wrongly returned candidates
      
      * cleanup
      
      * chore fmt
      
      * fix ensure check
      
      * make good case test work
      
      * more tests for bitfields
      
      * create sanitize_backed_candidates
      
      * fixup tests
      
      * update guide
      
      * add check referenced in the guide
      
      * improve weights code
      
      * fmt
      
      * fixins
      
      * Update roadmap/implementers-guide/src/runtime/inclusion.md
      Co-authored-by: default avatarZeke Mostov <32168567+emostov@users.noreply.github.com>
      
      * compiling + address review
      
      * add comments
      
      * fix weight calc
      
      * address review comments and test failure
      
      * fix
      
      * fix: condition
      
      * Fix random_sel function
      
      * Fix overlength block check
      
      * Zeke + Ladi commit for disputes filtering + integration test builder + runtime benchmarks + integration tests
      
      * Add benchmarks for code upgrades
      
      * Code upgrade bench; Feature gate TestWeightInfo
      
      * Try and make CI happier
      
      * Feature gate enter test to not(benchmarks)
      
      * Make sure no unused imports/fn
      
      * refactor, re-use, the beginning
      
      * Fix issue with frame benchmarking dep compilation
      
      * More precise feature gating for some derives
      
      * integrate piece-wise
      
      * foo
      
      * fixins
      
      * chore fmt
      
      * fixins
      
      * rename const generic
      
      * Update runtime/parachains/src/paras_inherent.rs
      Co-authored-by: default avatarZeke Mostov <z.mostov@gmail.com>
      
      * Fix compilation
      
      * limit to test
      
      * remove unused spam slots
      
      * spellcheck
      
      * remove a tick, fix a typo
      
      * Add Code upgrade weights
      
      * comment improvements + >=
      Co-authored-by: default avatarZeke Mostov <z.mostov@gmail.com>
      
      * remove another tick
      
      * Update runtime/parachains/src/paras_inherent/benchmarking.rs
      Co-authored-by: default avatarZeke Mostov <z.mostov@gmail.com>
      
      * saturating fixins + some spaces
      
      * fix
      
      * benchmarking - preliminary results
      
      * Add training wheels
      
      * Refactor some early exit logic for enter
      
      * Gracefully handle filtering bitfields & candidates (#4280)
      
      This updates the logic for sanitize_bitfields and sanitize_backed_candidates to never error when there is an issue, but instead to simply skip the problematic items.
      
      * Refactor inherent data weight limiting logic (#4287)
      
      * Apply suggestions from code review
      
      * Update runtime/parachains/src/builder.rs
      Co-authored-by: default avatarZeke Mostov <z.mostov@gmail.com>
      
      * Update runtime/parachains/src/builder.rs
      
      * Update runtime/parachains/src/paras_inherent.rs
      
      * final pass
      
      * Run cargo +nightly-2021-10-29 fmt
      
      * Update implementors guide with `sanitize_*` & `enter` (#4294)
      
      * Make spell check happier
      
      * Make wasm runtimes compile with benchmarks enabled (#4303)
      
      * comment stuff out, use old toml
      
      * Seems to be working?
      
      * Remove feature gating from builder
      
      * Remove commented out stuff
      
      * Remove generic from digest
      
      * Update weight files for runtime
      Co-authored-by: asynchronous rob's avatarRobert Habermeier <rphmeier@gmail.com>
      Co-authored-by: default avatarZeke Mostov <32168567+emostov@users.noreply.github.com>
      Co-authored-by: Lldenaurois's avatarLldenaurois <Ljdenaurois@gmail.com>
      Co-authored-by: default avatarZeke Mostov <z.mostov@gmail.com>
      Co-authored-by: default avatarBastian Köcher <info@kchr.de>
      c198d26d