1. 30 Nov, 2021 2 commits
    • Sergey Pepyakin's avatar
      pvf-precheck: Add `sign` in subsystem-util (#4407) · e4580d09
      Sergey Pepyakin authored
      Right now, most of operations that sign stuff in polkadot protocol are
      handled by a very convenient tool - `Signed`. However `Signed` assumes
      that whatever is signed is anchored to some `parent_hash` which works
      for most cases, but does not work for others.
      One instance of such a case is pre-checking (#3211). There validators
      submit signed votes on-chain. A vote is valid for the entire session. If
      we were to use `Signed` we would have to root a vote in some block of
      that session and during vote verification check that this block is
      indeed within the session. This is especially annoying since we agreed
      to use unsigned extrinsics to submit votes and we need to make the
      unsigned extrinsic validation as slim as possible.
      (FWIW, the definition of a pre-checking vote can be seen in the next
      diff in the stack)
      That's the reason why we opted-out from using `Signed` for pre-checking
      and decided to go with the manual signing approach. Almost every piece
      of machinery is in place except for signing which is presented in this
    • Bernhard Schuster's avatar
      make chain selection typed and more explicit in naming and logging (#4345) · a64476f1
      Bernhard Schuster authored
      * minor: assure conditions match
      This simplifies visual integrity checks that an overseer is connected
      when it has to be.
      * fix: avoid printing a misleading log in case of the disabled disputes feature
      * chore: comments
      * add expressive types for the selection algorithm
  2. 29 Nov, 2021 4 commits
  3. 28 Nov, 2021 1 commit
  4. 27 Nov, 2021 3 commits
  5. 26 Nov, 2021 4 commits
    • Sergey Pepyakin's avatar
      Do not log PVF prunning every hour (#4372) · e83aebaf
      Sergey Pepyakin authored
      This lowers the level of the PVF pruning.
      Closes https://github.com/paritytech/polkadot/issues/4361
    • Sergey Pepyakin's avatar
      Impose new restrictions on paras init and cleanup (#4360) · 62915a68
      Sergey Pepyakin authored
      * Impose new restrictions on paras init and cleanup
      For upcoming PVF pre-checking feature we will need to impose a couple of
      new restrictions for:
      - `schedule_para_initialize`.
      - `schedule_para_cleanup`.
      Specifically, for the former we do not want to allow registration of
      wasm blob that is empty, i.e. 0 bytes. While that currently already
      does not make a lot of sense, it allows us to simplify the PVF
      pre-checking logic: if this PR is deployed before the following changes
      for PVF prechecking then we can be sure that no paras onboarding have to
      have to go through the PVF pre-checking. In case, we deploy it
      altogether this property will allow us to distingush paras that came in
      before PVF pre-checking.
      For `schedule_para_cleanup` we do not want to allow offboarding of paras
      that are undergoing the upgrade process. While this is not a harsh
      restriction this change allows us to avoid making the PVF prechecking
      more complicated than it has to be.
      * Add a test for schedule_para_initialize
      * Link to `ParaLifecycle::is_stable` in docs.
      * `schedule_para_{init,cleanup}` docs
      Now they link to their original declarations in the pallet for more
    • sandreim's avatar
      Fix Provisioner dispute metrics naming (#4374) · 8d6c8826
      sandreim authored
      * update dispute metric names
      Signed-off-by: default avatarAndrei Sandu <andrei-mihail@parity.io>
      * update
      Signed-off-by: default avatarAndrei Sandu <andrei-mihail@parity.io>
    • Keith Yeung's avatar
      Update W3F URL links (#4376) · e90ee611
      Keith Yeung authored
  6. 25 Nov, 2021 2 commits
  7. 24 Nov, 2021 8 commits
  8. 23 Nov, 2021 1 commit
    • Zeke Mostov's avatar
      Inherent filtering follow up (#4305) · 89529740
      Zeke Mostov authored
      * Add feature more feature gating for benchmarking + tests
      * New line
      * cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
      * cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs
      * Do not assume we use max validators per core
      * Use kusama weights for rococo (hopefully temp)
      * cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras_inherent.rs
      * Add more validity votes when neccesary
      * Some fixes for the last commit
      * Restore westend weights
      * cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras_inherent.rs
      * Revert bad westend weights write
      * Make sure to update val idx before skipping
      * Fix validity vote range to max at group size'
      * Temp setup for rococo
      * cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_paras_inherent.rs --header=./file_header.txt
      * Augment generated Rococo weights
      * Make it compile
      * Revert range for enter_backed_candidates_variable
      * Delete runtime/kusama/src/weights/runtime_paras_paras_inherent.rs
      Co-authored-by: default avatarParity Bot <admin@parity.io>
  9. 22 Nov, 2021 3 commits
  10. 20 Nov, 2021 2 commits
    • Javier Viola's avatar
      introduce malus + zombienet based integration tests (#4131) · 2ffb293e
      Javier Viola authored
      * test/malus: craft the first maliciously disputing actor
      * initial draft
      * Add Dockerfile and instructions how to use it to build malus image locally
      * Forgot one flag for the build cmd
      * we are not docker specific, we are happy to use any containerruntime
      * shuffle things around
      * add initial tera based integration test
      * chores
      * fixins
      * simple setup to start
      * other samples (WIP)
      * add Docker version with cargo-chef
      * update substarte and small change of orders in commands in the container file
      * metrics one
      * fmt
      * minor
      * fixin
      * fix metric names
      * -d
      * add open gauge
      * fmt
      * spellcheck
      * fix test
      * adjust to changed error messages
      * refactor, more malus impls
      * more malus changes
      * foo
      * minor cleanup
      * suggest garbage candidate
      * chore
      * fix suggest garabge malus
      * malus: back garbage candidate
      * cargo lock
      * re-introduce metrics
      * chore: cargo fmt
      * undoe 1.54.0 output, CI uses 1.53.0 rustc
      * update location of js types
      * Fix trybuild
      * add tag to image name also; this will be replaced in the prod version
      * Tests fixed
      * add some fix me
      * add dockerfile for ci
      * Add docker file for malus for ci
      * use variables in .toml file
      * add chnages for malus test
      * some fixes
      * some more fixes
      * Update .gitlab-ci.yml
      * add local build for polkadot and malus
      * some fixes
      * enable disputes feature in CI
      * ok, ok
      * rename: MsgFilter -> MessageInterceptor
      * remove TODO that would not have worked
      * intercept
      * refactor
      * fix README and containers
      * fix
      * chore: cargo fmt
      * avoid some more malus-$VARIANT references
      * fix argument order
      * chore: add about
      * Update sanity check in relay chain selection
      * fix order, add dispute-unavailable-block malus
      * fixup: avoid underflow issue
      * it's all u32
      * fix conditional use
      * Revert "it's all u32"
      This reverts commit 6b3ae25b.
      * Revert "fixup: avoid underflow issue"
      This reverts commit 336cbe29.
      * Revert "Update sanity check in relay chain selection"
      This reverts commit 970647f3
      * update the malus bin
      * Update node/malus/integrationtests/0003-dispute-unavailable-block.feature
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      * add some FIXME reminders
      * update path to index.js
      * Update .gitlab-ci.yml
      * Update node/malus/integrationtests/0001-dispute-valid-block.toml
      * try 1: make malus test run
      * chore: cargo fmt
      * temporary fix
      * use subcommand syntax from latest gurke
      * cargo +nightly fmt
      * add collator to a a test
      * docs: add env vars to README
      * update ci to run dispute-valid-block test
      * needs the polkadot image
      * Fix path for nodejs container
      * post merge fix
      * download proper dir  containg configs for malus test
      * update the malus ci job
      * rm a whitespace
      * temp build for malus
      * use correct build command for temp malus
      * remove subcommands for now
      * set max validators per core in the default HostConfig
      * tabs
      * update beefy
      * fixup
      * fixup II
      * make one variant compile
      * make other variants compile
      * revert changes to chain_spec
      * fmt
      * build malus image from polkadot-test-malus again
      * revert unrelated changes
      * try fixing build-malus job
      * Revert "remove subcommands for now"
      This reverts commit 5d8292bc
      * try fixing build-malus job II
      * MVP working dispute-ancestor
      * renames
      * fix PVF execution on malus
      * fix test
      * fix typo
      * fmt
      * checkmate
      * try something
      * make it actually work
      * some tweaks to 01 feature test
      * fmt
      * sleep a bit more
      * complete wococoization
      * some tweaks to 01 feature test
      * typo fix
      * use correct metric names
      * fix
      * ffs
      * .
      * try some rearrangement
      * Attempt to wait till initial node bootstrap in test
      * Fix test syntax
      * Run malus tests with v2 script
      * Proper symlink created
      * simnet v14
      * add zombienet tests
      * add zombie net test - draft
      * add more tests to dispute suite
      * add within to fix tests
      * replace test directory and start test migration
      * migrate all the tests
      * add timeout to tests
      * reduce debug
      * make easy to test in dev
      * set appropriated debug
      * use image from ci
      * fix config for test
      * set images from ci
      * fix config
      * add COLIMAGE env
      * tweek tests
      * reduce debug
      * typo
      * wip, migrate old test to zombie-net
      * adjunt test config for zombie-net
      * run mauls 0001 test only
      * clean and setup smoke-test in zombie-net
      * add extra time to assertinons
      * clean code to merge and improve README
      * add info to access logs
      * improved readme
      * merge master and resolve conflicts
      * Update zombienet_tests/README.md
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      * clean and consolidate zombienet name
      * change runner in gitlab
      * add comment explain why we use wococo
      * change tag for runner
      * remove unused tests
      * remove dup Dockerfile and update description
      * fmt
      * fix compilation post-merge
      * fmt
      * cut me Some slack
      Co-authored-by: default avatarBernhard Schuster <bernhard@ahoi.io>
      Co-authored-by: Radu Popa's avatarradupopa2010 <radupopa2010@yahoo.com>
      Co-authored-by: default avatarBastian Köcher <info@kchr.de>
      Co-authored-by: default avatarAnton Gavrilov <AntonE.Gavrilov@gmail.com>
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      Co-authored-by: Lldenaurois's avatarLldenaurois <Ljdenaurois@gmail.com>
    • Lldenaurois's avatar
      Replicate Rob's PR (#4337) · cd9929fb
      Lldenaurois authored
  11. 19 Nov, 2021 7 commits
    • Bastian Köcher's avatar
      Companion for Taskmanager: Remove `clean_shutdown` (#4336) · 58e4ef8e
      Bastian Köcher authored
      * Companion for Taskmanager: Remove `clean_shutdown`
      * Update Substrate
      * Remove warning
    • Bernhard Schuster's avatar
      prefer code upgrades in inherent filtering (#4334) · 16fb4698
      Bernhard Schuster authored
      * impl prefered items
      Closes #4330
      * do not stop attempting to select, just because one did not fit
      * doc
      * prefered -> preferred
      * missing usage of the preferred indices
      * sigh
      * shuffle is not available for chacha
      * remove duplicate weight addition
      * ref vs no ref
    • Bernhard Schuster's avatar
      remove provisioner checks (#4254) · 71e76705
      Bernhard Schuster authored
      * chore/provisioner: move metrics to a separate module
      * avoid the duplicate names
      * reduce all checks
      * fixup tests
      * Update node/core/provisioner/src/lib.rs
      Co-authored-by: default avatarZeke Mostov <z.mostov@gmail.com>
      * chore: fmt
      * chore: spellcheck
      * doc
      * remove the enum anti-pattern
      * guide update - remove all the responsibilities
      * add another trivial check
      * Update node/core/provisioner/src/metrics.rs
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      * Update roadmap/implementers-guide/src/node/utility/provisioner.md
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      * Update node/core/provisioner/src/metrics.rs
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
      Co-authored-by: default avatarZeke Mostov <z.mostov@gmail.com>
      Co-authored-by: Andronik Ordian's avatarAndronik Ordian <write@reusable.software>
    • Zeke Mostov's avatar
      Log para inherent inputs (#4331) · 1ec4819c
      Zeke Mostov authored
    • Robert Klotzner's avatar
      Dispute spam protection (#4134) · d488955d
      Robert Klotzner authored
      * Mostly notes.
      * Better error messages.
      * Introduce Fatal/NonFatal + drop back channel participation
      - Fatal/NonFatal - in order to make it easier to use utility functions.
      - We drop the back channel in dispute participation as it won't be
      needed any more.
      * Better error messages.
      * Utility function for receiving `CandidateEvent`s.
      * Ordering module typechecks.
      * cargo fmt
      * Prepare spam slots module.
      * Implement SpamSlots mechanism.
      * Implement queues.
      * cargo fmt
      * Participation.
      * Participation taking shape.
      * Finish participation.
      * cargo fmt
      * Cleanup.
      * WIP: Cleanup + Integration.
      * Make `RollingSessionWindow` initialized by default.
      * Make approval voting typecheck.
      * Get rid of lazy_static & fix approval voting tests
      * Move `SessionWindowSize` to node primitives.
      * Implement dispute coordinator initialization.
      * cargo fmt
      * Make queues return error instead of boolean.
      * Initialized: WIP
      * Introduce chain api for getting finalized block.
      * Fix ordering to only prune candidates on finalized events.
      * Pruning of old sessions in spam slots.
      * New import logic.
      * Make everything typecheck.
      * Fix warnings.
      * Get rid of obsolete dispute-participation.
      * Fixes.
      * Add back accidentelly deleted Cargo.lock
      * Deliver disputes in an ordered fashion.
      * Add module docs for errors
      * Use type synonym.
      * hidden docs.
      * Fix overseer tests.
      * Ordering provider taking `CandidateReceipt`.
      ... To be kicked on one next commit.
      * Fix ordering to use relay_parent
      as included block is not unique per candidate.
      * Add comment in ordering.rs.
      * Take care of duplicate entries in queues.
      * Better spam slots.
      * Review remarks + docs.
      * Fix db tests.
      * Participation tests.
      * Also scrape votes on first leaf for good measure.
      * Make tests typecheck.
      * Spelling.
      * Only participate in actual disputes, not on every import.
      * Don't account backing votes to spam slots.
      * Fix more tests.
      * Don't participate if we don't have keys.
      * Fix tests, typos and warnings.
      * Fix merge error.
      * Spelling fixes.
      * Add missing docs.
      * Queue tests.
      * More tests.
      * Add metrics + don't short circuit import.
      * Basic test for ordering provider.
      * Import fix.
      * Remove dead link.
      * One more dead link.
      Co-authored-by: Lldenaurois's avatarLldenaurois <Ljdenaurois@gmail.com>
    • Bastian Köcher's avatar
      Dependabot: Ignore sub-tokens (#4328) · 9f059fb1
      Bastian Köcher authored
    • Cheng JIANG's avatar
      export hrmp config (#4324) · cceb775a
      Cheng JIANG authored
      * export hrmp config
      * fix build
  12. 18 Nov, 2021 3 commits