1. Apr 21, 2023
    • Marcin S.'s avatar
      PVF: Move PVF workers into separate crate (#7101) · e277f95b
      Marcin S. authored
      * Move PVF workers into separate crate
      
      * Fix indentation
      
      * Fix compilation errors
      
      * Fix more compilation errors
      
      * Rename `worker.rs` files, make host interface to worker more clear
      
      * Fix more compilation errors
      
      * Fix more compilation errors
      
      * Add link to issue
      
      * Address review comments
      
      * Update comment
      e277f95b
  2. Apr 20, 2023
    • Marcin S.'s avatar
      PVF: Don't dispute on missing artifact (#7011) · 0940cdd1
      Marcin S. authored
      * PVF: Don't dispute on missing artifact
      
      A dispute should never be raised if the local cache doesn't provide a certain
      artifact. You can not dispute based on this reason, as it is a local hardware
      issue and not related to the candidate to check.
      
      Design:
      
      Currently we assume that if we prepared an artifact, it remains there on-disk
      until we prune it, i.e. we never check again if it's still there.
      
      We can change it so that instead of artifact-not-found triggering a dispute, we
      retry once (like we do for AmbiguousWorkerDeath, except we don't dispute if it
      still doesn't work). And when enqueuing an execute job, we check for the
      artifact on-disk, and start preparation if not found.
      
      Changes:
      
      - [x] Integration test (should fail without the following changes)
      - [x] Check if artifact exists when executing, prepare if not
      - [x] Return an internal error when file is missing
      - [x] Retry once on internal errors
      - [x] Document design (update impl guide)
      
      * Add some context to wasm error message (it is quite long)
      
      * Fix impl guide
      
      * Add check for missing/inaccessible file
      
      * Add comment referencing Substrate issue
      
      * Add test for retrying internal errors
      
      ---------
      
      Co-authored-by: parity-processbot <>
      0940cdd1
  3. Apr 19, 2023
  4. Apr 17, 2023
  5. Apr 08, 2023
  6. Apr 07, 2023
    • Marcin S.'s avatar
      PVF: Minor refactor in workers code (#7012) · 5889119f
      Marcin S. authored
      * Move version check to `worker_event_loop`
      
      * More minor refactors
      
      - More consistent use of `format_invalid` and `format_internal`.
      - Fix a doc error.
      - Fix some poorly-named local variables.
      5889119f
  7. Apr 05, 2023
  8. Apr 04, 2023
  9. Mar 30, 2023
  10. Mar 29, 2023
    • s0me0ne-unkn0wn's avatar
      Check spawned worker version vs node version before PVF preparation (#6861) · 55b4aceb
      s0me0ne-unkn0wn authored
      
      
      * Check spawned worker version vs node version before PVF preparation
      
      * Address discussions
      
      * Propagate errors and shutdown preparation and execution pipelines properly
      
      * Add logs; Fix execution worker checks
      
      * Revert "Propagate errors and shutdown preparation and execution pipelines properly"
      
      This reverts commit b96cc3160ff58db5ff001d8ca0bfea9bd4bdd0f2.
      
      * Don't try to shut down; report the condition and exit worker
      
      * Get rid of `VersionMismatch` preparation error
      
      * Merge master
      
      * Add docs; Fix tests
      
      * Update Cargo.lock
      
      * Kill again, but only the main node process
      
      * Move unsafe code to a common safe function
      
      * Fix libc dependency error on MacOS
      
      * pvf spawning: Add some logging, add a small integration test
      
      * Minor fixes
      
      * Restart CI
      
      ---------
      
      Co-authored-by: default avatarMarcin S <[email protected]>
      55b4aceb
  11. Mar 23, 2023
  12. Mar 20, 2023
  13. Mar 15, 2023
  14. Mar 14, 2023
  15. Mar 08, 2023
  16. Feb 24, 2023
  17. Feb 20, 2023
  18. Feb 16, 2023
    • Bastian Köcher's avatar
      Change `MaxMemorySize` to `MaxMemoryPages` (#6728) · 83eb8453
      Bastian Köcher authored
      * Change `MaxMemorySize` to `MaxMemoryPages`
      
      We should set the max memory for the executor in pages (64KiB) and not in bytes.
      The wasm memory is always a multiple of a page and we should use the
      same terminology.
      
      * FMT
      
      * Fix warning
      83eb8453
  19. Feb 15, 2023
    • s0me0ne-unkn0wn's avatar
      Executor Environment parameterization (#6161) · dd0a5566
      s0me0ne-unkn0wn authored
      
      
      * Re-apply changes without Diener, rebase to the lastest master
      
      * Cache pruning
      
      * Bit-pack InstantiationStrategy
      
      * Move ExecutorParams version inside the structure itself
      
      * Rework runtime API and executor parameters storage
      
      * Pass executor parameters through backing subsystem
      
      * Update Cargo.lock
      
      * Introduce `ExecutorParams` to approval voting subsys
      
      * Introduce `ExecutorParams` to dispute coordinator
      
      * `cargo fmt`
      
      * Simplify requests from backing subsys
      
      * Fix tests
      
      * Replace manual config cloning with `.clone()`
      
      * Move constants to module
      
      * Parametrize executor performing PVF pre-check
      
      * Fix Malus
      
      * Fix test runtime
      
      * Introduce session executor params as a constant defined by session info
      pallet
      
      * Use Parity SCALE codec instead of hand-crafted binary encoding
      
      * Get rid of constants; Add docs
      
      * Get rid of constants
      
      * Minor typo
      
      * Fix Malus after rebase
      
      * `cargo fmt`
      
      * Use transparent SCALE encoding instead of explicit
      
      * Clean up
      
      * Get rid of relay parent to session index mapping
      
      * Join environment type and version in a single enum element
      
      * Use default execution parameters if running an old runtime
      
      * `unwrap()` -> `expect()`
      
      * Correct API version
      
      * Constants are back in town
      
      * Use constants for execution environment types
      
      * Artifact separation, first try
      
      * Get rid of explicit version
      
      * PVF execution queue worker separation
      
      * Worker handshake
      
      * Global renaming
      
      * Minor fixes resolving discussions
      
      * Two-stage requesting of executor params to make use of runtime API cache
      
      * Proper error handling in pvf-checker
      
      * Executor params storage bootstrapping
      
      * Propagate migration to v3 network runtimes
      
      * Fix storage versioning
      
      * Ensure `ExecutorParams` serialization determinism; Add comments
      
      * Rename constants to make things a bit more deterministic
      Get rid of stale code
      
      * Tidy up a structure of active PVFs
      
      * Minor formatting
      
      * Fix comment
      
      * Add try-runtime hooks
      
      * Add storage version write on upgrade
      
      Co-authored-by: default avatarAndronik <[email protected]>
      
      * Add pre- and post-upgrade assertions
      
      * Require to specify environment type; Remove redundant `impl`s
      
      * Add `ExecutorParamHash` creation from `H256`
      
      * Fix candidate validation subsys tests
      
      * Return splittable error from executor params request fn
      
      * Revert "Return splittable error from executor params request fn"
      
      This reverts commit a0b274177d8bb2f6e13c066741892ecd2e72a456.
      
      * Decompose approval voting metrics
      
      * Use more relevant errors
      
      * Minor formatting fix
      
      * Assert a valid environment type instead of checking
      
      * Fix `try-runtime` hooks
      
      * After-merge fixes
      
      * Add migration logs
      
      * Remove dead code
      
      * Fix tests
      
      * Fix tests
      
      * Back to the strongly typed implementation
      
      * Promote strong types to executor interface
      
      * Remove stale comment
      
      * Move executor params to `SessionInfo`: primitives and runtime
      
      * Move executor params to `SessionInfo`: node
      
      * Try to bump primitives and API version
      
      * Get rid of `MallocSizeOf`
      
      * Bump target API version to v4
      
      * Make use of session index already in place
      
      * Back to v3
      
      * Fix all the tests
      
      * Add migrations to all the runtimes
      
      * Make use of existing `SessionInfo` in approval voting subsys
      
      * Rename `TARGET` -> `LOG_TARGET`
      
      * Bump all the primitives to v3
      
      * Fix Rococo ParachainHost API version
      
      * Use `RollingSessionWindow` to acquire `ExecutorParams` in disputes
      
      * Fix nits from discussions; add comments
      
      * Re-evaluate queue logic
      
      * Rework job assignment in execution queue
      
      * Add documentation
      
      * Use `RuntimeInfo` to obtain `SessionInfo` (with blackjack and caching)
      
      * Couple `Pvf` with `ExecutorParams` wherever possible
      
      * Put members of `PvfWithExecutorParams` under `Arc` for cheap cloning
      
      * Fix comment
      
      * Fix CI tests
      
      * Fix clippy warnings
      
      * Address nits from discussions
      
      * Add a placeholder for raw data
      
      * Fix non exhaustive match
      
      * Remove redundant reexports and fix imports
      
      * Keep only necessary semantic features, as discussed
      
      * Rework `RuntimeInfo` to support mock implementation for tests
      
      * Remove unneeded bound
      
      * `cargo fmt`
      
      * Revert "Remove unneeded bound"
      
      This reverts commit 932463f26b00ce290e1e61848eb9328632ef8a61.
      
      * Fix PVF host tests
      
      * Fix PVF checker tests
      
      * Fix overseer declarations
      
      * Simplify tests
      
      * `MAX_KEEP_WAITING` timeout based on `BACKGING_EXECUTION_TIMEOUT`
      
      * Add a unit test for varying executor parameters
      
      * Minor fixes from discussions
      
      * Add prechecking max. memory parameter (see paritytech/srlabs_findings#110)
      
      * Fix and improve a test
      
      * Remove `ExecutionEnvironment` and `RawData`
      
      * New primitives versioning in parachain host API
      
      * `disputes()` implementation for Kusama and Polkadot
      
      * Move `ExecutorParams` from `vstaging` to stable primitives
      
      * Move disputes from `vstaging` to stable implementation
      
      * Fix `try-runtime`
      
      * Fixes after merge
      
      * Move `ExecutorParams` to the bottom of `SessionInfo`
      
      * Revert "Move executor params to `SessionInfo`: primitives and runtime"
      
      This reverts commit dfcfb85fefd1c5be6c8a8f72dc09fd1809cfa9ce.
      
      * Always use fresh activated live hash in pvf precheck
      (re-apply 34b09a4c20de17e7926ed942cd0d657d18f743fa)
      
      * Fixing tests (broken commit)
      
      * Fix candidate validation tests
      
      * Fix PVF host test
      
      * Minor fixes
      
      * Address discussions
      
      * Restore migration
      
      * Fix `use` to only include what is needed instead of `*`
      
      * Add comment to never touch `DEFAULT_CONFIG`
      
      * Update migration to set default `ExecutorParams` for `dispute_period`
      sessions back
      
      * Use `earliest_stored_session` instead of calculations
      
      * Nit
      
      * Add logs
      
      * Treat any runtime error as `NotSupported` again
      
      * Always return default executor params if not available
      
      * Revert "Always return default executor params if not available"
      
      This reverts commit b58ac4482ef444c67a9852d5776550d08e312f30.
      
      * Add paritytech/substrate#9997 workaround
      
      * `cargo fmt`
      
      * Remove migration (again!)
      
      * Bump executor params to API v4 (backport from #6698)
      
      ---------
      
      Co-authored-by: default avatarAndronik <[email protected]>
      dd0a5566
  20. Feb 14, 2023
    • Marcin S.'s avatar
      Refactor PVF preparation memory stats (#6693) · fd70d012
      Marcin S. authored
      * Refactor PVF preparation memory stats
      
      The original purpose of this change was to gate metrics that are unsupported by
      some systems behind conditional compilation directives (#[cfg]); see
      https://github.com/paritytech/polkadot/pull/6675#discussion_r1099996209.
      
      Then I started doing some random cleanups and simplifications and got a bit
      carried away. 🙈 The code should be overall tidier than before.
      
      Changes:
      - Don't register unsupported metrics (e.g. `max_rss` on non-Linux systems)
      - Introduce `PrepareStats` struct as an abstraction over the `Ok` values of
        `PrepareResult`. It is cleaner, and can be easily modified in the future.
      - Other small changes
      
      * Minor fixes to comments
      
      * Fix compile errors
      
      * Try to fix some Linux errors
      
      * Mep
      
      * Fix candidate-validation tests
      
      * Update docstring
      fd70d012
  21. Feb 09, 2023
  22. Feb 06, 2023
  23. Feb 03, 2023
  24. Jan 17, 2023
    • Gavin Wood's avatar
      XCM v3 (#4097) · 1a1bfd2a
      Gavin Wood authored
      
      
      * cargo fmt
      
      * Create benchmarks for XCM instructions introduced in v3 (#4564)
      
      * Create benchmarks for BurnAsset and ExpectAsset
      
      * Add benchmarks for ExpectOrigin and ExpectError
      
      * Add benchmarks for QueryPallet and ExpectPallet
      
      * Add benchmarks for ReportTransactStatus and ClearTransactStatus
      
      * cargo fmt
      
      * Use AllPalletsWithSystem in mocks
      
      * Update XCM generic benchmarks for westend
      
      * Remove default impls for some XCM weight functions
      
      * Fix compilation error
      
      * Add weight_args helper attribute
      
      * Remove manually written XcmWeightInfo
      
      * Parse trailing comma
      
      * Revert "Add weight_args helper attribute"
      
      This reverts commit 3b7c47a6182e1b9227036c38b406d494c3fcf6fd.
      
      * Fixes
      
      * Fixes
      
      * XCM v3: Introduce querier field into `QueryReponse` (#4732)
      
      * Introduce querier field into QueryReponse
      
      * Convert &Option<MultiLocation> to Option<&MultiLocation>
      
      &Option<T> is almost always never quite useful, most of the time it
      still gets converted to an Option<&T> via `as_ref`, so we should simply
      make functions that accept Option<&T> instead.
      
      * Fix tests
      
      * cargo fmt
      
      * Fix benchmarks
      
      * Appease spellchecker
      
      * Fix test
      
      * Fix tests
      
      * Fix test
      
      * Fix mock
      
      * Fixes
      
      * Fix tests
      
      * Add test for response queriers
      
      * Update xcm/pallet-xcm/src/lib.rs
      
      * Test for non-existence of querier
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * Fixes
      
      * Fixes
      
      * Add `starts_with` function to `MultiLocation` and `Junctions` (#4835)
      
      * add matches_prefix function to MultiLocation and Junctions
      
      * rename matches_prefix to starts_with
      
      * remove unnecessary main in doc comment
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      
      * Make use of starts_with in match_and_split
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * XCM v3: Bridge infrastructure (#4681)
      
      * XCM bridge infrastructure
      
      * Missing bit of cherry-pick
      
      * Revamped XCM proc macros; new NetworkIds
      
      * Fixes
      
      * Formatting
      
      * ExportMessage instruction and config type
      
      * Add MessageExporter definitions
      
      * Formatting
      
      * Missing files
      
      * Fixes
      
      * Initial bridging config API
      
      * Allow for two-stage XCM execution
      
      * Update xcm/src/v3/mod.rs
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * XCM crate building again
      
      * Initial bridging primitive
      
      * Docs
      
      * Docs
      
      * More work
      
      * More work
      
      * Merge branch 'gav-xcm-v3' into gav-xcm-v3-bridging
      
      * Make build
      
      * WithComputedOrigin and SovereignPaidRemoteExporter
      
      * Remove TODOs
      
      * Slim bridge API and tests.
      
      * Fixes
      
      * More work
      
      * First bridge test passing
      
      * Formatting
      
      * Another test
      
      * Next round of bridging tests
      
      * Repot tests
      
      * Cleanups
      
      * Paid bridging
      
      * Formatting
      
      * Tests
      
      * Spelling
      
      * Formatting
      
      * Fees and refactoring
      
      * Fixes
      
      * Formatting
      
      * Refactor SendXcm to become two-phase
      
      * Fix tests
      
      * Refactoring of SendXcm and ExportXcm complete
      
      * Formatting
      
      * Rename CannotReachDestination -> NotApplicable
      
      * Remove XCM v0
      
      * Minor grumbles
      
      * Formatting
      
      * Formatting
      
      * Fixes
      
      * Fixes
      
      * Cleanup XCM config
      
      * Fee handling
      
      * Fixes
      
      * Formatting
      
      * Fixes
      
      * Bump
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * Bump Substrate
      
      * XCM v3: `ExchangeAsset` and Remote-locking (#4945)
      
      * Asset Exchange and Locks
      
      * Make sure XCM typers impl MaxEncodedLen
      
      * Basic implementation for locks
      
      * Bump Substrate
      
      * Missing files
      
      * Use new API
      
      * Introduce  instruction
      
      * Big refactor
      
      * Docs
      
      * Remove deprecated struct
      
      * Remove deprecated struct
      
      * Repot XCM builder tests
      
      * ExchangeAsset test
      
      * Exchange tests
      
      * Locking tests
      
      * Locking tests
      
      * Fixes and tests
      
      * Fixes
      
      * Formatting
      
      * Spelling
      
      * Add simulator test for remote locking
      
      * Fix tests
      
      * Bump
      
      * XCM v3: Support for non-fungibles (#4950)
      
      * NFT support and a test
      
      * New files.
      
      * Integration tests for sending NFTs
      
      * Formatting
      
      * Broken Cargo features
      
      * Use 2021 edition
      
      * Fixes
      
      * Formatting
      
      * Formatting
      
      * Update xcm/xcm-builder/src/asset_conversion.rs
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * Update xcm/xcm-builder/src/nonfungibles_adapter.rs
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * Update xcm/xcm-executor/src/lib.rs
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Formatting
      
      * Fixes
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * XCM v3: Context & ID hash (#4756)
      
      * send_xcm returns message hash
      
      * cargo fmt
      
      * Create topic register and instructions
      
      * Fix weights
      
      * Use tabs
      
      * Sketch out XcmContext
      
      * Fix doc test
      
      * Add the XCM context as a parameter to executor trait fns
      
      * Fixes
      
      * Add XcmContext parameter
      
      * Revert adding context as an arg to SendXcm trait methods
      
      * Revert adding context argument to ConvertOrigin trait methods
      
      * cargo fmt
      
      * Do not change the API of XcmExecutor::execute
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Remove convenience method
      
      * Fixes
      
      * Fixes
      
      * cargo fmt
      
      * Fixes
      
      * Add benchmarks for XCM topic instructions
      
      * cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs
      
      * Remove context argument on FilterAssetLocation
      
      * Fixes
      
      * Remove unused import
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Accept XCM hash parameter in ExecuteXcm trait methods
      
      * cargo fmt
      
      * Properly enable sp-io/std
      
      * Fixes
      
      * default-features = false
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Make XcmContext optional in withdraw_asset
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Modify tests to check for the correct XCM hash
      
      * Small refactor
      
      * cargo fmt
      
      * Check for expected hash in xcm-builder unit tests
      
      * Add doc comment for the optionality of the XCM context in withdraw_asset
      
      * Update xcm/src/v3/traits.rs
      
      * Update xcm/src/v3/traits.rs
      
      * Store XcmContext and avoid rebuilding
      
      * Use ref for XcmContext
      
      * Formatting
      
      * Fix incorrect hash CC @KiChjang
      
      
      
      * Refactor and make clear fake hashes
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Fix broken hashing
      
      * Docs
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Formatting
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Remove unknowable hash
      
      * Formatting
      
      * Use message hash for greater identifiability
      
      * Formatting
      
      * Fixes
      
      * Formatting
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      Co-authored-by: default avatarParity Bot <[email protected]>
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Formatting
      
      * Fixes
      
      * Formatting
      
      * Fixes
      
      * Fixes
      
      * Formatting
      
      * Formatting
      
      * Remove horrible names
      
      * Bump
      
      * Remove InvertLocation trait (#5092)
      
      * Remove InvertLocation trait
      
      * Remove unneeded functions
      
      * Formatting
      
      * Fixes
      
      * Remove XCMv1 (#5094)
      
      * Remove XCMv1
      
      * Remove XCMv1
      
      * Formatting
      
      * Fixes
      
      * Fixes
      
      * Formatting
      
      * derive serialize/deserialize for xcm primitives (#5036)
      
      * derive serialize/deserialize for xcm primitives
      
      * derive serialize/deserialize for xcm primitives
      
      * update v3
      
      * update v2
      
      Co-authored-by: default avatarGav Wood <[email protected]>
      
      * Update lock
      
      * Fixes
      
      * Add benchmarks for the ExchangeAsset instruction
      
      * `AliasOrigin` instruction stub (#5122)
      
      * AliasOrigin instruction stub
      
      * Fixes
      
      * Fixes
      
      * Update substrate
      
      * Fixes
      
      * Ensure same array length before using copy_from_slice
      
      * Fixes
      
      * Add benchmarks for the UniversalOrigin instruction
      
      * Remove unused import
      
      * Remove unused import
      
      * Add benchmarks for SetFeesMode instruction
      
      * Add benchmarks for asset (un)locking instructions
      
      * Leave AliasOrigin unbenchmarked
      
      * Fixes after merge
      
      * cargo fmt
      
      * Fixes
      
      * Fixes
      
      * Set TrustedReserves to None on both Kusama and Westend
      
      * Remove extraneous reserve_asset_deposited benchmark
      
      * Fix universal_origin benchmark
      
      * cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark pallet --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs
      
      * Don't rely on skipped benchmark functions
      
      * Fixes
      
      * cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark pallet --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/kusama/src/weights/xcm/pallet_xcm_benchmarks_generic.rs
      
      * Fix unused variables
      
      * Fixes
      
      * Spelling
      
      * Fixes
      
      * Fix codec index of VersionedXcm
      
      * Allows to customize how calls are dispatched from XCM (#5657)
      
      * CallDispatcher trait
      
      * fmt
      
      * unused import
      
      * fix test-runtime
      
      * remove JustDispatch type
      
      * fix typo in test-runtime
      
      * missing CallDispatcher
      
      * more missing CallDispatcher
      
      * Update comment `NoteAssetLocked` -> `NoteUnlockable`
      
      * Fixes
      
      * Fixes
      
      * Adjust MultiAssets weights based on new wild card variants
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Some late fixes for XCMv3 (#5237)
      
      * Maximise chances that trapped assets can be reclaimed
      
      * Do origin check as part of ExportMessage for security
      
      * Formatting
      
      * Fixes
      
      * Cleanup export XCM APIs
      
      * Formatting
      
      * Update xcm/src/v3/junctions.rs
      
      * UnpaidExecution instruction and associated barrier.
      
      * Tighten barriers (ClearOrigin/QueryResponse)
      
      * Allow only 1 ClearOrigin instruction in AllowTopLevelPaidExecutionFrom
      
      * Bi-directional teleport accounting
      
      * Revert other fix
      
      * Build fixes]
      
      * Tests build
      
      * Benchmark fixes
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      
      * Update Substrate
      
      * Re-export `pub` stuff from universal_exports.rs + removed unecessary clone (#6145)
      
      * Re-export `pub` stuff from universal_exports.rs
      
      * Removed unnecessary clone
      
      * Use 2D weights in XCM v3 (#6134)
      
      * Depend upon sp-core instead of sp-runtime
      
      * Make sp-io a dev-dependency
      
      * Use 2D weights in XCM v3
      
      * cargo fmt
      
      * Add XCM pallet migration to runtimes
      
      * Use from_parts
      
      * cargo fmt
      
      * Fixes
      
      * cargo fmt
      
      * Remove XCMWeight import
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Use translate in migration
      
      * Increase max upward message size in tests
      
      * Fix doc test
      
      * Remove most uses of from_ref_time
      
      * cargo fmt
      
      * Fixes
      
      * Fixes
      
      * Add extrinsic benchmarking to XCM pallet
      
      * cargo fmt
      
      * Fixes
      
      * Use old syntax
      
      * cargo fmt
      
      * Fixes
      
      * Remove hardcoded weights
      
      * Add XCM pallet to benchmarks
      
      * Use successful origin
      
      * Fix weird type parameter compilation issue
      
      * Fixes
      
      * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
      
      * Use benchmarked XCM pallet weights
      
      * Fixes
      
      * Fixes
      
      * Use override instead of skip
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
      
      * Fixes
      
      * ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
      
      Co-authored-by: command-bot <>
      
      * Replace Weight::MAX with 100b weight units
      
      * Add test to ensure all_gte in barriers is correct
      
      * Update xcm/src/v3/junction.rs
      
      Co-authored-by: default avatarasynchronous rob <[email protected]>
      
      * Add more weight tests
      
      * cargo fmt
      
      * Create thread_local in XCM executor to limit recursion depth (#6304)
      
      * Create thread_local in XCM executor to limit recursion depth
      
      * Add unit test for recursion limit
      
      * Fix statefulness in tests
      
      * Remove panic
      
      * Use defer and environmental macro
      
      * Fix the implementation
      
      * Use nicer interface
      
      * Change ThisNetwork to AnyNetwork
      
      * Move recursion check up to top level
      
      * cargo fmt
      
      * Update comment
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      
      * Add upper limit on the number of overweight messages in the queue (#6298)
      
      * Add upper limit on the number of ovwerweight messages in the queue
      
      * Add newline
      
      * Introduce whitelist for Transact and limit UMP processing to 10 messages per block (#6280)
      
      * Add SafeCallFilter to XcmConfig
      
      * Limit UMP to receive 10 messages every block
      
      * Place 10 message limit on processing instead of receiving
      
      * Always increment the message_processed count whenever a message is processed
      
      * Add as_derivative to the Transact whitelist
      
      * cargo fmt
      
      * Fixes
      
      * Update xcm/xcm-builder/src/universal_exports.rs
      
      Co-authored-by: default avatarBranislav Kontur <[email protected]>
      
      * Fixes
      
      * Fixes
      
      * Remove topic register and instead use the topic field in XcmContext
      
      * Derive some common traits for DispatchBlobError
      
      * Fixes
      
      * cargo fmt
      
      * Fixes
      
      * Fixes
      
      * Fix comments
      
      * Fixes
      
      * Introduce WithOriginFilter and apply it as the CallDispatcher for runtimes
      
      * Fixes
      
      * Appease clippy and fixes
      
      * Fixes
      
      * Fix more clippy issues
      
      * Fixes
      
      * ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
      
      * Add benchmark function for ExportMessage
      
      * Fix comment
      
      * Add upper limit to DownwardMessageQueues size
      
      * Add max size check for queue in can_queue_downward_message
      
      * Fixes
      
      * Make Transact runtime call configurable
      
      * Return Weight::MAX when there is no successful send XCM origin
      
      * Update substrate
      
      * Fixes
      
      * Fixes
      
      * Remove ExportMessage benchmark
      
      * Remove assertion on Transact instruction benchmark
      
      * Make reachable destination configurable in XCM pallet benchmarks
      
      * Fixes
      
      * Fixes
      
      * Remove cfg attribute in fuzzer
      
      * Fixes
      
      * Remove cfg attribute for XCM pallet in test runtime
      
      * Fixes
      
      * Use ReachableDest where possible
      
      * Fixes
      
      * Add benchmark for UnpaidExecution
      
      * Update substrate
      
      * Ensure benchmark functions pass filters
      
      * Add runtime-benchmarks feature to fuzzer
      
      * Ensure FixedRateOfFungible accounts for proof size weights
      
      * cargo fmt
      
      * Whitelist remark_with_event when runtime-benchmarks feature is enabled
      
      * Use remark_with_event for Transact benchmarks
      
      * Fix Cargo.lock
      
      * Allow up to 3 DescendOrigin instructions before UnpaidExecution
      
      * cargo fmt
      
      * Edit code comment
      
      * Check check_origin for unpaid execution privilege
      
      * Fixes
      
      * Small nits for xcm-v3 (#6408)
      
      * Add possibility to skip benchmark for export_message
      
      * ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic
      
      * Revert
      
      * ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic
      
      * Add HaulBlobError to `fn haul_blob`
      
      * ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic
      
      Co-authored-by: command-bot <>
      
      * Revert changes to UnpaidExecution
      
      * Change AllowUnpaidExecutionFrom to be explicit
      
      * Fix log text
      
      * cargo fmt
      
      * Add benchmarks for XCM pallet version migration (#6448)
      
      * Add benchmarks for XCM pallet version migration
      
      * cargo fmt
      
      * Fixes
      
      * Fixes
      
      * Fixes
      
      * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
      
      * Fix benchmarks
      
      * Fix benchmarks
      
      * ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm
      
      * ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
      
      Co-authored-by: command-bot <>
      
      * Merge remote-tracking branch 'origin/master' into gav-xcm-v3
      
      * Fixes
      
      * Fix comments (#6470)
      
      * Specify Ethereum networks by their chain id (#6286)
      
      Co-authored-by: default avatarSquirrel <[email protected]>
      
      * Use  for Kusama
      
      * Use WithComputedOrigin for Polkadot, Rococo and Westend
      
      * Update lock
      
      * Fix warning
      
      * Update xcm/pallet-xcm/src/tests.rs
      
      Co-authored-by: default avatarSquirrel <[email protected]>
      
      * Update runtime/parachains/src/ump/migration.rs
      
      Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Update xcm/pallet-xcm/src/migration.rs
      
      Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Fixes
      
      * cargo fmt
      
      * Typo
      
      * Update xcm/src/v3/mod.rs
      
      Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Docs
      
      * Docs
      
      * Docs
      
      * Docs
      
      * Docs
      
      * Update xcm/src/v3/multiasset.rs
      
      Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      
      * Add tests for MultiAssets::from_sorted_and_deduplicated
      
      * Fail gracefully when same instance NFTs are detected during push
      
      * Update Substrate to fix benchmarks
      
      * Apply suggestions from code review
      
      * Update runtime/kusama/src/xcm_config.rs
      
      * Rename arguments
      
      * Attempt to fix benchmark
      
      * ".git/.scripts/commands/bench/bench.sh" runtime polkadot-dev runtime_parachains::ump
      
      * Use actual weights for UMP pallet in Polkadot
      
      * ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::ump
      
      * ".git/.scripts/commands/bench/bench.sh" runtime westend-dev runtime_parachains::ump
      
      * ".git/.scripts/commands/bench/bench.sh" runtime rococo-dev runtime_parachains::ump
      
      Co-authored-by: default avatarKeith Yeung <[email protected]>
      Co-authored-by: default avatarAlexander Popiak <[email protected]>
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      Co-authored-by: default avatarParity Bot <[email protected]>
      Co-authored-by: default avatarstanly-johnson <[email protected]>
      Co-authored-by: default avatarnanocryk <[email protected]>
      Co-authored-by: default avatarBranislav Kontur <[email protected]>
      Co-authored-by: default avatarasynchronous rob <[email protected]>
      Co-authored-by: command-bot <>
      Co-authored-by: default avatarVincent Geddes <[email protected]>
      Co-authored-by: default avatarSquirrel <[email protected]>
      Co-authored-by: default avatarOliver Tale-Yazdi <[email protected]>
      Co-authored-by: default avatarShawn Tabrizi <[email protected]>
      1a1bfd2a
  25. Jan 12, 2023
    • Marcin S.'s avatar
      pre-checking: Reject failed PVFs (#6492) · 2efa3bab
      Marcin S. authored
      
      
      * pre-checking: Reject failed PVFs
      
      * paras: immediately reject any PVF that cannot reach a supermajority
      
      * Make the `quorum` reject condition a bit more clear semantically
      
      * Add comment
      
      * Update implementer's guide
      
      * Update a link
      
      Not related to the rest of the PR, but I randomly noticed and fixed this.
      
      * Update runtime/parachains/src/paras/tests.rs
      
      Co-authored-by: default avatars0me0ne-unkn0wn <[email protected]>
      
      * Remove unneeded loop
      
      * Log PVF retries using `info!`
      
      * Change retry logs to `warn!` and add preparation failure log
      
      * Log PVF execution failure
      
      * Clarify why we reject failed PVFs
      
      * Fix PVF reject runtime benchmarks
      
      Co-authored-by: default avatars0me0ne-unkn0wn <[email protected]>
      2efa3bab
  26. Jan 11, 2023
  27. Jan 10, 2023
    • Marcin S.'s avatar
      Minor fixes (#6533) · 3cf18f11
      Marcin S. authored
      * Minor fixes
      
      * Fix compile errors
      3cf18f11
    • Marcin S.'s avatar
      Replace async-std with tokio in PVF subsystem (#6419) · 44fd9566
      Marcin S. authored
      
      
      * Replace async-std with tokio in PVF subsystem
      
      * Rework workers to use `select!` instead of a mutex
      
      The improvement in code readability is more important than the thread overhead.
      
      * Remove unnecessary `fuse`
      
      * Add explanation for `expect()`
      
      * Update node/core/pvf/src/worker_common.rs
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      
      * Update node/core/pvf/src/worker_common.rs
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      
      * Address some review comments
      
      * Shutdown tokio runtime
      
      * Run cargo fmt
      
      * Add a small note about retries
      
      * Fix up merge
      
      * Rework `cpu_time_monitor_loop` to return when other thread finishes
      
      * Add error string to PrepareError::IoErr variant
      
      * Log when artifacts fail to prepare
      
      * Fix `cpu_time_monitor_loop`; fix test
      
      * Fix text
      
      * Fix a couple of potential minor data races.
      
      First data race was due to logging in the CPU monitor thread even if the
      job (other thread) finished. It can technically finish before or after the log.
      
      Maybe best would be to move this log to the `select!`s, where we are guaranteed
      to have chosen the timed-out branch, although there would be a bit of
      duplication.
      
      Also, it was possible for this thread to complete before we executed
      `finished_tx.send` in the other thread, which would trigger an error as the
      receiver has already been dropped. And right now, such a spurious error from
      `send` would be returned even if the job otherwise succeeded.
      
      * Update Cargo.lock
      
      Co-authored-by: default avatarBastian Köcher <[email protected]>
      44fd9566
  28. Jan 04, 2023
  29. Dec 20, 2022
    • Marcin S.'s avatar
      PVF preparation: do not conflate errors (#6384) · e0a0475a
      Marcin S. authored
      * PVF preparation: do not conflate errors
      
      + Adds some more granularity to the prepare errors.
      + Better distinguish whether errors occur on the host side or the worker.
      + Do not kill the worker if the error happened on the host side.
      + Do not retry preparation if the error was `Panic`.
      + Removes unnecessary indirection with `Selected` type.
      
      * Add missing docs, resolve TODOs
      
      * Address review comments and remove TODOs
      
      * Fix error in CI
      
      * Undo unnecessary change
      
      * Update couple of comments
      
      * Don't return error for stream shutdown
      
      * Update node/core/pvf/src/worker_common.rs
      e0a0475a
  30. Dec 06, 2022
  31. Dec 05, 2022
  32. Nov 30, 2022
    • Marcin S.'s avatar
      Use CPU clock timeout for PVF jobs (#6282) · 28a4e909
      Marcin S. authored
      * Put in skeleton logic for CPU-time-preparation
      
      Still needed:
      - Flesh out logic
      - Refactor some spots
      - Tests
      
      * Continue filling in logic for prepare worker CPU time changes
      
      * Fix compiler errors
      
      * Update lenience factor
      
      * Fix some clippy lints for PVF module
      
      * Fix compilation errors
      
      * Address some review comments
      
      * Add logging
      
      * Add another log
      
      * Address some review comments; change Mutex to AtomicBool
      
      * Refactor handling response bytes
      
      * Add CPU clock timeout logic for execute jobs
      
      * Properly handle AtomicBool flag
      
      * Use `Ordering::Relaxed`
      
      * Refactor thread coordination logic
      
      * Fix bug
      
      * Add some timing information to execute tests
      
      * Add section about the mitigation to the IG
      
      * minor: Change more `Ordering`s to `Relaxed`
      
      * candidate-validation: Fix build errors
      28a4e909
    • alexgparity's avatar
      Clippyfy (#6341) · 9ea14e66
      alexgparity authored
      
      
      * Add clippy config and remove .cargo from gitignore
      
      * first fixes
      
      * Clippyfied
      
      * Add clippy CI job
      
      * comment out rusty-cachier
      
      * minor
      
      * fix ci
      
      * remove DAG from check-dependent-project
      
      * add DAG to clippy
      
      Co-authored-by: default avataralvicsam <[email protected]>
      9ea14e66
    • Mara Broda's avatar
      sync versions with current release (0.9.33) (#6363) · b76086c6
      Mara Broda authored
      * westend: update transaction version
      
      * polkadot: update transaction version
      
      * kusama: update transaction version
      
      * Bump spec_version to 9330
      
      * bump versions to 0.9.33
      b76086c6
  33. Nov 28, 2022