• s0me0ne-unkn0wn's avatar
    Executor Environment parameterization (#6161) · dd0a5566
    s0me0ne-unkn0wn authored
    
    
    * Re-apply changes without Diener, rebase to the lastest master
    
    * Cache pruning
    
    * Bit-pack InstantiationStrategy
    
    * Move ExecutorParams version inside the structure itself
    
    * Rework runtime API and executor parameters storage
    
    * Pass executor parameters through backing subsystem
    
    * Update Cargo.lock
    
    * Introduce `ExecutorParams` to approval voting subsys
    
    * Introduce `ExecutorParams` to dispute coordinator
    
    * `cargo fmt`
    
    * Simplify requests from backing subsys
    
    * Fix tests
    
    * Replace manual config cloning with `.clone()`
    
    * Move constants to module
    
    * Parametrize executor performing PVF pre-check
    
    * Fix Malus
    
    * Fix test runtime
    
    * Introduce session executor params as a constant defined by session info
    pallet
    
    * Use Parity SCALE codec instead of hand-crafted binary encoding
    
    * Get rid of constants; Add docs
    
    * Get rid of constants
    
    * Minor typo
    
    * Fix Malus after rebase
    
    * `cargo fmt`
    
    * Use transparent SCALE encoding instead of explicit
    
    * Clean up
    
    * Get rid of relay parent to session index mapping
    
    * Join environment type and version in a single enum element
    
    * Use default execution parameters if running an old runtime
    
    * `unwrap()` -> `expect()`
    
    * Correct API version
    
    * Constants are back in town
    
    * Use constants for execution environment types
    
    * Artifact separation, first try
    
    * Get rid of explicit version
    
    * PVF execution queue worker separation
    
    * Worker handshake
    
    * Global renaming
    
    * Minor fixes resolving discussions
    
    * Two-stage requesting of executor params to make use of runtime API cache
    
    * Proper error handling in pvf-checker
    
    * Executor params storage bootstrapping
    
    * Propagate migration to v3 network runtimes
    
    * Fix storage versioning
    
    * Ensure `ExecutorParams` serialization determinism; Add comments
    
    * Rename constants to make things a bit more deterministic
    Get rid of stale code
    
    * Tidy up a structure of active PVFs
    
    * Minor formatting
    
    * Fix comment
    
    * Add try-runtime hooks
    
    * Add storage version write on upgrade
    
    Co-authored-by: default avatarAndronik <[email protected]>
    
    * Add pre- and post-upgrade assertions
    
    * Require to specify environment type; Remove redundant `impl`s
    
    * Add `ExecutorParamHash` creation from `H256`
    
    * Fix candidate validation subsys tests
    
    * Return splittable error from executor params request fn
    
    * Revert "Return splittable error from executor params request fn"
    
    This reverts commit a0b274177d8bb2f6e13c066741892ecd2e72a456.
    
    * Decompose approval voting metrics
    
    * Use more relevant errors
    
    * Minor formatting fix
    
    * Assert a valid environment type instead of checking
    
    * Fix `try-runtime` hooks
    
    * After-merge fixes
    
    * Add migration logs
    
    * Remove dead code
    
    * Fix tests
    
    * Fix tests
    
    * Back to the strongly typed implementation
    
    * Promote strong types to executor interface
    
    * Remove stale comment
    
    * Move executor params to `SessionInfo`: primitives and runtime
    
    * Move executor params to `SessionInfo`: node
    
    * Try to bump primitives and API version
    
    * Get rid of `MallocSizeOf`
    
    * Bump target API version to v4
    
    * Make use of session index already in place
    
    * Back to v3
    
    * Fix all the tests
    
    * Add migrations to all the runtimes
    
    * Make use of existing `SessionInfo` in approval voting subsys
    
    * Rename `TARGET` -> `LOG_TARGET`
    
    * Bump all the primitives to v3
    
    * Fix Rococo ParachainHost API version
    
    * Use `RollingSessionWindow` to acquire `ExecutorParams` in disputes
    
    * Fix nits from discussions; add comments
    
    * Re-evaluate queue logic
    
    * Rework job assignment in execution queue
    
    * Add documentation
    
    * Use `RuntimeInfo` to obtain `SessionInfo` (with blackjack and caching)
    
    * Couple `Pvf` with `ExecutorParams` wherever possible
    
    * Put members of `PvfWithExecutorParams` under `Arc` for cheap cloning
    
    * Fix comment
    
    * Fix CI tests
    
    * Fix clippy warnings
    
    * Address nits from discussions
    
    * Add a placeholder for raw data
    
    * Fix non exhaustive match
    
    * Remove redundant reexports and fix imports
    
    * Keep only necessary semantic features, as discussed
    
    * Rework `RuntimeInfo` to support mock implementation for tests
    
    * Remove unneeded bound
    
    * `cargo fmt`
    
    * Revert "Remove unneeded bound"
    
    This reverts commit 932463f26b00ce290e1e61848eb9328632ef8a61.
    
    * Fix PVF host tests
    
    * Fix PVF checker tests
    
    * Fix overseer declarations
    
    * Simplify tests
    
    * `MAX_KEEP_WAITING` timeout based on `BACKGING_EXECUTION_TIMEOUT`
    
    * Add a unit test for varying executor parameters
    
    * Minor fixes from discussions
    
    * Add prechecking max. memory parameter (see paritytech/srlabs_findings#110)
    
    * Fix and improve a test
    
    * Remove `ExecutionEnvironment` and `RawData`
    
    * New primitives versioning in parachain host API
    
    * `disputes()` implementation for Kusama and Polkadot
    
    * Move `ExecutorParams` from `vstaging` to stable primitives
    
    * Move disputes from `vstaging` to stable implementation
    
    * Fix `try-runtime`
    
    * Fixes after merge
    
    * Move `ExecutorParams` to the bottom of `SessionInfo`
    
    * Revert "Move executor params to `SessionInfo`: primitives and runtime"
    
    This reverts commit dfcfb85fefd1c5be6c8a8f72dc09fd1809cfa9ce.
    
    * Always use fresh activated live hash in pvf precheck
    (re-apply 34b09a4c20de17e7926ed942cd0d657d18f743fa)
    
    * Fixing tests (broken commit)
    
    * Fix candidate validation tests
    
    * Fix PVF host test
    
    * Minor fixes
    
    * Address discussions
    
    * Restore migration
    
    * Fix `use` to only include what is needed instead of `*`
    
    * Add comment to never touch `DEFAULT_CONFIG`
    
    * Update migration to set default `ExecutorParams` for `dispute_period`
    sessions back
    
    * Use `earliest_stored_session` instead of calculations
    
    * Nit
    
    * Add logs
    
    * Treat any runtime error as `NotSupported` again
    
    * Always return default executor params if not available
    
    * Revert "Always return default executor params if not available"
    
    This reverts commit b58ac4482ef444c67a9852d5776550d08e312f30.
    
    * Add paritytech/substrate#9997 workaround
    
    * `cargo fmt`
    
    * Remove migration (again!)
    
    * Bump executor params to API v4 (backport from #6698)
    
    ---------
    
    Co-authored-by: default avatarAndronik <[email protected]>
    dd0a5566