• Kian Paimani's avatar
    Trie version migration pallet (#10073) · 6cfdbe5c
    Kian Paimani authored
    
    
    * starting
    
    * Updated from other branch.
    
    * setting flag
    
    * flag in storage struct
    
    * fix flagging to access and insert.
    
    * added todo to fix
    
    * also missing serialize meta to storage proof
    
    * extract meta.
    
    * Isolate old trie layout.
    
    * failing test that requires storing in meta when old hash scheme is used.
    
    * old hash compatibility
    
    * Db migrate.
    
    * runing tests with both states when interesting.
    
    * fix chain spec test with serde default.
    
    * export state (missing trie function).
    
    * Pending using new branch, lacking genericity on layout resolution.
    
    * extract and set global meta
    
    * Update to branch 4
    
    * fix iterator with root flag (no longer insert node).
    
    * fix trie root hashing of root
    
    * complete basic backend.
    
    * Remove old_hash meta from proof that do not use inner_hashing.
    
    * fix trie test for empty (force layout on empty deltas).
    
    * Root update fix.
    
    * debug on meta
    
    * Use trie key iteration that do not include value in proofs.
    
    * switch default test ext to use inner hash.
    
    * small integration test, and fix tx cache mgmt in ext.
    test  failing
    
    * Proof scenario at state-machine level.
    
    * trace for db upgrade
    
    * try different param
    
    * act more like iter_from.
    
    * Bigger batches.
    
    * Update trie dependency.
    
    * drafting codec changes and refact
    
    * before removing unused branch no value alt hashing.
    more work todo rename all flag var to alt_hash, and remove extrinsic
    replace by storage query at every storage_root call.
    
    * alt hashing only for branch with value.
    
    * fix trie tests
    
    * Hash of value include the encoded size.
    
    * removing fields(broken)
    
    * fix trie_stream to also include value length in inner hash.
    
    * triedbmut only using alt type if inner hashing.
    
    * trie_stream to also only use alt hashing type when actually alt hashing.
    
    * Refactor meta state, logic should work with change of trie treshold.
    
    * Remove NoMeta variant.
    
    * Remove state_hashed trigger specific functions.
    
    * pending switching to using threshold, new storage root api does not
    make much sense.
    
    * refactoring to use state from backend (not possible payload changes).
    
    * Applying from previous state
    
    * Remove default from storage, genesis need a special build.
    
    * rem empty space
    
    * Catch problem: when using triedb with default: we should not revert
    nodes: otherwhise thing as trie codec cannot decode-encode without
    changing state.
    
    * fix compilation
    
    * Right logic to avoid switch on reencode when default layout.
    
    * Clean up some todos
    
    * remove trie meta from root upstream
    
    * update upstream and fix benches.
    
    * split some long lines.
    
    * UPdate trie crate to work with new design.
    
    * Finish update to refactored upstream.
    
    * update to latest triedb changes.
    
    * Clean up.
    
    * fix executor test.
    
    * rust fmt from master.
    
    * rust format.
    
    * rustfmt
    
    * fix
    
    * start host function driven versioning
    
    * update state-machine part
    
    * still need access to state version from runtime
    
    * state hash in mem: wrong
    
    * direction likely correct, but passing call to code exec for genesis
    init seem awkward.
    
    * state version serialize in runtime, wrong approach, just initialize it
    with no threshold for core api < 4 seems more proper.
    
    * stateversion from runtime version (core api >= 4).
    
    * update trie, fix tests
    
    * unused import
    
    * clean some TODOs
    
    * Require RuntimeVersionOf for executor
    
    * use RuntimeVersionOf to resolve genesis state version.
    
    * update runtime version test
    
    * fix state-machine tests
    
    * TODO
    
    * Use runtime version from storage wasm with fast sync.
    
    * rustfmt
    
    * fmt
    
    * fix test
    
    * revert useless changes.
    
    * clean some unused changes
    
    * fmt
    
    * removing useless trait function.
    
    * remove remaining reference to state_hash
    
    * fix some imports
    
    * Follow chain state version management.
    
    * trie update, fix and constant threshold for trie layouts.
    
    * update deps
    
    * Update to latest trie pr changes.
    
    * fix benches
    
    * Verify proof requires right layout.
    
    * update trie_root
    
    * Update trie deps to  latest
    
    * Update to latest trie versioning
    
    * Removing patch
    
    * update lock
    
    * extrinsic for sc-service-test using layout v0.
    
    * Adding RuntimeVersionOf to CallExecutor works.
    
    * fmt
    
    * error when resolving version and no wasm in storage.
    
    * use existing utils to instantiate runtime code.
    
    * migration pallet
    
    * Patch to delay runtime switch.
    
    * Revert "Patch to delay runtime switch."
    
    This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.
    
    * fix test
    
    * fix child migration calls.
    
    * useless closure
    
    * remove remaining state_hash variables.
    
    * Fix and add more tests
    
    * Remove outdated comment
    
    * useless inner hash
    
    * fmt
    
    * remote tests
    
    * finally ksm works
    
    * batches are broken
    
    * clean the benchmarks
    
    * Apply suggestions from code review
    
    Co-authored-by: default avatarGuillaume Thiolliere <[email protected]>
    
    * Apply suggestions from code review
    
    Co-authored-by: default avatarGuillaume Thiolliere <[email protected]>
    
    * Update frame/state-trie-migration/src/lib.rs
    
    Co-authored-by: default avatarJoshy Orndorff <[email protected]>
    
    * Update frame/state-trie-migration/src/lib.rs
    
    * brand new version
    
    * fix build
    
    * Update frame/state-trie-migration/src/lib.rs
    
    Co-authored-by: default avatarGuillaume Thiolliere <[email protected]>
    
    * Update frame/state-trie-migration/src/lib.rs
    
    Co-authored-by: default avatarGuillaume Thiolliere <[email protected]>
    
    * Update primitives/storage/src/lib.rs
    
    Co-authored-by: default avatarcheme <[email protected]>
    
    * Update frame/state-trie-migration/src/lib.rs
    
    Co-authored-by: default avatarcheme <[email protected]>
    
    * Update frame/state-trie-migration/src/lib.rs
    
    Co-authored-by: default avatarcheme <[email protected]>
    
    * fmt and opt-in feature to apply state change.
    
    * feature gate core version, use new test feature for node and test node
    
    * Use a 'State' api version instead of Core one.
    
    * fix merge of test function
    
    * use blake macro.
    
    * Fix state api (require declaring the api in runtime).
    
    * Opt out feature, fix macro for io to select a given version
    instead of latest.
    
    * run test nodes on new state.
    
    * fix
    
    * new test structure
    
    * new testing stuff from emeric
    
    * Add commit_all, still not working
    
    * Fix all tests
    
    * add comment
    
    * we have PoV tracking baby
    
    * document stuff, but proof size is still wrong
    
    * FUCK YEAH
    
    * a big batch of review comments
    
    * add more tests
    
    * tweak test
    
    * update config
    
    * some remote-ext stuff
    
    * delete some of the old stuff
    
    * sync more files with master to minimize the diff
    
    * Fix all tests
    
    * make signed migration a bit more relaxed
    
    * add witness check to signed submissions
    
    * allow custom migration to also go above limit
    
    * Fix these pesky tests
    
    * ==== removal of the unsigned stuff ====
    
    * Make all tests work again
    
    * separate the tests from the logic so it can be reused easier
    
    * fix overall build
    
    * Update frame/state-trie-migration/src/lib.rs
    
    Co-authored-by: default avatarcheme <[email protected]>
    
    * Update frame/state-trie-migration/src/lib.rs
    
    Co-authored-by: default avatarcheme <[email protected]>
    
    * Slightly better termination
    
    * some final tweaks
    
    * Fix tests
    
    * Restrict access to signed migrations
    
    * address most of the review comments
    
    * fix defensive
    
    * New simplified code
    
    * Fix weights
    
    * fmt
    
    * Update frame/state-trie-migration/src/lib.rs
    
    Co-authored-by: default avatarBastian Köcher <[email protected]>
    
    * make the tests correctly fail
    
    * Fix build
    
    * Fix build
    
    * try and fix the benchmarks
    
    * fix build
    
    * Fix cargo file
    
    * Fix runtime deposit
    
    * make rustdoc happy
    
    * cargo run --quiet --profile=production  --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_state_trie_migration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/state-trie-migration/src/weights.rs --template=./.maintain/frame-weight-template.hbs
    
    Co-authored-by: default avatarcheme <[email protected]>
    Co-authored-by: default avatarGuillaume Thiolliere <[email protected]>
    Co-authored-by: default avatarJoshy Orndorff <[email protected]>
    Co-authored-by: default avatarBastian Köcher <[email protected]>
    Co-authored-by: default avatarParity Bot <[email protected]>
    6cfdbe5c