Companion for Store voters in unsorted bags to get good stake iteration properties (#3413)

* add voter bags generation script and generated values for all runtimes

* VoterBagThresholds for test-runtime

* semicolon

* turns out the header is called file_header.txt

* perform benchmarks to get real numbers

* add note about when to re-run this script

* Suggestions for #3413 (Store voters in unsorted bags companion) (#3488)

Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: radupopa2010 <[email protected]>
Co-authored-by: CI system <>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: thiolliere <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Bernhard Schuster <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Robert Klotzner <[email protected]>
Co-authored-by: Alexander Theißen <[email protected]>
Co-authored-by: Lldenaurois <[email protected]>
Co-authored-by: Qinxuan Chen <[email protected]>
Co-authored-by: Seun Lanlege <[email protected]>
Co-authored-by: Denis Pisarev <[email protected]>
Co-authored-by: Andreas Doerr <[email protected]>
Co-authored-by: Chevdor <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Denis Tsai <[email protected]>
Co-authored-by: Pierre Besson <[email protected]>
Co-authored-by: kianenigma <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Xiliang Chen <[email protected]>
Co-authored-by: Hernando Castano <[email protected]>
Co-authored-by: ferrell-code <[email protected]>
Co-authored-by: Shaun W <[email protected]>

* Try resolve Cargo.lock conflicts

* Add temp VoterSnapshotPerBlock value to compile

* Improve voter_bags test with nominator/validator breakdown

* Remove miner.log

* Update README

* Module doc comments

* Build working

* remote-ext-tests very  WIP

* Remove system module query

* formatting and log target

* clean up

* Save

* Fix issues with polkadot runtime after mergin

* Save

* Some updates to runtimes to make sure things compile

* save

* udpate weights

* Use UseNominatorsAndUpdateBagsList

* make remote-ext tests a bin

* Adjust remote-ext test to use pub; update some comments

* Use list_bags_get

* Try fix cargo.lock

* Gate kusama feat code

* Carg.lock .. again

* point bags-list deps to master

* Update voter-bags  CLI to take total issuance and min balance as args

* Switch polkadot runtime to UseNominatorMap

* Some tweaks to the remote-test

* Delete some stuff for polkadot; comments

* Various fixes

* Add comment for VoterSnapshotPerBlock

* Remove bags-list import from polakdot cargo

* use benchmarking feature for pallet-bags-list

* Add custom migration for westend and kusama for bags-list

* :facepalm

* Fix kusama runtime

* Small clean up

* Be more alphabetical ordered

* Use 22_500 for VoterSnapshotPerBlock

* point back to master

* remove unused type

* SortedListProvider for staking runtime

* Remove polkadot for remote-ext bags test

* Revert "Remove polkadot for remote-ext bags test"

This reverts commit 3608c466002063761cd669880a00b2e05c1ff7d1.

* Remove polkadot for remote-ext bags test

* update Substrate

Co-authored-by: Zeke Mostov <[email protected]>
Co-authored-by: Andronik Ordian <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: radupopa2010 <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: thiolliere <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Bernhard Schuster <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Robert Habermeier <[email protected]>
Co-authored-by: Robert Klotzner <[email protected]>
Co-authored-by: Alexander Theißen <[email protected]>
Co-authored-by: Lldenaurois <[email protected]>
Co-authored-by: Qinxuan Chen <[email protected]>
Co-authored-by: Seun Lanlege <[email protected]>
Co-authored-by: Denis Pisarev <[email protected]>
Co-authored-by: Andreas Doerr <[email protected]>
Co-authored-by: Chevdor <[email protected]>
Co-authored-by: Tomasz Drwięga <[email protected]>
Co-authored-by: Denis Tsai <[email protected]>
Co-authored-by: Pierre Besson <[email protected]>
Co-authored-by: kianenigma <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Xiliang Chen <[email protected]>
Co-authored-by: Hernando Castano <[email protected]>
Co-authored-by: ferrell-code <[email protected]>
Co-authored-by: Shaun W <[email protected]>
Co-authored-by: parity-processbot <>