- May 03, 2021
-
-
Peter Goodspeed-Niklaus authored
* Refactor election solution trimming for efficiency The previous version always trimmed the `CompactOf<T>` instance, which was intrinsically inefficient: that's a packed data structure, which is naturally expensive to edit. It's much easier to edit the unpacked data structures: the `voters` and `assignments` lists. * rework length-trim tests to work with the new interface Test suite now compiles. Tests still don't pass because the macro generating the compact structure still generates `unimplemented!()` for the actual `compact_length_of` implementation. * simplify * add a fuzzer which can validate `Compact::encoded_size_for` The `Compact` solution type is generated distinctly for each runtime, and has both three type parameters and a built-in limit to the number of candidates that each voter can vote for. Finally, they have an optional `#[compact]` attribute which changes the encoding behavior. The assignment truncation algorithm we're using depends on the ability to efficiently and accurately determine how much space a `Compact` solution will take once encoded. Together, these two facts imply that simple unit tests are not sufficient to validate the behavior of `Compact::encoded_size_for`. This commit adds such a fuzzer. It is designed such that it is possible to add a new fuzzer to the family by simply adjusting the `generate_solution_type` macro invocation as desired, and making a few minor documentation edits. Of course, the fuzzer still fails for now: the generated implementation for `encoded_size_for` is still `unimplemented!()`. However, once the macro is updated appropriately, this fuzzer family should allow us to gain confidence in the correctness of the generated code. * Revert "add a fuzzer which can validate `Compact::encoded_size_for`" This reverts commit 916038790887e64217c6a46e9a6d281386762bfb. The design of `Compact::encoded_size_for` is flawed. When `#[compact]` mode is enabled, every integer in the dataset is encoded using run- length encoding. This means that it is impossible to compute the final length faster than actually encoding the data structure, because the encoded length of every field varies with the actual value stored. Given that we won't be adding that method to the trait, we won't be needing a fuzzer to validate its performance. * revert changes to `trait CompactSolution` If `CompactSolution::encoded_size_for` can't be implemented in the way that we wanted, there's no point in adding it. * WIP: restructure trim_assignments_length by actually encoding This is not as efficient as what we'd hoped for, but it should still be better than what it's replacing. Overall efficiency of `fn trim_assignments_length` is now `O(edges * lg assignments.len())`. * fix compiler errors * don't sort voters, just assignments Sorting the `voters` list causes lots of problems; an invariant that we need to maintain is that an index into the voters list has a stable meaning. Luckily, it turns out that there is no need for the assignments list to correspond to the voters list. That isn't an invariant, though previously I'd thought that it was. This simplifies things; we can just leave the voters list alone, and sort the assignments list the way that is convenient. * WIP: add `IndexAssignment` type to speed up repeatedly creating `Compact` Next up: `impl<'a, T> From<&'a [IndexAssignmentOf<T>]> for Compact`, in the proc-macro which makes `Compact`. Should be a pretty straightforward adaptation of `from_assignment`. * Add IndexAssignment and conversion method to CompactSolution This involves a bit of duplication of types from `election-provider-multi-phase`; we'll clean those up shortly. I'm not entirely happy that we had to add a `from_index_assignments` method to `CompactSolution`, but we couldn't define `trait CompactSolution: TryFrom<&'a [Self::IndexAssignment]` because that made trait lookup recursive, and I didn't want to propagate `CompactSolutionOf<T> + TryFrom<&[IndexAssignmentOf<T>]>` everywhere that compact solutions are specified. * use `CompactSolution::from_index_assignment` and clean up dead code * get rid of `from_index_assignments` in favor of `TryFrom` * cause `pallet-election-provider-multi-phase` tests to compile successfully Mostly that's just updating the various test functions to keep track of refactorings elsewhere, though in a few places we needed to refactor some test-only helpers as well. * fix infinite binary search loop Turns out that moving `low` and `high` into an averager function is a bad idea, because the averager gets copies of those values, which of course are never updated. Can't use mutable references, because we want to read them elsewhere in the code. Just compute the average directly; life is better that way. * fix a test failure * fix the rest of test failures * remove unguarded subtraction * fix npos-elections tests compilation * ensure we use sp_std::vec::Vec in assignments * add IndexAssignmentOf to sp_npos_elections * move miner types to `unsigned` * use stable sort * rewrap some long comments * use existing cache instead of building a dedicated stake map * generalize the TryFrom bound on CompactSolution * undo adding sp-core dependency * consume assignments to produce index_assignments * Add a test of Assignment -> IndexAssignment -> Compact * fix `IndexAssignmentOf` doc * move compact test from sp-npos-elections-compact to sp-npos-elections This means that we can put the mocking parts of that into a proper mock package, put the test into a test package among other tests. Having the mocking parts in a mock package enables us to create a benchmark (which is treated as a separate crate) import them. * rename assignments -> sorted_assignments * sort after reducing to avoid potential re-sort issues * add runtime benchmark, fix critical binary search error "Why don't you add a benchmark?", he said. "It'll be good practice, and can help demonstrate that this isn't blowing up the runtime." He was absolutely right. The biggest discovery is that adding a parametric benchmark means that you get a bunch of new test cases, for free. This is excellent, because those test cases uncovered a binary search bug. Fixing that simplified that part of the code nicely. The other nice thing you get from a parametric benchmark is data about what each parameter does. In this case, `f` is the size factor: what percent of the votes (by size) should be removed. 0 means that we should keep everything, 95 means that we should trim down to 5% of original size or less. ``` Median Slopes Analysis ======== -- Extrinsic Time -- Model: Time ~= 3846 + v 0.015 + t 0 + a 0.192 + d 0 + f 0 µs Min Squares Analysis ======== -- Extrinsic Time -- Data points distribution: v t a d f mean µs sigma µs % <snip> 6000 1600 3000 800 0 4385 75.87 1.7% 6000 1600 3000 800 9 4089 46.28 1.1% 6000 1600 3000 800 18 3793 36.45 0.9% 6000 1600 3000 800 27 3365 41.13 1.2% 6000 1600 3000 800 36 3096 7.498 0.2% 6000 1600 3000 800 45 2774 17.96 0.6% 6000 1600 3000 800 54 2057 37.94 1.8% 6000 1600 3000 800 63 1885 2.515 0.1% 6000 1600 3000 800 72 1591 3.203 0.2% 6000 1600 3000 800 81 1219 25.72 2.1% 6000 1600 3000 800 90 859 5.295 0.6% 6000 1600 3000 800 95 684.6 2.969 0.4% Quality and confidence: param error v 0.008 t 0.029 a 0.008 d 0.044 f 0.185 Model: Time ~= 3957 + v 0.009 + t 0 + a 0.185 + d 0 + f 0 µs ``` What's nice about this is the clear negative correlation between amount removed and total time. The more we remove, the less total time things take.
-
- Apr 23, 2021
-
-
François Garillot authored
* Simplify some Option / Result / ? operator patterns When those match a combinator exactly. Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust). * adjust after review * adjust post-review
-
Bastian Köcher authored
* Use host max log level when initializing the `RuntimeLogger` This should fix performance problems introduced by logging under certain circumstances. Before we always called into the host and the host was doing the log filtering, now as the correct max log level is set, we don't call into the host for every log line to check if it should be logged. However, we would still call into the host to determine if something should be logged when `something=trace` is given as we don't forward the log targets that are enabled. * Finish the pr
-
- Apr 22, 2021
-
-
Xiliang Chen authored
* impl Zero and One for fixed point numbers * update librocksdb-sys to support M1 * fix build * trigger CI
-
- Apr 18, 2021
-
-
Pierre Krieger authored* Update to libp2p 0.37 * Line widths * Fix tests
-
- Apr 17, 2021
-
-
thiolliere authored
-
- Apr 16, 2021
-
-
Bastian Köcher authored
* Remove `serde` requirement from FRAME macros Currently there is some implicit requirement on `serde` being present in the `Cargo.toml` of a pallet when `GenesisConfig` is used. This pr removes this requirement by using the serde attribute `serde(crate = "..")`. * build a unique reexport of serde in impl_opaque_keys, by abusing paste doc concatenation * Optimize Co-authored-by:thiolliere <[email protected]>
-
- Apr 14, 2021
-
-
Bastian Köcher authored
* Introduce a "dynamic" block size limit for proposing This adds support for using a dynamic block size limit per call to `propose`. This is required for Cumulus/Parachains to always use stay in the limits of the maximum allowed PoV size. As described in the docs, the block limit is only checked in the process of pushing transactions. As we normally do some other operations in `on_finalize`, it can happen that the block size still grows when there is some proof being collected (as we do for parachains). This means, that the given block limit needs to be rather conservative on the actual value and should not be the upper limit. * Update client/basic-authorship/src/basic_authorship.rs Co-authored-by:
Andronik Ordian <[email protected]> * More future proof encoded size updating * Use `ProofRecorderInner` * Update client/basic-authorship/src/basic_authorship.rs Co-authored-by:
André Silva <[email protected]> * Update client/basic-authorship/src/basic_authorship.rs Co-authored-by:
-
Vladislav authored
* Add Sora network SS58 Address Signed-off-by:
Vladislav Markushin <[email protected]> * Update Sora network SS58 Address to `69` Signed-off-by:
-
- Apr 13, 2021
-
-
Alexander Theißen authored
* contracts: Add RPC that allows instantiating of a contract * Encode `debug_message` as bytes because usage of `String` is forbidden * Remove erroneous derive attribute * Fix rpc tests for new `debug_message` encoding * Fix typo Co-authored-by:
Andrew Jones <[email protected]> Co-authored-by:
-
thiolliere authored
* impl * fix tests * impl in execute_block * fix tests * add a test in frame-executive * fix some panic warning * use trait to get call from extrinsic * remove unused * fix test * fix testing * fix tests * return index of extrinsic on error * fix test * Update primitives/inherents/src/lib.rs Co-authored-by:
Kian Paimani <[email protected]> * address comments rename trait, and refactor * refactor + doc improvment * fix tests Co-authored-by:
-
- Apr 09, 2021
-
-
Kian Paimani authored
-
Xavier Lau authored
* `ModuleId` to `PalletId` - part of #8372 * fix doc * move `PalletId` to `frame-support` * fix compile * fix tests * `ModuleId` to `PalletId` * subcommand `moduleid` to `palletid`
-
- Apr 08, 2021
-
-
Shawn Tabrizi authored
* Added a function to estimate proof size for benchmarking * integrate proof_size into benchmarking pipeline * Update client/db/src/bench.rs Co-authored-by:
Bastian Köcher <[email protected]> * Update client/db/src/bench.rs Co-authored-by:
Alexander Popiak <[email protected]> * Update utils/frame/benchmarking-cli/src/command.rs Co-authored-by:
arkpar <[email protected]> Co-authored-by:
-
- Apr 07, 2021
-
-
asynchronous rob authored
* begin maybe-compressed-blob * fix build * implement blob compression / decompression * add some tests * decode -> decompress * decompress code if compressed * make API of compresseed blob crate take limit as parameter * use new API in sc-executro * wasm-builder: compress wasm * fix typo * simplify * address review * fix wasm_project.rs * Update primitives/maybe-compressed-blob/Cargo.toml Co-authored-by:
-
- Apr 05, 2021
-
-
Liu-Cheng Xu authored
* Add some trivial improvements * Finish primitives/runtime
-
- Apr 04, 2021
-
-
Bastian Köcher authored
To make the life of people easier ;)
-
- Mar 30, 2021
-
-
Bastian Köcher authored
* Make grandpa work * Introduce `SharedData` * Add test and fix bugs * Switch to `SharedData` * Make grandpa tests working * More Babe work * Make it async * Fix fix * Use `async_trait` in sc-consensus-slots This makes the code a little bit easier to read and also expresses that there can always only be one call at a time to `on_slot`. * Make grandpa tests compile * More Babe tests work * Fix network test * Start fixing service test * Finish service-test * Fix sc-consensus-aura * Fix fix fix * More fixes * Make everything compile *yeah* * Fix build when we have Rust 1.51 * Update client/consensus/common/src/shared_data.rs Co-authored-by:
-
- Mar 29, 2021
-
-
Sukhveer Sanghera authored
* Add Social Network * Remove TNET * Update ss58-registry.json * Update ss58-registry.json * Update ss58-registry.json * Add back comment Co-authored-by:
everhusk <[email protected]> Co-authored-by:
Shawn Tabrizi <[email protected]> Co-authored-by:
Github Actions <[email protected]>
-
- Mar 28, 2021
-
-
Peter Goodspeed-Niklaus authored
* make types within `generate_solution_type` macro explicit Closes #8444. Just changes the parsing logic for that macro; does not change any emitted code. The associated types associated with the macro now require explicit, keyword-style declaration. **Old**: ```rust sp_npos_elections::generate_solution_type!( #[compact] pub struct TestCompact::<VoterIndex, TargetIndex, PerU16>(16) ); ``` **New**: ```rust sp_npos_elections::generate_solution_type!( #[compact] pub struct TestCompact::<VoterIndex = VoterIndex, CandidateIndex = TargetIndex, Accuracy = PerU16>(16) ); ``` * un-ignore doc-tests * use new form in bin/node/runtime/ * rename CandidateIndex -> TargetIndex * add tests demonstrating some potential compile failures
-
- Mar 27, 2021
-
-
Gavin Wood authored
* Reservable, Transferrable Fungible(s), plus adapters. * Repot into new dir * Imbalances for Fungibles * Repot and balanced fungible. * Clean up names and bridge-over Imbalanced. * Repot frame_support::trait. Finally. * Make build. * Docs * Good errors * Fix tests. Implement fungible::Inspect for Balances. * Implement additional traits for Balances. * Revert UI test "fixes" * Fix UI error * Fix UI test * Fixes * Update lock * Grumbles * Grumbles * Fixes Co-authored-by:Bastian Köcher <[email protected]>
-
- Mar 24, 2021
-
-
Roman Borschel authored
* Update to libp2p-0.36 * Some more Cargo.lock updates.
-
- Mar 21, 2021
-
-
Shaun Wang authored
-
- Mar 18, 2021
-
-
Arkadiy Paronyan authored
* Transaction indexing * Tests and fixes * Fixed a comment * Style * Build * Style * Apply suggestions from code review Co-authored-by:
cheme <[email protected]> * Code review suggestions * Add missing impl * Apply suggestions from code review Co-authored-by:
-
Shaun Wang authored
* Use 'Pallet' struct in construct_runtime. * Fix genesis and metadata macro. * Fix 'Pallet' type alias. * Replace 'Module' with 'Pallet' for all construct_runtime use cases. * Replace more deprecated 'Module' struct. * Bring back AllModules and AllPalletsWithSystem type, but deprecate them. * Replace deprecated 'Module' struct from merge master. * Minor fix. * Fix UI tests. * Revert UI override in derive_no_bound. * Fix more deprecated 'Module' use from master branch. * Fix more deprecated 'Module' use from master branch.
-
- Mar 17, 2021
-
-
Bastian Köcher authored
* Make slot duration being exposed as `Duration` to the outside * Some slot info love * Add `build_aura_worker` utility function * Copy copy copy
-
Jon Häggblad authored
* primitives/runtime: initial changes on supporting multiple Justifications * primitives/runtime: make Justifications strongly typed * Encode/decode Justifications * primitives/runtime: add Justification type * backend: apply_finality and finalize_block takes a single Justification * manual-seal: create engine id and let rpc take encoded justification * backend: skeleton functions for appending justifications * backend: initial implementation append_justification Initial implementation of append_justification on the Backend trait, and also remove unused skeleton functions for append_justificaton on Finaziler trait. k * backend: guard against duplicate consensus engine id * client/db: add check for block finality * client/api: add append_justification to in_mem db * client/light: add no-op append_justification * network: fix decode call for Justification * network: only send a single Justification in BlockData * network: minor comment update * protocol: update field names to distinguish single justification * client: further field renames to plural * client: update function names to plural justifications * client/db: upgrade existing database for new format * network: remove dependency on grandpa crate * db: fix check for finalized block * grandpa: check for multiple grandpa justifications hwne importing * backend: update Finalizer trait to take multiple Justifications * db: remove debugging statements in migration code * manual-seal: update note about engine id * db: fix check for finalized block * client: update variable name to reflect it is now plural * grandpa: fix incorrect empty Justications in test * primitives: make Justifications opaque to avoid being empty * network: fix detecting empty Justification * runtime: doc strings for Justifications functions * runtime: add into_justifications * primitives: check for duplicates in when adding to Justifications * network/test: use real grandpa engine id in test * client: fix reviewer comments * primitives: rename Justifications::push to append * backend: revert changes to Finalizer trait * backend: revert mark_finalized * backend: revert changes to finalize_block * backend: revert finalized_blocks * db: add a quick early return for performance * client: minor reviewer comments * service/test: use local ConsensusEngineId * network: add link to issue for sending multiple Justifications * Apply suggestions from code review Co-authored-by:
Pierre Krieger <[email protected]> * Apply suggestions from code review Co-authored-by:
André Silva <[email protected]>
-
Peter Goodspeed-Niklaus authored
* Improve complexity of CompactAssignments::unique_targets Original implementation was O(n**2). Current impl is O(n log n). Avoided the original proposed mitigation because it does not retain the de-duplicating property present in the original implementation. This implementation does a little more work, but retains that property. * Explicitly choose sp_std Vec and BTreeSet Ensures that the macro still works if someone uses it in a context in which sp_std is not imported or is renamed. * explicitly use sp_std vectors throughout compact macro
-
- Mar 16, 2021
-
-
Gavin Wood authored
* Initial abstraction * Alter rest of APIs * Fixes * Some extra getters in Gilt pallet. * Refactor Gilt to avoid u128 conversions * Simplify and improve pow in per_things * Add scalar division to per_things * Renaming from_fraction -> from_float, drop _approximation * Fixes * Fixes * Fixes * Fixes * Make stuff build * Fixes * Fixes * Fixes * Fixes * Update .gitignore Co-authored-by:
-
Kian Paimani authored
* Base features and traits. * pallet and unsigned phase * Undo bad formattings. * some formatting cleanup. * Small self-cleanup. * Make it all build * self-review * Some doc tests. * Some changes from other PR * Fix session test * Update Cargo.lock * Update frame/election-provider-multi-phase/src/lib.rs Co-authored-by:
Peter Goodspeed-Niklaus <[email protected]> * nit * better doc * Line width * Fix build * Self-review * Self-review * Fix wan * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * fix build and review comments. * Update frame/election-provider-multi-phase/src/lib.rs Co-authored-by:
Parity Benchmarking Bot <[email protected]> Co-authored-by:
-
Peter Goodspeed-Niklaus authored
* Add PJR challenge functions - Updates the PJR check to return a counterexample if one exists - Adds functions to cheaply check counterexamples This is in support of off-chain PJR challenges: if a miner discovers that an accepted election solution does not satisfy PJR, it will be eligible for substantial rewards. This helps ensure that validator elections have an absolute quality floor, so even if someone manages to censor well-behaved solutions to give themselves unfair representation, we can catch them in the act and penalize them. * counterexample -> counter_example * reorganize: high -> low abstraction * reorganize challenges high -> low abstraction * add note justifying linear search * Simplify max_pre_score validation Co-authored-by:
-
- Mar 15, 2021
-
-
Qinxuan Chen authored
Signed-off-by:koushiro <[email protected]>
-
- Mar 11, 2021
-
-
Bastian Köcher authored
The timestamp inherent type was up to now just a simple `u64`. This worked, but doesn't give you that much guarantees at compile time about the type. This pr changes that by converting this type to a unit type wrapper, similar to what we have done for `Slot`. This is required for some future pr that touches quite a lot of the inherents stuff :) Besides this unit wrapper type, this pr also moves the `OnTimestampSet` trait to `frame_support::traits`.
-
dependabot[bot] authored
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.22 to 0.1.25. - [Release notes](https://github.com/tokio-rs/tracing/releases) - [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.22...tracing-0.1.25 ) Signed-off-by:
dependabot[bot] <[email protected]> Co-authored-by:
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
-
Peter Goodspeed-Niklaus authored
* Apply. * get rid of glob import * use meaningful generic type name * pjr_check operates on `Supports` struct used elsewhere * improve algorithmic complexity of `prepare_pjr_input` * fix rustdoc warnings * improve module docs * typo * simplify debug assertion * add test finding the phase-change threshold value for a constructed scenario * add more threshold scenarios to disambiguate plausible interpretations * add link to npos paper reference * docs: staked_assignment -> supports Co-authored-by:
kianenigma <[email protected]> Co-authored-by:
-
- Mar 10, 2021
-
-
Tomasz Drwięga authored
* Add MMR custom RPC. * Change RuntimeApi to avoid hardcoding leaf type. * Properly implement the new RuntimeAPI and wire up RPC. * Extract Offchain DB as separate execution extension. * Enable offchain DB access for offchain calls. * Fix offchain_election tests. * Skip block initialisation for proof generation. * Fix integration test setup. * Fix offchain tests. Not sure how I missed them earlier
🤷 . * Fix long line. * One more test missing. * Update mock for multi-phase. * Address review grumbbles. * Address review grumbles. * Fix line width of a comment -
Ashley authored
* Make changes * Add serialize/deserialize, copy babe epoch config defaults from node runtime * Fix line widths and turn default features off for serde * Remove ser/deser from Epoch, fix node-cli * Apply suggestions * Add comment to BABE_GENESIS_EPOCH_CONFIG in bin * Apply suggestions * Add a sketchy migration function * Add a migration test * Check for PendingEpochConfigChange as well * Make epoch_config in node-cli * Move updating EpochConfig out of the if * Fix executor tests * Calculate weight for add_epoch_configurations * Fix babe test * Apply suggestions from code review Co-authored-by:
-
- Mar 09, 2021
-
-
Bastian Köcher authored
-
- Mar 08, 2021
-
-
honeywest authored
* optimize code * fix clippy replace = with += or %= * fix redundant closure found warning * redundant field names in struct initialization * fix clippy warning and optimize code * fix clippy warning * fix clippy warning * fix test error
-
- Mar 05, 2021
-
-
thiolliere authored
* implement * make default pre/post_upgrade * simplify Cargo.toml * revert removal of outdated/private links * link in pallet-mmr
-
![dependabot[bot]'s avatar](https://secure.gravatar.com/avatar/bd5a8d6c673b738d52b0ac42a110045f3f964b3ebfc1d60ea805af743b1dc0e6?s=80&d=identicon "dependabot[bot]")
